Skip to main content

From Parquet to GPU memory

Parquet scans identify column byte ranges, stage buffers, and decode data into GPU VRAM using cuDF and KvikIO. Host staging buffers remain in system RAM.

Projection and scan pruning​

Parquet projection and pruningIllustrative Parquet scan after file selection. Each grid cell is a column chunk in a row group. Projection excludes notes; footer-based row-group pruning excludes all of RG 1. Retained compressed pages pass through KvikIO into GPU memory, then cuDF decodes them into columns.Selected Parquet filemetadatacuDF decodecustomeramountstatusnotesskipskipskipskipRG 2RG 2RG 2skipRG 3RG 3RG 3skipProjection skips notes; statistics skip RG 1.Footer metadataSchema, offsets, statisticsProject and pruneColumns · row groupsRead selected pagesKvikIO → GPU byte buffersDecoded columnsVRAM buffers for GPU operators
In this single-file example, projection omits the notes column and metadata pruning skips all of RG 1. The retained row groups still require predicate evaluation.

Scans prune unneeded data across multiple granularities:

UnitHow the scan is reduced
FilesNative lowering preserves the file set supplied by DataFusion's scan, including upstream partition pruning. An Iceberg table reaches lowering as a standard s3:// Parquet scan, and Iceberg manifest pruning has already reduced its file set.
Column chunksProjection keeps only columns referenced by downstream operators and predicates.
Row groupsLowering evaluates footer statistics (min/max and null counts) to prune row groups for supported predicates. Bloom filters apply only inside the cuDF reader, to a predicate pushed into it. An s3:// scan prunes at planning time only when the session holds its footers.
RowsPredicates evaluate within the cuDF reader or a downstream GPU filter, reducing output rows.

When a predicate is pushed into the cuDF reader, its statistics and Bloom filters reduce row groups before decompression. Footer pruning of s3:// sources before admission is an embedded-session setting, AlgeonNativeExecutionConfig::with_lakehouse_footer_pruning; Flight SQL server sessions do not expose object-store pruning options.

Source-read observations report pruning_input_row_groups, row_groups_after_stats_filter and row_groups_after_bloom_filter when the reader supplies pruning evidence. Compare these counts with output_rows to distinguish metadata pruning from row filtering.

Row groups, source chunks and I/O tasks​

UnitDefined bySize describes
Parquet row groupThe file writerA set of rows stored as separate column chunks, with footer metadata
Source chunkAlgeon's source plannerSelected row groups from one file, grouped against a decoded-byte target, a row-group cap and the backend's per-column element limit
KvikIO I/O taskThe I/O layerA range of stored bytes to read, bounded by KVIKIO_TASK_SIZE

The planner can combine several row groups into one source chunk. Compatible source chunks can then be batched into a reader call. The reader selects the column byte ranges, and KvikIO divides reads into I/O tasks on its shared pool. One source chunk can therefore require many I/O tasks. The I/O task size controls the read subdivision; the source-chunk target still comes from the query's memory policy. Throughput grouping also widens chunks of s3:// files whose footers chunk planning reads, the same footers that size the chunks. A chunk never spans files.

Every planned read, and every batch of chunks read together, decodes into one cuDF table. Its row count and the child value count of each list column stay within the 2,147,483,647 elements one cuDF column indexes, counted from the footer's per-leaf num_values. A row group that alone exceeds that count fails planning with size_overflow and the row group size that fits.

The cuDF reader needs the footer of each file it reads. Within one query attempt, the first read of part of a file parses that file's footer, and later reads of the file, retries included, reuse the parsed copy. Each read gives the reader a copy holding only its own row groups. A read that covers a whole file is that file's only read in its source and parses the footer for itself alone. Parsed footers occupy host memory outside the query's device grant until the attempt ends. A parsed footer stays current while the file keeps the size and modification time it had when the footer was parsed; a rewrite that keeps both is not detected.

Source sizing uses estimates of decoded columns because dictionary-encoded strings can expand substantially during decoding. The native settings define throughput grouping, byte targets and their overrides. Follow chunk execution to see how a query's grant constrains that planning.

The transport depends on the source​

Storage payload routesPayload routes for a PCIe-attached GPU. Default local compatibility path: NVMe file, POSIX read to pinned host bounce buffer, PCIe copy to VRAM. Conditional GDS path: storage DMA through cuFile to GPU memory. S3 path: HTTP range request through libcurl host buffers, copy to pinned host staging, PCIe copy to VRAM. cuObject path, drawn dashed and not yet supported by Algeon: an object server with cuObject writes the payload by RDMA over InfiniBand or RoCE into a registered VRAM buffer. Each route occupies one horizontal row.Local file · default compatibility pathLocal file · conditional GPUDirect Storage pathS3 · HTTP range readsS3 · cuObject RDMA · not yet supportedpreadPCIe copyStorage DMAGDS / cuFile over PCIe · supported storage and configurationHTTPhost copyPCIe copyRDMA writecuObject over InfiniBand / RoCENVMe fileCompressed pagesPinned host bufferReusable bounce bufferGPU VRAMFor decodingNVMe fileSupported storageGPU VRAMFor decodingS3 objectSelected rangeslibcurl buffersHost memoryPinned stagingBatched curl chunksGPU VRAMFor decodingS3 objectcuObject serverGPU VRAMFor decoding
The diagram shows a PCIe-attached GPU. Payload ends in VRAM; metadata and I/O submission remain on the CPU. The GDS row requires a supported and configured direct path. The dashed cuObject row shows a route Algeon does not support yet.

Local NVMe and compatibility mode​

Local reads use KvikIO compatibility mode. KvikIO issues POSIX reads such as pread into a pinned host bounce buffer, reused across chunks, then copies the bytes to device memory. That copy crosses the host/device link (PCIe in the diagram, NVLink-C2C on some CPU/GPU systems) into VRAM, and selecting fewer column pages reduces both storage reads and that traffic.

Buffered POSIX reads can pass through the operating system's page cache. Where enabled and supported, O_DIRECT bypasses that cache. The host staging and GPU copy still belong to this compatibility path. Larger range reads amortize syscall and submission overhead. Source reads run as SupervisedWorker jobs on the blocking lane of the GPU execution runtime, each backed by an attempt auxiliary permit. A read issued from an execution lane executes inline on that lane. When auxiliary permits are unavailable, a required read executes inline on the coordinator while prefetch pauses. Source read tasks do not execute on the host runtime. A max_source_reads_in_flight value above 1 enables prefetch.

KvikIO's settings are global to the process. Algeon's cuDF component applies them once, while AlgeonGpuBackend::build() builds the backend and before any source read runs. The server, library embedders and the benchmark binaries all build their backend through that call. The defaults are compatibility mode ON (also libcudf's default), 64 I/O threads and a 4 MiB task size. KvikIO's environment variables KVIKIO_COMPAT_MODE, KVIKIO_NTHREADS and KVIKIO_TASK_SIZE override their fields; a field whose variable is unset keeps the Algeon default. At startup the server logs one source I/O ready line, and its kvikio_compat_mode, kvikio_threads and kvikio_task_size_bytes fields give the effective values.

KvikIO splits a local read into tasks no larger than the task size and runs them on its thread pool. A process has one such pool. Local pread tasks and s3:// range reads share it, as do all devices of a multi-GPU backend. Each compatibility-mode task borrows a 16 MiB pinned bounce buffer from a separate KvikIO buffer pool. With all threads busy, that buffer pool holds up to 16 MiB per thread, or 1 GiB at the default 64 threads. KvikIO keeps the buffers for later reads. The server releases them at shutdown, after the last source read ends; a library process keeps them until it calls cudf::io::drain_kvikio_bounce_buffer_pools or exits.

GPUDirect Storage​

With a suitable filesystem, driver and device configuration, KvikIO can use cuFile and GPUDirect Storage (GDS). Storage hardware transfers the payload to GPU memory through DMA, direct memory access, avoiding the host payload bounce buffer. CPU code still opens files and submits requests.

AUTO permits a POSIX fallback, and cuFile also has its own compatibility mode. Small or unaligned requests may use alternate paths. An enabled setting alone therefore does not establish that a read used direct DMA. See KvikIO's mode controls and the GDS overview.

Object storage and libcurl​

Algeon's native S3 datasource uses KvikIO HTTP range reads executed on the shared thread pool. libcurl receives chunks into host memory; KvikIO stages them in pinned memory before copying to device VRAM. Direct storage-to-VRAM transfers via cuObject RDMA are not supported. Footer reads and cache fills for s3:// sources run on a process-wide algeon-object-store-control Tokio runtime (Runtimes and threads).

A physical optimizer rule cannot wait on the network, so planning never reads a remote footer from the store. It reads the footer of an s3:// object from the session's DataFusion file metadata cache, which DataFusion's ListingTable fills during its async scan when datafusion.execution.collect_statistics is true (the default). Planning reaches that cache through its invocation, so it exists for AlgeonAttempt planning, Flight SQL server statements, plan_with_native_diagnostics and validate_query. A plain ctx.sql(...) on a session from install_on plans without an invocation and has no remote footers. DataFusion keys the cache by the object path inside the store, without the bucket, and validates an entry by size and last-modified time. Algeon also refuses an entry whose e_tag or version differs from the scanned object's when both have one. A scan of several remote objects with a missing or refused footer is rejected with native_source_remote_footer_unavailable (remedies).

KvikIO's remote transfer implementation reflects this host staging and device copy sequence.

Caches along the path​

A first run of a query leaves state in four places, and each one covers a different part of the next read.

  • The OS page cache holds file pages from buffered pread. A hit skips the NVMe read; host staging, the device copy and decoding still run.
  • Planning looks up local and remote Parquet footers in the session's DataFusion file metadata cache, which outlives the query and is keyed by object path without the bucket (Object storage). DataFusion fills it while it infers a table's schema and statistics. A local footer missing there is parsed once per planning pass and put back for the session's next query; a remote one is unavailable. The cache is charged for decoded footers, so a footer larger than the limit is parsed again on every planning pass. The Flight SQL server limits it to 1 GiB by default (metadata_cache_limit_bytes). DataFusion's own default, 50 MB, applies to an embedded session until its RuntimeEnvBuilder sets with_metadata_cache_limit.
  • Decoded local-Parquet sources and constructed graphs can stay on the device under cache_cap_bytes (Admission settings). The default is 0, so queries share nothing on the GPU unless a server sets a cap. See Reuse in bounded servers and Completion and resident data.
  • The RMM memory resource pool and the KvikIO bounce buffer pool keep their allocations after a query. The next query reuses that memory and reads its own bytes into it.

The benchmarks page states which of these layers held data for each published run.

Source and workspace​

Local and S3 Parquet are read-only sources. SQL views belong to the mutable DataFusion workspace; creating a view does not rewrite a Parquet file or materialize it on the GPU.