Inside GPU execution
The engine runs a compiled schedule for each native fragment, coordinated from a worker thread in the device's GPU execution pool (Runtimes and threads). Operators execute GPU kernels on decoded columns in VRAM, retaining intermediate results until downstream consumers finish.
CUDA streams and dependencies
CudaStream identifies an ordered queue of GPU work. Work on separate
streams requires explicit completion dependencies before exchanging data.
The coordinator runs as a QueryExecution job on one pool worker, executing
root tasks on the attempt's root stream. Expanded chunk work (aggregate chunks,
join probes, and fused top-k candidates) executes on blocking-lane jobs. An
execution lane consists of one auxiliary permit, one leased CUDA stream, and
a derived context. Its width is bounded by source_work_per_attempt, available
permits, and pending work items; runs allocated fewer than two lanes execute
inline on the coordinator. At the default source_work_per_attempt = 1, chunk
work executes inline. The coordinator merges lane outputs in dispatch order
under the required SQL semantics; DataFusion CPU partitions do not dictate
lane allocation or stream width. A lane's stream lease remains held for the
entire run, while chunk jobs may execute on any blocking-lane thread.
Related terms distinguish lane scopes:
- An attempt lane (
lane_count, which equalsmax_active_attempts) is the identity slot an admitted attempt holds for its lifetime (Capacity and allocation). - An execution lane is the permit, stream, and context described above.
- Chunk lane work is a blocking-lane job executing on an execution lane.
Cross-stream dependencies use completion fences such as CUDA events. Buffer ownership lasts through final consumption, and all lanes allocate within one query memory cap. An aggregate's source chunks on separate lanes are live at the same time, so its chunk target divides the grant by the regions one chunk keeps live times the run's lane width; two lanes halve the target a serial run plans. Within composed island executions, shared execution scopes coordinate multi-reader lifecycles, retaining resident handles until the final reader completes.
Partitioned joins
A partitioned hash join splits its build side by key hash into one frame per partition and splits each probe piece the same way, so each pair holds only matching keys. When the probe side streams in, the join builds the hash table of each build partition at the first probe piece and probes it with every later piece. The attempt's kept tables stay within the share of its grant described in kept join tables. A partition outside that share builds one table per batch of probe input instead of one per piece. The join copies that partition's rows out of each probe piece and holds them. When the rows held by the attempt's joins would pass one partitioned-join target, the join concatenates the rows of its partition holding the most bytes, joins them against a table built for that batch, and drops the table. It repeats until the rest fits, and joins the rows still held when the probe input ends. A probe of a single piece keeps no table and holds no rows. A retry that repartitions a pair builds its own tables for the smaller pairs.
A build side that streams in chunk by chunk decides after each chunk whether to
partition. Plan row and byte estimates bound the build from above and may far
exceed it, for example after a LIKE filter over a large dimension. The build
commits to partitions once its counted rows or bytes reach the large-build
thresholds of KeyedHashJoinStrategyOptions. It also commits when its estimate
allows a large build and the whole chunks it holds reach the quarter of the
grant that also bounds kept join tables.
From then on each chunk is hash-partitioned as it arrives. A build that never
commits is concatenated, and its strategy is chosen from the counted rows and
bytes alone, where a large estimate no longer counts as large-build evidence.
A build of medium size whose probe is large still ends partitioned, after that
one concatenation.
The coordinator builds the kept tables on the root stream before probe chunks reach execution lanes. Lanes probe them concurrently on their own streams, which only read the table. A lane returns its rows of partitions without a kept table to the coordinator, which holds them and joins their batches on the root stream. The outputs of one probe piece's partitions, with the batches its rows pushed out, are concatenated into one frame. The next operator therefore receives at most one frame per probe piece, none for a piece whose rows were all held, and one for the batches joined at the end of the input. That concatenation holds the piece's output twice at its peak.
Runtime dynamic filters
The compiled executor derives dynamic filters from the producer's complete
join build or aggregate group-key state. max_build_rows bounds the producer
rows admitted to this work. Exact membership payloads retain distinct non-null
keys in the attempt's query memory domain and share them across consumers and
chunks. Empty and dense range payloads retain no key column. cuDF orders
membership reads against the retained producer stream and retains completion
dependencies until consumption ends.
Route discovery places a consumer only on a join input whose unmatched rows
the join discards. Inner and semi joins discard them on both inputs. A left
outer, anti or mark join discards them on its right input alone; each left row
reaches the output, or the mark column, whether it matches or not. A
null-aware anti join reads the nulls and the emptiness of its right input and
takes no route on either input. Only single-key joins under
NullsDoNotMatch produce a key route. Native execution builds the right input
of every keyed left outer and mark join. Build-side selection may move a keyed
semi or anti join's build to its left input, and the route then reduces the
right input. A join built on its left input keeps one match flag per left row:
each right chunk marks the left rows it matches and is released, and the
output is one gather of the marked rows for a semi join or the unmarked rows
for an anti join. The join retains no right chunk after marking it. A
null-aware anti join always builds its right input. A key route at a left
outer or mark join, or at an anti join that builds its right input, would
consume the left input and is rejected as UnsupportedJoinShape.
Integer bounds and distinct counts classify empty, dense range, small-IN-list and larger exact keysets without copying the key column to the host. Resident exact payload metadata carries the existing kind, count and range diagnostics; host predicate helpers cannot apply that payload. A source route requiring reader or row-group pruning downloads the distinct keys, retains the device keyset for post-decode membership, and preserves the source pruning policy. The large reader-only producer policy continues to derive a scalar range. Producer key limits and consumer type bounds follow native lowering contracts.
Relational data into GPU algorithms
SQL table functions integrate cuGraph, cuVS, and cuML within queries. cuDF columns supply input relations, and algorithm output columns feed downstream relational operators. GPU table functions details buffer ownership, zero-copy interop, and layout requirements across RAPIDS libraries.
Result egress and host synchronization
When a query fragment completes its relational and algorithmic stages,
execution reaches its terminal sink (ResultSink::HostArrow or a deferred host
batch cursor). The engine transfers the output column buffers from VRAM to host
memory, packaging them into Arrow RecordBatch instances for DataFusion.
The device-to-host copy is queued on the execution stream after the upstream kernels and starts once they complete.
The host runtime polls the DataFusion stream and awaits each pool job without
blocking a host worker. The first poll runs admission and the fragment's
schedule. Each chunk then takes one pool job, which sizes a row range of the
result to at most 4 MiB of Arrow buffers, reserves that estimate from
DataFusion's MemoryPool (one live reservation per stream), enqueues the
device-to-host copy and waits in cudaStreamSynchronize. Sizing reads only
the string and list offsets of the range, and the copy reads each column
through a view of it instead of first copying the range on the device. The wait holds the pool worker until the copy
completes. Algeon sets no CUDA device scheduling flag. Under the runtime
default, cudaDeviceScheduleAuto, the waiting thread spins on its CPU core
while the process has no more active CUDA contexts than logical processors,
and yields to other threads otherwise.
Once cudaStreamSynchronize returns, the worker hands fixed-width values and
packed null masks to Arrow arrays without another copy. Boolean values are
packed to bits, Decimal32 and Decimal64 columns are widened when the schema
asks for Decimal128, and string columns are rebuilt row by row. The last chunk
ends the result stream,
and the attempt releases its grant
once no fragment is running.