Skip to main content

Integrating Algeon

Algeon runs GPU execution behind ordinary DataFusion APIs: you keep your SQL, DataFrame code, and Arrow results, and choose the integration surface that matches the service boundary you own. Throughout these docs, "native" means Algeon's GPU execution path.

SurfaceUse it whenStart here
DataFusion sessionYou already own a Rust backend, DataFusion session lifecycle, auth, tenancy, and result formatting.DataFusion Session
Flight SQL serverYou want a remote Arrow Flight SQL endpoint for non-Rust clients, notebooks, agents, or service-to-service calls.Flight SQL server
Embedded backend exampleYou want one Rust process that serves product-owned REST workflows and an additional Flight SQL endpoint from one shared GPU backend.Embedded Backend Example

All three surfaces share one GPU backend. By default, Algeon selects supported relational SQL for cuDF execution. Choose functions_only to keep ordinary SQL on DataFusion CPU while explicit GPU functions run on GPU, or native_required to reject plans containing DataFusion CPU execution. See Choose an execution mode.

The cugraph feature makes the cugraph_* table functions executable. The cuvs feature makes the three documented cuvs_* functions executable once the session installs cuVS SQL and a backend is configured; see cuVS execution.

Iceberg support

Iceberg is not supported in this release, and it cannot be enabled from source: there is no iceberg feature in any manifest. The DataFusion integration moved out of apache/iceberg-rust into apache/datafusion-iceberg (iceberg-rust#3029, merged 2026-09-17). That project already targets DataFusion 55, but it is not published to crates.io and still pins iceberg by git rev, so it cannot be referenced from a published manifest. Iceberg support returns once it ships a release.

Common flow

  1. Build first. Build the native libraries and then the Rust stack with the features your deployment needs, following Build & Test. Every surface below starts from that build.
  2. Pick an entry point. Embed a DataFusion session in your Rust service, start the Flight SQL server for remote clients, or run the embedded example that combines both. In each case, declare the GPUs the process may use before running queries; the server-side settings are in Configuration.
  3. Provide tables. In an embedded session, register tables as you would in plain DataFusion (CREATE EXTERNAL TABLE ... STORED AS PARQUET or register_parquet); see Execute one attempt per query. The server reads local Parquet or Iceberg tables from a REST or Glue catalog configured under Iceberg and workspace. If you use the cugraph_* or cuvs_* functions, list and validate the installed calls with Discover & Validate GPU Functions.
  4. Check whether a query runs on the GPU. Before shipping a query, ask for its planned execution path, and the reason for any part that stays on the CPU, with GPU coverage validation: algeon_explain_coverage in embedded SQL, EXPLAIN GPU through Flight SQL, or validate_query in Rust. With the default native_preferred mode, relational candidates not selected for GPU execution stay in DataFusion.
  5. Handle failures. A GPU execution failure is not replayed on CPU; it comes back as an error with a stable code, kind, and status. At your application boundary, act on errors that carry a typed Algeon identity and keep your existing policy for ordinary DataFusion failures. See Handling Errors. Queries that cannot get GPU execution capacity in time return service_overloaded backpressure rather than waiting indefinitely; waiters are served first-in, first-out.

The cuGraph SQL API and cuVS SQL API sections are the function-level contracts. They share discovery and validation while documenting their distinct relation syntax, input semantics, lifecycle, and execution constraints.