Skip to main content

TSVD

UDTF: cuml_tsvd

Official cuML reference: Python API

cuML truncated SVD fits and transforms dense feature rows.

Quickstart​

Register the relations referenced by the parenthesized SELECT clauses below. For metadata validation, the descriptor names input_vectors.

SELECT id, pc_0, pc_1
FROM cuml_tsvd(
input => (SELECT id, d0, d1 FROM t),
n_components => 2
)
ORDER BY row_ordinal;

Inputs​

Each relation argument is a parenthesized SELECT subquery. Metadata validation resolves a registered table or view for the same role without scanning its rows. See ML Inputs for the ID, feature element, Float64 narrowing, null, finite-value, and runtime-dimension contract.

RoleRequiredValidation referenceDescription
inputyestableDense feature rows evaluated by the call.

Vector element types​

Feature values must be Float32 or Float64.

Arguments and options​

Scalar SQL arguments​

ArgumentTypeRequiredDefaultConstraintsDescription
n_componentsintegeryesNo defaultminimum 1; maximum 2147483647Number of transformed pc_<index> columns.
n_iterationsintegerno15minimum 1; maximum 4294967295Maximum number of eigensolver iterations.
solverstringno"cov_eig_dq"one of "cov_eig_dq", "cov_eig_jacobi"Covariance eigensolver used by the query-local fit.
tolnumberno0minimum 0Finite nonnegative eigensolver tolerance.

Vector binding shapes​

Each relation subquery projects an id and either feature columns in dimension order or one list column, with the element types listed under Vector element types. Nullable field types are accepted. A null ID, feature value, list row, or list element fails the query.

The list-column shape excludes other feature columns. See ML Inputs for the allowed list containers and runtime checks.

Output​

ColumnTypeNullableDescription
row_ordinalUInt64noEvaluated input row position, including when IDs repeat. Follows the input's top-level ORDER BY on output columns; unspecified without one.
idsame_as_input.idnoLogical input ID.
pc_<index>Float32noTransformed component value, repeated n_components times. The output contains n_components such columns, pc_0 through pc_&lt;n_components - 1&gt;.

The output schema depends on the call. Run gpu_validate_call against registered relations to inspect the schema that the validated ID types and literal options produce.

Examples​

This example uses the EMBER2024 demo dataset.

What does two-component truncated SVD show in the challenge cohort?​

The query projects all 3,225 Win32 challenge histograms from 512 dimensions to two components. The CPU geometry check measured 62.853% uncentered energy in those components.

An independent CPU comparison of the transformed Gram matrix passed with relative Frobenius error 9.668349094726904e-07 against a 0.0001 tolerance.

Family metadata is joined after the fit and does not enter the TSVD input.

-- Project all Win32 challenge histograms to two truncated-SVD components for
-- comparison with cuVS PCA and UMAP.
SELECT p.id,
p.pc_0,
p.pc_1,
v.label,
m.week_id,
m.sha256,
m.family
FROM cuml_tsvd(
input => (
SELECT r.id, r.histogram AS vector
FROM ember_research_vectors_challenge r
JOIN ember_metadata_challenge m ON m.id = r.id
WHERE r.split = 'challenge' AND m.file_type = 'Win32'
ORDER BY r.id
),
n_components => 2
) p
JOIN ember_research_vectors_challenge v ON v.id = p.id AND v.split = 'challenge'
JOIN ember_metadata_challenge m ON m.id = v.id
ORDER BY p.row_ordinal;

Hover or click a dot, or focus the map and press ← / →.

Clusters

3,225 of 3,225 points match

doenerium
EMBER ID
20000000
Week
0
Cluster
5
Membership
1.000
058f63432dd8f679aef17b6fa16dd513273f10d8512f9897fa850585ea1d6d02
All 3,225 saved challenge Win32 rows are included. Projection and cluster values come from saved SQL results. Switching the projection or changing a filter does not rerun SQL. Family is post-fit metadata. HDBSCAN clusters were fitted to PCA-16 coordinates, while this map can display PCA-2, TSVD-2, or UMAP-2. Cluster IDs are local to this capture. Coordinates are visual projections with no calibrated malware-risk interpretation. Noise (-1) records rows left unassigned under these HDBSCAN parameters. That label alone carries no novelty classification. Download the saved capture data.
Saved SQL capture IDs
  • PCA-2: 65e2e2b8-9734-4623-966b-88a7925a0ca4
  • TSVD-2: ba1ddc3c-65cb-43b8-a150-2bef95a41549
  • UMAP-2: da5ef3a6-7158-4d34-8f69-23c12e92bb63
  • PCA-16 plus HDBSCAN: 9cdfba96-0299-4220-861c-fef49bf1a52d

Limits​

  • Empty input returns an empty result without a native call. Nonempty input requires at least two evaluated rows and two feature dimensions. n_components cannot exceed dimension; known dimensions are checked during planning, otherwise execution reports a structured error.
  • Float64 features, as wide columns or list elements, are narrowed to Float32 on the device before training or inference. Each value rounds to the nearest Float32, which keeps about seven significant digits; a value beyond the Float32 range becomes infinite and fails the query.
  • Component sign is not semantic; this operation returns transformed rows only, not components, explained variance, or mean.
  • TSVD allocates a temporary rows * n_components score matrix inside the query device cap.