Use Case · Multimodal Model Training

From raw multimodal data to training-ready dataset releases.

A unified multimodal training data pipeline that transforms raw data into versioned, training-ready datasets with SQL processing, GPU-accelerated labeling, embedding, and scalable execution from local to Ray clusters.

Run the pipeline →Request a demo →

raw · multimodal

dataset release

Execution timeline

Overlaps heterogeneous resources.

Traditional pipelines create stage barriers and pipeline bubbles. Vane overlaps CPU, GPU, and I/O workloads through streaming execution and dynamic batching.

Same pipelineTraditional and Vane run the same stages on one time scale; earlier completion means a shorter critical path.

Camera frame at an urban intersection — camera frame · ts 00:14.280

inputs

camera framesLiDAR sweepsradar returnsego pose + calib

decode

decode framesload sweeps

align

time syncego-pose align

fuse

sensor projection

package

label + tracksample embed

Traditional Pipelinestage barriers · longer critical path

CPUdecoding

GPUwaitinginference

I/Owrite

Stage barriers serialize CPU decoding, GPU inference, and I/O, leaving the GPU waiting for decoded batches.

Vane Pipelineoverlapped streaming · shorter critical path

CPUdecoding

GPUinference

I/Ostreaming

Dynamic batching overlaps CPU decoding, GPU inference, and I/O streaming, completing the same pipeline earlier.

Why Vane

Faster pipelines, in far less code.

High Performance — Extreme throughput, maximum resource utilization

From data preparation to embedding, multimodal AI workloads are bottlenecked by pipeline efficiency. Vane maximizes end-to-end throughput.

Efficient heterogeneous executionOverlap CPU processing, GPU inference, data movement, and I/O asynchronously, enabling heterogeneous resources to work concurrently instead of waiting on each other.
Streaming execution with backpressure & dynamic batchingContinuously process large-scale media and sensor data with adaptive batching and flow control, maximizing throughput while maintaining bounded memory usage.
Ray-Native distributed scalingExecute petabyte-scale historical reprocessing as a unified scalable graph on Ray, replacing fragmented multi-system pipelines and long-running batch workflows.

See the benchmarks →

Simplicity — One engine, no glue code

Multimodal data workflows often require multiple systems and layers of orchestration. Vane unifies data processing, AI inference, and dataset preparation into a single execution graph.

One engine, one graphDuckDB-compatible SQL, Python UDFs, AI functions, and Ray execution in one unified pipeline.
DuckDB-compatible APILow migration cost from existing Ray, Spark, or Daft pipelines.
From raw data to release-ready datasetsA complete pipeline in one readable graph — no glue code required.

Read the code →

Representative code

The training-data release pipeline in one graph.

File selection, media decoding, GPU captioning or auto-labeling, quality filters, deduplication, embedding, and packaged output stay in one readable pipeline.

SQL selectionRay GPU UDFSQL quality gateEmbedding + release

training_data_release.py

import vane


con = vane.connect()


raw = con.sql("""
    SELECT id, uri, media_type, content_hash
    FROM read_parquet('s3://training-corpus/*.parquet')
    WHERE split = 'train'
""")


def CaptionAndScore(table):
    ...


labeled = raw.map_batches(
    CaptionAndScore,
    schema=release_schema,
    gpus=1,
)
labeled.to_table("labeled")


release = con.sql("""
    SELECT id, uri, caption, quality_score,
           ai_embed(
               caption,
               struct_pack(
                   provider := 'transformers',
                   model := 'sentence-transformers/all-MiniLM-L6-v2'
               )
           ) AS caption_embedding
    FROM labeled
    WHERE quality_score >= 0.8
    QUALIFY row_number() OVER (
        PARTITION BY content_hash
        ORDER BY quality_score DESC
    ) = 1
""")


release.write_parquet("s3://dataset-releases/mm-v42/part-00000.parquet")

Build a reproducible multimodal training-data pipeline.

Become a design partner →Read the docs