vane
StarContact usGet Started →
Use Cases

AI pipelines Vane is built for

Real user scenarios, not just examples. Each one is the same shape: the problem, the pipeline, the code, what goes in and out, and when to reach for it.

Web Text to EmbeddingsSemantic SearchText DeduplicationImage PipelinesImage GenerationMultimodal Structured OutputVoice AI Analytics

Web Text to Embeddings

embeddings
WARC rows→decode English pages→chunk text→embed_text→write files
Problem

Web crawl records need to be decoded, language-filtered, chunked, embedded, and written without losing their source IDs.

Input / Output
input
5 built-in WARC-shaped records
output
3 English pages · 7 chunks · 384-d embeddings
When to use it

Preparing retrieval or training data from Common Crawl WET/WARC input.

common_crawl.py
# Uses the default Ray runner
python examples/common_crawl.py
Example: examples/common_crawl.pyOpen example →

Text Deduplication

preprocessing
normalize→MinHash→LSH candidates→components→keep one
Problem

Near-duplicate text must be grouped reproducibly, with candidate pairs and cluster decisions available for inspection.

Input / Output
input
10 built-in documents
output
5 retained representatives + audit CSVs
When to use it

Cleaning text before tokenization, training, or embedding.

minhash_dedupe.py
# No model-specific dependency required
python examples/minhash_dedupe.py
Example: examples/minhash_dedupe.pyOpen example →

Image Pipelines

vision
image bytes→AnalyzeRedRegionsBatch→rank→save images→save masks
Problem

Image bytes need a batched decode-and-analysis path that preserves metadata and emits inspectable previews.

Input / Output
input
5 generated sample images
output
ranked metadata · PNG previews · red masks
When to use it

Validating image UDF batching and file outputs before adapting a real dataset.

querying_images.py
# Pillow is the only example-specific dependency
python examples/querying_images.py
Example: examples/querying_images.pyOpen example →

Image Generation

generation
prompt rows→GenerateImageFromTextBatch→PNG files→metadata CSV
Problem

A prompt table needs a reproducible batched generation path and a manifest that keeps each image tied to its source row.

Input / Output
input
4 built-in prompts
output
4 deterministic placeholder PNGs + metadata
When to use it

Checking pipeline behavior locally before opting into a diffusion model and GPU.

image_generation.py
# Placeholder is the default backend
python examples/image_generation.py
Example: examples/image_generation.pyOpen example →

Multimodal Structured Output

multimodal
synthetic image+question→VLM with image→VLM text-only→compare→optional judge
Problem

Vision-language answers need typed parsing and a controlled comparison with the same question asked without its image.

Input / Output
input
1 synthetic multiple-choice image + provider key
output
typed answers + evaluation quadrant
When to use it

Evaluating whether a VLM actually uses visual evidence and diagnosing failures.

multimodal_structured_outputs.py
export HF_TOKEN=your_hugging_face_token
python examples/multimodal_structured_outputs.py --limit 1 --skip-judge
Example: examples/multimodal_structured_outputs.pyOpen example →

Voice AI Analytics

audio
audio rows→transcribe→summarize→subtitle rows→embed_text
Problem

Audio analytics combines transcription-level metadata with searchable embeddings for each subtitle segment.

Input / Output
input
3 generated WAV samples
output
3 transcripts/summaries · 6 embedded segments
When to use it

Validating an audio-to-search pipeline before enabling Whisper or hosted summaries.

voice_ai_analytics.py
# Placeholder transcription; local summary; real embeddings
python examples/voice_ai_analytics.py
Example: examples/voice_ai_analytics.pyOpen example →

Build your first AI pipeline on multimodal data.

Read the Docs →See benchmarks