vane
StarContact usGet Started →
Use Cases

AI pipelines Vane is built for

Real user scenarios, not just examples. Each one is the same shape: the problem, the pipeline, the code, what goes in and out, and when to reach for it.

Web Text to EmbeddingsSemantic SearchText DeduplicationImage PipelinesImage GenerationBasic PromptVoice AI Analytics

Web Text to Embeddings

embeddings
WARC rows→decode English pages→chunk text→vane.ai.embed→write files
Problem

Web crawl records need to be decoded, language-filtered, chunked, embedded, and written without losing their source IDs.

Input / Output
input
5 built-in WARC-shaped records
output
3 English pages · 7 chunks · 384-d embeddings
When to use it

Preparing retrieval or training data from Common Crawl WET/WARC input.

common_crawl.py
# Uses the default Ray runner
python examples/common_crawl.py
Example: examples/common_crawl.pyOpen example →

Text Deduplication

preprocessing
normalize→MinHash→LSH candidates→components→keep one
Problem

Near-duplicate text must be grouped reproducibly, with candidate pairs and cluster decisions available for inspection.

Input / Output
input
10 built-in documents
output
5 retained representatives + audit CSVs
When to use it

Cleaning text before tokenization, training, or embedding.

minhash_dedupe.py
# No model-specific dependency required
python examples/minhash_dedupe.py
Example: examples/minhash_dedupe.pyOpen example →

Image Pipelines

vision
image bytes→AnalyzeRedRegionsBatch→rank→save images→save masks
Problem

Image bytes need a batched decode-and-analysis path that preserves metadata and emits inspectable previews.

Input / Output
input
5 generated sample images
output
ranked metadata · PNG previews · red masks
When to use it

Validating image UDF batching and file outputs before adapting a real dataset.

querying_images.py
# Pillow is the only example-specific dependency
python examples/querying_images.py
Example: examples/querying_images.pyOpen example →

Image Generation

generation
prompt rows→GenerateImageFromTextBatch→PNG files→metadata CSV
Problem

A prompt table needs a reproducible batched generation path and a manifest that keeps each image tied to its source row.

Input / Output
input
4 built-in prompts
output
4 deterministic placeholder PNG files + metadata
When to use it

Checking pipeline behavior locally before opting into a diffusion model and GPU.

image_generation.py
# Placeholder is the default backend
python examples/image_generation.py
Example: examples/image_generation.pyOpen example →

Basic Prompt

multimodal
image+prompt row→plain response→STRUCT response→raw response→SQL
Problem

Text and image prompts need row-aligned plain, structured, and raw outputs that work through both Python and SQL.

Input / Output
input
Bundled example image + OpenAI credential
output
plain · structured · raw · SQL responses
When to use it

Learning the Prompt contract before adding model calls to a larger Relation workflow.

basic_prompt.py
export OPENAI_API_KEY=your_openai_key
python examples/basic_prompt.py
Example: examples/basic_prompt.pyOpen example →

Voice AI Analytics

audio
audio rows→transcribe→summarize→subtitle rows→vane.ai.embed
Problem

Audio analytics combines transcription-level metadata with searchable embeddings for each subtitle segment.

Input / Output
input
3 generated WAV samples
output
3 transcripts/summaries · 6 embedded segments
When to use it

Validating an audio-to-search pipeline before enabling Whisper or hosted summaries.

voice_ai_analytics.py
# Placeholder transcription; local summary; real embeddings
python examples/voice_ai_analytics.py
Example: examples/voice_ai_analytics.pyOpen example →

Build your first AI pipeline on multimodal data.

Read the Docs →See benchmarks