Vane Data / Concepts

AI Functions

Vane AI Functions are provider-backed, stateful batch UDFs that return an Expression. They place one model result beside each input row, so identifiers, source text, and other review fields can remain in the same projection.

This page covers only the Expression UDF surface. SQL and Python are two ways to build the same row-preserving operation; choosing one does not change its execution semantics.

1. Expression Contract

Vane currently exposes two AI operations as Expressions:

Operation	SQL Expression	Python Expression	Result	Supported providers
Generate text	ai_prompt(messages [, options])	vane.ai.prompt(expression, ...)	One VARCHAR value per input row	openai, google, anthropic, vllm
Create an embedding	ai_embed(text [, options])	vane.ai.embed(expression, ...)	One FLOAT[] value per input row, or FLOAT[n] when dimensions=n	openai, google, transformers

Both operations preserve row count. Keep any source columns needed later in the projection, name the generated column with SQL AS or Python .alias(), and carry a stable ID so every provider result can be traced to its input.

SQL options are planning configuration, not row data. The optional struct_pack(...) argument must therefore contain constants; provider names, models, and execution settings cannot vary by input row in one Expression.

There is no Expression classification function. The provider capability table above is intentionally limited to operations that return an Expression.

2. Providers

Provider adapters map the same Expression contract to different hosted APIs or local model runtimes. Install Vane once; Vane then loads provider adapters lazily, so only the library for the selected provider needs to be added.

shell

uv pip install vane-ai

Provider	Prompt	Embed	Runtime and setup
openai	Yes	Yes	OpenAI API or an OpenAI-compatible endpoint; install openai and set OPENAI_API_KEY in the worker environment
google	Yes	Yes	Google Generative AI; install google-genai and set GOOGLE_API_KEY in the worker environment
anthropic	Yes	No	Anthropic Messages API; install anthropic and set ANTHROPIC_API_KEY in the worker environment
vllm	Yes	No	Local or remote vLLM engine; install vllm, provide model access, and use Ray when GPU actors are required
transformers	No	Yes	Local SentenceTransformers model; install sentence-transformers, transformers, and torch

Set provider and model explicitly when reproducibility matters. Model IDs, limits, and availability belong to the configured endpoint, so choose a model that endpoint supports.

3. Configure OpenAI

The examples below use the OpenAI provider. Install its client and expose the credential to the process that executes the query:

shell

uv pip install vane-ai openai
export OPENAI_API_KEY="<your-token>"


# Optional: set this only for an OpenAI-compatible endpoint.
export OPENAI_BASE_URL="https://provider.example/v1"

The OpenAI client reads both variables when it is created on a worker. OPENAI_API_KEY accepts either an OpenAI API key or the token issued by a compatible service. OpenAI API users can omit OPENAI_BASE_URL; the client then uses the official API root. For Ray jobs, set the variables and install the client on every worker that can execute the Expression.

The examples below demonstrate the other supported form: passing the non-secret base_url explicitly in struct_pack(...) or OpenAIProviderOptions. Replace that value for a compatible service. An explicit option takes precedence over OPENAI_BASE_URL; when relying on the environment variable, omit the option from the Expression. In both forms, use the API root, typically ending in /v1, rather than an operation path such as /chat/completions.

Keep the token in OPENAI_API_KEY. Never place it in SQL, a Python options object, notebook output, or a committed file. SQL binding rejects credential-like option fields.

4. SQL Expression

SQL is a compact default when the surrounding transformation is already a query. A Vane connection registers ai_prompt and ai_embed; the second argument is a constant options struct. Ray is Vane's default runner, and the examples' show() calls materialize the relation through RayRunner. Call vane.configure(runner="ray") before creating the connection only when you need to override an existing local runner setting.

Generate Text with ai_prompt

example.py

import vane


con = vane.connect()
documents = con.sql("""
    SELECT *
    FROM (VALUES
        (1, 'The customer asked for a refund after a duplicate charge.'),
        (2, 'The shipment is delayed because the address is incomplete.')
    ) AS t(document_id, text)
""")


prompted = con.sql("""
    SELECT
        document_id,
        text,
        ai_prompt(
            text,
            struct_pack(
                provider := 'openai',
                model := 'gpt-4o-mini',
                base_url := 'https://api.openai.com/v1',
                system_message := 'Write one short support summary. Return plain text only.',
                timeout := 60.0,
                concurrency := 1,
                max_api_concurrency := 4,
                max_output_tokens := 64,
                temperature := 0.0
            )
        ) AS summary
    FROM documents
    ORDER BY document_id
""")


prompted.show()

concurrency selects the number of provider actors. For prompt calls, max_api_concurrency bounds simultaneous API requests inside each actor. Start conservatively and raise either value only within the endpoint's rate and capacity limits.

Create Vectors with ai_embed

example.py

embedded = con.sql("""
    SELECT
        document_id,
        text,
        ai_embed(
            text,
            struct_pack(
                provider := 'openai',
                model := 'text-embedding-3-small',
                base_url := 'https://api.openai.com/v1',
                encoding_format := 'float',
                normalize := true,
                concurrency := 1
            )
        ) AS embedding
    FROM documents
    ORDER BY document_id
""")


embedded.show()

Without dimensions, the SQL result type is FLOAT[]. Set a positive dimensions value only when the selected embedding model supports that size; the result type then becomes fixed-size FLOAT[n]. normalize := true L2-normalizes each non-null vector.

5. Python Expression

Python Expression has the same output and execution contract. It separates provider configuration from operation-specific request options with typed objects.

example.py

import vane


con = vane.connect()
documents = con.sql("""
    SELECT *
    FROM (VALUES
        (1, 'The customer asked for a refund after a duplicate charge.'),
        (2, 'The shipment is delayed because the address is incomplete.')
    ) AS t(document_id, text)
""")


prompted = documents.select(
    vane.col("document_id"),
    vane.col("text"),
    vane.ai.prompt(
        vane.col("text"),
        provider="openai",
        model="gpt-4o-mini",
        provider_options=vane.ai.OpenAIProviderOptions(
            base_url="https://api.openai.com/v1",
            timeout=60.0,
            concurrency=1,
            max_api_concurrency=4,
        ),
        prompt_options=vane.ai.OpenAIPromptOptions(
            max_output_tokens=64,
            temperature=0.0,
        ),
        system_message="Write one short support summary. Return plain text only.",
    ).alias("summary"),
).order("document_id")


embedded = documents.select(
    vane.col("document_id"),
    vane.col("text"),
    vane.ai.embed(
        vane.col("text"),
        provider="openai",
        model="text-embedding-3-small",
        provider_options=vane.ai.OpenAIProviderOptions(
            base_url="https://api.openai.com/v1",
            timeout=60.0,
            concurrency=1,
        ),
        embedding_options=vane.ai.OpenAIEmbeddingOptions(
            encoding_format="float",
        ),
        normalize=True,
    ).alias("embedding"),
).order("document_id")


prompted.show()
embedded.show()

Use OpenAIProviderOptions for client and actor settings, OpenAIPromptOptions for generation settings, and OpenAIEmbeddingOptions for embedding request settings. Plain dictionaries are accepted too, but typed objects make provider-specific configuration clearer and catch misspelled fields earlier.

Edit this page