Vane Data / Concepts

AI Functions

Vane AI Functions are provider-backed, stateful batch UDFs that return an Expression. They place one model result beside each input row, so identifiers, source text, and other review fields can remain in the same projection.

This page covers only the Expression UDF surface. SQL and Python are two ways to build the same row-preserving operation; choosing one does not change its execution semantics.

1. Expression Contract

Vane currently exposes two AI operations as Expressions:

Operation	SQL Expression	Python Expression	Result	Supported providers
Generate text or structured output	ai_prompt(messages, ...)	vane.ai.prompt(expression, ...)	One VARCHAR or native STRUCT value per input row	openai, google, anthropic, vllm
Create an embedding	ai_embed(text, ...)	vane.ai.embed(expression, ...)	One fixed-size FLOAT[n] value per input row	openai, google, transformers

Both operations preserve row count. Keep any source columns needed later in the projection, name the generated column with SQL AS or Python .alias(), and carry a stable ID so every provider result can be traced to its input.

SQL options are planning configuration, not row data. Pass provider and model selection plus common operation settings as named arguments, and pass provider request and execution settings through options := struct_pack(...). Those values must be constants; provider names, models, and execution settings cannot vary by input row in one Expression.

There is no Expression classification function. The provider capability table above is intentionally limited to operations that return an Expression.

2. Providers

Provider adapters map the same Expression contract to different hosted APIs or local model runtimes. Install Vane once; Vane then loads provider adapters lazily, so only the library for the selected provider needs to be added.

shell

uv pip install vane-ai

Provider	Prompt	Embed	Runtime and setup
openai	Yes	Yes	OpenAI API or an OpenAI-compatible endpoint; install vane-ai[openai] and set OPENAI_API_KEY in the worker environment
google	Yes	Yes	Google Generative AI; install vane-ai[google] and set GOOGLE_API_KEY in the worker environment
anthropic	Yes	No	Anthropic Messages API; install vane-ai[anthropic] and set ANTHROPIC_API_KEY in the worker environment
vllm	Yes	No	Local or remote vLLM engine; install vane-ai[vllm], provide model access, and use Ray when GPU actors are required
transformers	No	Yes	Local SentenceTransformers model; install vane-ai[transformers]

Set provider and model explicitly when reproducibility matters. Model IDs, limits, and availability belong to the configured endpoint, so choose a model that endpoint supports.

3. Configure OpenAI

The examples below use the OpenAI provider. Install its client and expose the credential to the process that executes the query:

shell

uv pip install 'vane-ai[openai]'
export OPENAI_API_KEY="<your-token>"


# Optional: set this only for an OpenAI-compatible endpoint.
export OPENAI_BASE_URL="https://provider.example/v1"

The OpenAI client reads both variables when it is created on a worker. OPENAI_API_KEY accepts either an OpenAI API key or the token issued by a compatible service. OpenAI API users can omit OPENAI_BASE_URL; the client then uses the official API root. For Ray jobs, set the variables and install the client on every worker that can execute the Expression.

The examples below demonstrate the other supported form: passing the non-secret base_url explicitly in struct_pack(...) or as a Python keyword. Replace that value for a compatible service. An explicit option takes precedence over OPENAI_BASE_URL; when relying on the environment variable, omit the option from the Expression. In both forms, use the API root, typically ending in /v1, rather than an operation path such as /chat/completions. For a compatible endpoint that implements Chat Completions but not the OpenAI Responses API, also set use_chat_completions := true in SQL or use_chat_completions=True in Python.

Keep the token in OPENAI_API_KEY. Never place it in SQL, a Python options object, notebook output, or a committed file. SQL binding rejects credential-like option fields.

4. SQL Expression

SQL is a compact default when the surrounding transformation is already a query. A Vane connection registers ai_prompt and ai_embed; provider and model selection plus common operation settings are named arguments, while the options struct contains provider request and execution settings. Ray is Vane's default runner, and the examples' show() calls materialize the relation through RayRunner. Call vane.configure(runner="ray") before creating the connection only when you need to override an existing local runner setting.

Generate Text with ai_prompt

example.py

import vane


con = vane.connect()
documents = con.sql("""
    SELECT *
    FROM (VALUES
        (1, 'The customer asked for a refund after a duplicate charge.'),
        (2, 'The shipment is delayed because the address is incomplete.')
    ) AS t(document_id, text)
""")


prompted = con.sql("""
    SELECT
        document_id,
        text,
        ai_prompt(
            text,
            system_message := 'Write one short support summary. Return plain text only.',
            provider := 'openai',
            model := 'gpt-4o-mini',
            options := struct_pack(
                base_url := 'https://api.openai.com/v1',
                timeout := 60.0,
                actor_number := 1,
                max_concurrency_per_actor := 4,
                max_output_tokens := 64,
                temperature := 0.0
            )
        ) AS summary
    FROM documents
    ORDER BY document_id
""")


prompted.show()

actor_number selects the number of provider actors. For prompt calls, max_concurrency_per_actor bounds simultaneous API requests inside each actor. Start conservatively and raise either value only within the endpoint's rate and capacity limits.

Create Vectors with ai_embed

example.py

embedded = con.sql("""
    SELECT
        document_id,
        text,
        ai_embed(
            text,
            provider := 'openai',
            model := 'text-embedding-3-small',
            options := struct_pack(
                base_url := 'https://api.openai.com/v1',
                encoding_format := 'float',
                normalize := true,
                actor_number := 1
            )
        ) AS embedding
    FROM documents
    ORDER BY document_id
""")


embedded.show()

Every embedding Expression has a fixed-size FLOAT[n] result. Vane knows the default dimensions of supported official models. For an OpenAI-compatible endpoint, or when requesting a supported shortened embedding, pass a positive dimensions := n named argument explicitly. normalize := true L2-normalizes each non-null vector.

5. Python Expression

Python Expression has the same output and execution contract. Its closed keyword surface keeps provider, request, and execution options together and rejects unknown names.

example.py

import vane


con = vane.connect()
documents = con.sql("""
    SELECT *
    FROM (VALUES
        (1, 'The customer asked for a refund after a duplicate charge.'),
        (2, 'The shipment is delayed because the address is incomplete.')
    ) AS t(document_id, text)
""")


prompted = documents.select(
    vane.col("document_id"),
    vane.col("text"),
    vane.ai.prompt(
        vane.col("text"),
        provider="openai",
        model="gpt-4o-mini",
        base_url="https://api.openai.com/v1",
        timeout=60.0,
        actor_number=1,
        max_concurrency_per_actor=4,
        max_output_tokens=64,
        temperature=0.0,
        system_message="Write one short support summary. Return plain text only.",
    ).alias("summary"),
).order("document_id")


embedded = documents.select(
    vane.col("document_id"),
    vane.col("text"),
    vane.ai.embed(
        vane.col("text"),
        provider="openai",
        model="text-embedding-3-small",
        base_url="https://api.openai.com/v1",
        timeout=60.0,
        actor_number=1,
        encoding_format="float",
        normalize=True,
    ).alias("embedding"),
).order("document_id")


prompted.show()
embedded.show()

The same keywords work in relation-returning calls such as vane.ai.prompt(documents, vane.col("text"), ...). Keep credentials in the worker environment: credential-like option names are rejected rather than serialized into a plan.

Edit this page