Vane Data / Concepts

AI Functions

Vane AI Functions let you generate text or structured output and create embeddings from SQL or Python. Each input row produces one result on the same row.

This page introduces the Expression UDF APIs. SQL and Python are two ways to build the same row-preserving operation; choosing one does not change its execution semantics.

For exact Python and SQL signatures, Relation overloads, provider options, and failure behavior, see the AI Functions API reference.

1. Choose an Operation

Vane currently provides two AI operations as Expressions:

Operation	SQL Expression	Python Expression	Result	Supported providers
Generate text or structured output	ai_prompt(messages, ...)	vane.ai.prompt(expression, ...)	One VARCHAR or native STRUCT value per input row	openai, google, anthropic, vllm
Create an embedding	ai_embed(text, ...)	vane.ai.embed(expression, ...)	One fixed-size FLOAT[n] value per input row	openai, google, transformers

Both operations preserve row count. Keep any input columns needed later in the projection, name the generated column with SQL AS or Python .alias(), and include a stable ID so each result can be traced back to its input.

Provider, model, and operation settings apply to the whole operation; they cannot vary by row. In SQL, pass top-level settings as named arguments and put provider request and execution settings in options := struct_pack(...).

2. Providers

Choose a Provider based on the operation and where the model runs. Vane loads each integration only when it is used, so install the optional dependencies for the selected Provider.

shell

uv pip install vane-ai

Provider	Prompt	Embed	Runtime and setup
openai	Yes	Yes	OpenAI API or an OpenAI-compatible endpoint; install vane-ai[openai] and set OPENAI_API_KEY in the worker environment
google	Yes	Yes	Google Generative AI; install vane-ai[google] and set GOOGLE_API_KEY in the worker environment
anthropic	Yes	No	Anthropic Messages API; install vane-ai[anthropic] and set ANTHROPIC_API_KEY in the worker environment
vllm	Yes	No	Local or remote vLLM engine; install vane-ai[vllm], make the model available to the runtime, and use Ray when GPU actors are required
transformers	No	Yes	Local SentenceTransformers model; install vane-ai[transformers]

Set provider and model explicitly when reproducibility matters. The configured endpoint determines which model IDs are available and which limits apply.

3. Configure OpenAI

The examples below use the OpenAI provider. Install its client and expose the credential to the process that executes the query:

shell

uv pip install 'vane-ai[openai]'
export OPENAI_API_KEY="<your-token>"

The OpenAI client reads OPENAI_API_KEY when it is created on a worker. The variable accepts either an OpenAI API key or the token issued by a compatible service. Vane uses the official OpenAI API root by default. For Ray jobs, set the variable and install the client on every worker that can execute the Expression.

For a compatible endpoint, explicitly pass the non-secret base_url in struct_pack(...) or as a Python keyword, as shown below. Use the API root, typically ending in /v1, rather than an operation path such as /chat/completions. If the endpoint implements Chat Completions but not the OpenAI Responses API, also set use_chat_completions := true in SQL or use_chat_completions=True in Python.

Keep the token in OPENAI_API_KEY. Never place it in SQL, a Python options object, notebook output, or a committed file. Vane rejects credential-like option fields.

4. SQL Expression

Use SQL when the surrounding transformation is already a query. A Vane connection registers ai_prompt and ai_embed. Pass provider, model, and operation settings as named arguments; put provider request and execution settings in the options struct.

Ray is Vane's default runner and executes the examples when show() is called. If an existing configuration selects the local runner, call vane.configure(runner="ray") before creating the connection.

Generate Text with ai_prompt

example.py

import vane


con = vane.connect()
documents = con.sql("""
    SELECT *
    FROM (VALUES
        (1, 'The customer asked for a refund after a duplicate charge.'),
        (2, 'The shipment is delayed because the address is incomplete.')
    ) AS t(document_id, text)
""")


prompted = con.sql("""
    SELECT
        document_id,
        text,
        ai_prompt(
            text,
            system_message := 'Write one short support summary. Return plain text only.',
            provider := 'openai',
            model := 'gpt-4o-mini',
            options := struct_pack(
                base_url := 'https://api.openai.com/v1',
                timeout := 60.0,
                actor_number := 1,
                max_concurrency_per_actor := 4,
                max_output_tokens := 64,
                temperature := 0.0
            )
        ) AS summary
    FROM documents
    ORDER BY document_id
""")


prompted.show()

actor_number selects the number of provider actors. For Prompt calls, max_concurrency_per_actor bounds simultaneous API requests inside each actor. Start conservatively and raise either value only within the endpoint's rate and capacity limits.

Create Vectors with ai_embed

example.py

embedded = con.sql("""
    SELECT
        document_id,
        text,
        ai_embed(
            text,
            provider := 'openai',
            model := 'text-embedding-3-small',
            options := struct_pack(
                base_url := 'https://api.openai.com/v1',
                encoding_format := 'float',
                normalize := true,
                actor_number := 1
            )
        ) AS embedding
    FROM documents
    ORDER BY document_id
""")


embedded.show()

Every Embed Expression returns a fixed-size FLOAT[n]. Vane includes dimension metadata for its supported default models. Pass a positive dimensions := n for an OpenAI-compatible endpoint, an unknown model, or a model that supports reduced output dimensions. normalize := true L2-normalizes each non-NULL vector.

5. Python Expression

Python Expressions follow the same row and result rules as SQL. Pass provider request and execution settings as keyword arguments; unknown options are rejected.

example.py

import vane


con = vane.connect()
documents = con.sql("""
    SELECT *
    FROM (VALUES
        (1, 'The customer asked for a refund after a duplicate charge.'),
        (2, 'The shipment is delayed because the address is incomplete.')
    ) AS t(document_id, text)
""")


prompted = documents.select(
    vane.col("document_id"),
    vane.col("text"),
    vane.ai.prompt(
        vane.col("text"),
        provider="openai",
        model="gpt-4o-mini",
        base_url="https://api.openai.com/v1",
        timeout=60.0,
        actor_number=1,
        max_concurrency_per_actor=4,
        max_output_tokens=64,
        temperature=0.0,
        system_message="Write one short support summary. Return plain text only.",
    ).alias("summary"),
).order("document_id")


embedded = documents.select(
    vane.col("document_id"),
    vane.col("text"),
    vane.ai.embed(
        vane.col("text"),
        provider="openai",
        model="text-embedding-3-small",
        base_url="https://api.openai.com/v1",
        timeout=60.0,
        actor_number=1,
        encoding_format="float",
        normalize=True,
    ).alias("embedding"),
).order("document_id")


prompted.show()
embedded.show()

The same keyword arguments work in Relation-returning calls such as vane.ai.prompt(documents, vane.col("text"), ...). Keep credentials in the worker environment rather than passing them as function options.

Edit this page