Enterprise Multimodal Data Infrastructure

Transform messy multimodal business data into trusted, auditable insights.

Bring parsed document rows, media references, tables, logs, and model outputs into one pipeline. Use SQL for deterministic checks, explicit UDF stages for extraction, and model-assisted review where it belongs.

messy materialsauditable facts
document rows · media refs · tables · logs · model outputs

messy materials

contract.pdfPDF
inspection photoimage
video framevideo
call audioaudio
spreadsheettable
system loglog
Vane

outputs

Agent-ready outputscontext attached
Context attached
insightsstructured signals and issues
evidencedocument · rule · source URI
recommendationsnext action with rule context
The Problem

The hard part isn't calling a model — it's rebuilding the evidence chain.

Today that chain is stitched across OCR scripts, temp files, model calls, SQL jobs, and review tools.

BEFORE — a fragmented chain

Messy Materials
OCR scripts
Temp files
LLM calls
More scripts
SQL rules
Manual review

⚠ scattered systems · glue code everywhere · lost source references · hard to debug · poor reproducibility

AFTER — one pipeline

Messy MaterialsVANEAuditable Factsinsights · evidence · recommendations
messy materialsauditable facts
How Vane Works

From source records to auditable insights — in one unified data pipeline.

STEP 01

Compose SQL, UDFs, and model review as relations

SQL rulesUDF stagesmodel review
relation pipeline

Start with parsed rows and source metadata, then add SQL rules, explicit UDF stages, and model-assisted review without splitting the workflow across separate jobs.

STEP 02

Make source references part of the output

model output
review rowdocument ID · rule hit · model response · source URI

SQL ai_prompt returns a scalar result; alias it in the projection, then keep document IDs, rule hits, the model response, and source URIs together in the same review row.

STEP 03

Move to Ray after local validation

local sampleRay runner
same relation shape

Validate locally first, then switch runner and UDF backends when distribution helps. The relation shape stays stable; worker storage, dependencies, and credentials still matter.

Real Example

Insurance document audit workflow

Register a Python policy rule once, then call it beside ai_prompt in SQL. Business IDs and source references stay on every audit row.

Parsed document table
document_idtextsource_uri
DOC-1029no signatureclaim.pdf
DOC-1030policy expiredpolicy.pdf
DOC-1031coverage limitmemo.pdf
Vane · one pipeline
1Python rule
2SQL AI review
3audit rows
Outputs
rule hit
AI finding
source URI
insurance_document_audit.py
import vane


@vane.func(return_dtype="VARCHAR")
def policy_check(text):
    return "missing_signature" if "missing signature" in text.lower() else None


con = vane.connect()
vane.attach_function(
    policy_check,
    parameters=["VARCHAR"],
    connection=con,
)


findings = con.sql("""
    SELECT claim_id, document_id,
           policy_check(text) AS rule_hit,
           ai_prompt(
               text,
               struct_pack(
                   provider := 'openai',
                   model := 'gpt-4o-mini',
                   system_message := 'Find missing claim evidence.'
               )
           ) AS ai_finding,
           source_uri
    FROM read_parquet('claims/*.parquet')
""")


findings.write_parquet("audit_findings.parquet")

Have document rows, media references, logs, or model outputs to turn into auditable facts?