Build data pipelines as SQL — versioned, tested, dependency-aware.
SQLAnvil is a free, open-source SQL workflow tool. You write your transformations as SQL, declare how they depend on each other, and SQLAnvil compiles the whole project into a dependency graph and runs it — in order, idempotently — against BigQuery, PostgreSQL, Supabase, or MySQL.
What it actually is
SQLAnvil is a fork of Dataform — the SQL-first, config-driven transformation framework — repositioned around one thing Dataform's OSS walked away from: first-class PostgreSQL & Supabase support, alongside BigQuery and MySQL. It's the transformation (the "T" in ELT) layer: it doesn't move data into your warehouse, it turns the raw data that's already there into clean, documented, tested tables.
Everything runs from the sqlanvil CLI — no server, no account, no hosted service
required. It's a Node package (@sqlanvil/cli), Apache-2.0 open source, and it works
the same on your laptop, in CI, or driven by an AI agent.
How it works
A project is a folder of SQL definitions plus a workflow_settings.yaml.
Define models as SQL
Each model is a SELECT with a small config block — its type (table, view,
incremental, …), schema, tags, and assertions. No boilerplate DDL.
Reference, don't hardcode
${ref("schema","orders")} links one model to another. Those references
are the edges of a dependency graph SQLAnvil builds at compile time.
Compile
sqlanvil compile resolves refs, applies your environment overrides, and produces
the exact SQL that will run — inspectable before anything touches the warehouse.
Run — in dependency order
sqlanvil run executes the graph topologically: upstreams first, then dependents,
creating/replacing tables and views and executing assertions. Incremental models only process
new rows.
What's in the box
Every model type
Tables, views, incremental tables, declarations (for existing sources), operations (raw SQL), and assertions — with pre/post-operations on every build path.
Assertions
Declarative tests — uniqueness, non-null, row conditions, custom SQL — that run as part of the graph and fail the build (and your CI) when data is wrong.
Run exactly what you mean
--tags and --actions pick a subset; --include-deps
(upstream) and --include-dependents (downstream) expand it along the graph.
Named environments
--environment applies per-env overrides — schema suffixes, table prefixes,
variables, default database — so dev, staging, and prod share one codebase.
Catch bad SQL before it runs
sqlanvil validate walks the DAG and checks every model against the warehouse
planner (EXPLAIN / dry-run) in an isolated shadow schema — without executing.
Import & export
type:"export" writes tables out to Parquet/CSV/JSON; type:"import"
loads files back in as ref()-able tables — via a bundled DuckDB bridge on Postgres.
Cross-warehouse reads
Named connections + foreign-data-wrapper support let a Postgres model read a BigQuery dataset (billing to your own project) — join across engines in one pipeline.
Queryable artifacts & docs
Every compile writes a Parquet catalog; query, inspect, and
docs turn your project + run history into something you (or an agent) can query.
Introspect existing tables
sqlanvil introspect reads an existing table's columns, types, and comments and
scaffolds a declaration — so you can adopt SQLAnvil over a live warehouse.
Runs on your warehouse
One codebase, four engines — with native SQL for each, not a lowest-common-denominator dialect.
- PostgreSQL — the first-class focus; idiomatic DDL, native partitioning and indexes.
- Supabase — everything Postgres, plus RLS policies, Realtime publications, pgvector; session-pooler aware.
- BigQuery — native partitioning, clustering, and dry-run validation.
- MySQL / MariaDB — native partitioning, matview emulation, COMMENT metadata.
Built for the agent-native era
SQLAnvil is SQL and plain files — which makes it ideal for AI agents to author and maintain. Bring
your own model (Claude Code, or any agent), point it at the project, and let it write and refactor
SQL while SQLAnvil's compile step, assertions, and validate keep it honest. There's
also a VS Code extension and a Claude skill for the workflow.