Python Script Actions

import { Aside } from ‘@astrojs/starlight/components’;

A python: script action runs a Python script as a first-class node in your DAG — the ingestion slot most SQL workflow tools leave outside the pipeline. Download a file, unzip an archive, call an API, reshape a CSV: steps that used to live in a crontab beside the pipeline become actions with dependencies, run history, and validation.

python: fetch_openaddresses.py  →  import (CSV)  →  staging view  →  mart  →  assertions

One sqlanvil run executes the whole chain, in order.

Declaring one

Script actions are declared in definitions/actions.yaml (not a new file extension), pointing at a plain .py file anywhere in your project:

actions:
  - python:
      name: load_openaddresses
      file: loader/load_openaddresses.py
      args: ["northeast"]                    # passed verbatim as sys.argv[1:]
      requirements: loader/requirements.txt  # optional — validated, never installed
      pythonVersion: ">=3.11"                # optional PEP 440 specifier
      venv: .venv                            # optional — use this venv's interpreter
      dependencies: ["some_upstream_action"] # standard dependencies
      tags: ["ingest"]

Downstream actions depend on it by name like any action — typically a type: "import" loads the file the script staged:

-- definitions/sources/orders_in.sqlx
config {
  type: "import",
  dependencyTargets: [{name: "load_openaddresses"}],
  import: { location: "staged/orders.csv", format: "csv", overwrite: true }
}

The execution contract

The runner spawns <interpreter> <file> <args...> with cwd = your project directory.
Interpreter: the declared venv’s own bin/python if set, otherwise python3 from PATH. A declared venv with no interpreter fails loudly (rather than silently running a different environment than the one you declared).
Environment variables: SA_VARS (your vars as a JSON object) and SA_ACTION_NAME.
No warehouse credentials are ever injected. Scripts stage files; import is the loading boundary. This keeps SQLAnvil’s zero-credential-custody and single-write-warehouse models intact — and keeps the feature from becoming “Airflow but worse.”
Exit code 0 is success. Output streams into the run log (and the tail is attached to the error on failure).
Timeout: 30 minutes by default; set timeoutMillis per action.

# loader/load_openaddresses.py — the script side of the contract
import json, os, sys

vars = json.loads(os.environ["SA_VARS"])      # workflow_settings.yaml vars / --vars
region = sys.argv[1] if len(sys.argv) > 1 else "all"
print(f"staging {region} for action {os.environ['SA_ACTION_NAME']}")
# ... write files under the project dir; a downstream import loads them ...

You own the environment; SQLAnvil validates it

SQLAnvil never installs anything — no pip, no resolver, no lockfiles. requirements: declares the contract; your venv, container, or CI step satisfies it (pip install -r …).

What you get in exchange is real validation. sqlanvil validate (and run --dry-run) checks each script action without executing it — the Python analog of validating a model with EXPLAIN:

the resolved interpreter satisfies pythonVersion;
every requirements: spec is satisfied by the installed packages (checked offline via importlib.metadata);
the script parses (a compile check, catching syntax errors).

Failures read like a to-do list, and anything downstream of a failed script is BLOCKED:

  FAIL   public.load_openaddresses
           python interpreter: installed 3.9.6 does not satisfy ">=3.11"
           requirements: requests is not installed (need >=2.31)
  SKIP   oa_ext.orders_in (import — not validated)
  BLOCK  public.stg_orders — blocked by an upstream failure

Compile stays hermetic: sqlanvil compile never runs Python. It checks config shape, that the script and requirements files exist in the project, and that pythonVersion is a well-formed specifier — all deterministic.

Where it runs

Local CLI — works out of the box (python3 on PATH, or your venv:).
Your own CI — run actions/setup-python + pip install -r requirements.txt before sqlanvil run; validate passes there because the env satisfies the contract.
SQLAnvil Cloud — hosted runs reject script actions at compile time (like local file paths): arbitrary user code on the shared runner is not supported. Use the local CLI or your own CI for projects with script actions.

Worked example

The in-repo examples/supabase_bigquery_mailing_list stages OpenAddresses.io address points with a python: action (bundled sample by default; set an oa_source_url var to download a real region), imports the staged CSV, and joins it to customers for address-level distance filtering — with an assertion guarding coordinate quality.

FAQ

Is this compile-time Python, like .js files? No. .js files are compile-time preprocessors that shape SQL; python: actions are execution-time steps that run when the DAG runs. Compile never executes Python.

Can a script return rows / a dataframe? No — scripts produce files, and import loads them. That boundary is deliberate (dbt-style Python models require warehouse-native Python compute, which Postgres/Supabase don’t have).

Other languages? The underlying action is language-neutral (script: with a language field — python: is sugar), so additional runtimes can arrive without breaking changes. Python is the only supported language today.