Configs Reference

Protocol Documentation

configs.proto
Scalar Value Types

configs.proto

ActionConfig

Action config defines the contents of actions.yaml configuration files.

Field	Type	Label	Description
table	ActionConfig.TableConfig
view	ActionConfig.ViewConfig
incrementalTable	ActionConfig.IncrementalTableConfig
assertion	ActionConfig.AssertionConfig
operation	ActionConfig.OperationConfig
declaration	ActionConfig.DeclarationConfig
notebook	ActionConfig.NotebookConfig
dataPreparation	ActionConfig.DataPreparationConfig
rlsPolicy	ActionConfig.RlsPolicyConfig
realtimePublication	ActionConfig.RealtimePublicationConfig
foreignWrapper	ActionConfig.ForeignWrapperConfig
vectorIndex	ActionConfig.VectorIndexConfig
export	ActionConfig.ExportConfig
import	ActionConfig.ImportConfig
script	ActionConfig.ScriptConfig

ActionConfig.AssertionConfig

Field	Type	Label	Description
name	string		The name of the assertion.
dataset	string		The dataset (schema) of the assertion.
project	string		The Google Cloud project (database) of the assertion.
dependencyTargets	ActionConfig.Target	repeated	Targets of actions that this action is dependent on.
filename	string		Path to the source file that the contents of the action is loaded from.
tags	string	repeated	A list of user-defined tags with which the action should be labeled.
disabled	bool		If set to true, this action will not be executed. However, the action can still be depended upon. Useful for temporarily turning off broken actions.
description	string		Description of the assertion.
hermetic	bool		If true, this indicates that the action only depends on data from explicitly-declared dependencies. Otherwise if false, it indicates that the action depends on data from a source which has not been declared as a dependency.
dependOnDependencyAssertions	bool		If true, assertions dependent upon any of the dependencies are added as dependencies as well.
reservation	string		Optional. The BigQuery reservation to use for execution. If unset, the value from workflow_settings.yaml is used. If neither is set, default BigQuery behavior applies. sqlanvil CLI only (GCP sqlanvil support pending).
metadata	ActionConfig.Metadata		Metadata for this assertion.

ActionConfig.ColumnDescriptor

Field	Type	Label	Description
path	string	repeated	The identifier for the column, using multiple parts for nested records.
description	string		A text description of the column.
bigqueryPolicyTags	string	repeated	A list of BigQuery policy tags that will be applied to the column.
tags	string	repeated	A list of tags for this column which will be applied.

ActionConfig.DataPreparationConfig

Field	Type	Label	Description
name	string		The name of the data preparation.
dataset	string		The dataset (schema) of the destination table.
project	string		The Google Cloud project (database) of the destination table.
dependencyTargets	ActionConfig.Target	repeated	Targets of actions that this action is dependent on.
filename	string		Path to the source file that the contents of the action is loaded from.
tags	string	repeated	A list of user-defined tags with which the action should be labeled.
disabled	bool		If set to true, this action will not be executed. However, the action can still be depended upon. Useful for temporarily turning off broken actions.
description	string		Description of the data preparation.
errorTable	ActionConfig.DataPreparationConfig.ErrorTableConfig
loadMode	ActionConfig.LoadModeConfig

ActionConfig.DataPreparationConfig.ErrorTableConfig

Field	Type	Description
name	string	The name of the error table.
dataset	string	The dataset (schema) of the error table.
project	string	The Google Cloud project (database) of the error table.
retentionDays	int32

ActionConfig.DeclarationConfig

Field	Type	Label	Description
name	string		The name of the declaration.
dataset	string		The dataset (schema) of the declaration.
project	string		The Google Cloud project (database) of the declaration.
description	string		Description of the declaration.
columns	ActionConfig.ColumnDescriptor	repeated	Descriptions of columns within the declaration.
filename	string		Path to the source file that the contents of the action is loaded from.
tags	string	repeated	A list of user-defined tags with which the action should be labeled.
connection	string		Optional. Name of a connection (from WorkflowSettings.connections) that this declaration reads from. Only valid on declarations.
columnTypes	ActionConfig.DeclarationConfig.ColumnTypesEntry	repeated	Optional. Column name -> SQL type, used to generate the foreign table when `connection` bridges via FDW. Distinct from `columns` (descriptions).

ActionConfig.DeclarationConfig.ColumnTypesEntry

Field	Type	Label	Description
key	string
value	string

ActionConfig.ExportConfig

Configuration for a type: "export" action: writes a SELECT result to a Parquet/CSV/JSON file at a cloud or local location.

Field	Type	Label	Description
name	string		The name of the export.
dataset	string		The dataset (schema) used to qualify the export’s target name.
project	string		The Google Cloud project (database) of the export.
dependencyTargets	ActionConfig.Target	repeated	Targets of actions that this action is dependent on.
filename	string		Path to the source file that the contents of the action is loaded from.
tags	string	repeated	A list of user-defined tags with which the action should be labeled.
disabled	bool		If set to true, this action will not be executed.
description	string		Description of the export.
hermetic	bool		If true, this action only depends on data from explicitly-declared dependencies.
export	ActionConfig.ExportOptions		The export destination + format options (the `export: {}` block).

ActionConfig.ExportOptions

The user-facing export: {} block on a type: "export" action.

Field	Type	Label	Description
location	string		Destination folder/prefix URI: gs://
format	string		”parquet"
overwrite	bool		Overwrite an existing object/file. Defaults to true (defaulted in core when absent).
filename	string		Output base filename; defaults to the action name.
options	ActionConfig.ExportOptions.OptionsEntry	repeated	Format-specific passthrough options (e.g. compression, csv header/delimiter).

ActionConfig.ExportOptions.OptionsEntry

Field	Type	Label	Description
key	string
value	string

ActionConfig.ForeignWrapperConfig

Field	Type	Label
name	string
wrapper	string
server	string
options	ActionConfig.ForeignWrapperConfig.OptionsEntry	repeated
filename	string
dependencyTargets	ActionConfig.Target	repeated

ActionConfig.ForeignWrapperConfig.OptionsEntry

Field	Type	Label	Description
key	string
value	string

ActionConfig.IcebergTableConfig

Field	Type	Description
fileFormat	ActionConfig.IcebergTableConfig.FileFormat	The file format for the BigQuery table.
connection	string	The connection specifying the credentials to be used to read and write to external storage, such as Cloud Storage. The connection can have the form `{project}.{location}.{connection_id}` or `projects/{project}/locations/{location}/connections/{connection_id}”, or be set to DEFAULT.
bucketName	string	The name of the Cloud Storage bucket where table data is stored. This value is be used to construct the storage URI in the following way: gs://{bucket_name}/{table_folder_root}/{table_folder_subpath}``. If storage_uri` is provided, this value is ignored.
tableFolderRoot	string	The name of the first-level folder inside the Cloud Storage bucket where table data is stored. This value will be used to construct the storage URI in the following way: gs://{bucket_name}/{table_folder_root}/{table_folder_subpath}``. If storage_uri` is provided, this value is ignored.
tableFolderSubpath	string	The path under the first-level folder of the Cloud Storage bucket where table data is stored. This value will be used to construct the storage URI in the following way: gs://{bucket_name}/{table_folder_root}/{table_folder_subpath}``. If storage_uri` is provided, this value is ignored.

ActionConfig.ImportConfig

Configuration for a type: "import" action: loads a Parquet/CSV/JSON file into a table in the warehouse (the inverse of type: "export"). The resulting table is ref()-able.

Field	Type	Label	Description
name	string		The name of the import (the destination table name).
dataset	string		The dataset (schema) used to qualify the import’s target name.
project	string		The Google Cloud project (database) of the import.
dependencyTargets	ActionConfig.Target	repeated	Targets of actions that this action is dependent on.
filename	string		Path to the source .sqlx file the action is defined in (generated).
tags	string	repeated	A list of user-defined tags with which the action should be labeled.
disabled	bool		If set to true, this action will not be executed.
description	string		Description of the import.
hermetic	bool		If true, this action only depends on data from explicitly-declared dependencies.
import	ActionConfig.ImportOptions		The import source + format options (the `import: {}` block).

ActionConfig.ImportOptions

The user-facing import: {} block on a type: "import" action.

Field	Type	Label	Description
location	string		Source file/glob/URI to read: gs://
format	string		”parquet"
overwrite	bool		Replace the destination table (drop + create). Defaults to true. When false, rows are appended (INSERT … SELECT) into an existing table.
options	ActionConfig.ImportOptions.OptionsEntry	repeated	Format-specific passthrough options (reserved for future reader options).

ActionConfig.ImportOptions.OptionsEntry

Field	Type	Label	Description
key	string
value	string

ActionConfig.IncrementalTableConfig

Field	Type	Label	Description
name	string		The name of the incremental table.
dataset	string		The dataset (schema) of the incremental table.
project	string		The Google Cloud project (database) of the incremental table.
dependencyTargets	ActionConfig.Target	repeated	Targets of actions that this action is dependent on.
filename	string		Path to the source file that the contents of the action is loaded from.
tags	string	repeated	A list of user-defined tags with which the action should be labeled.
disabled	bool		If set to true, this action will not be executed. However, the action can still be depended upon. Useful for temporarily turning off broken actions.
preOperations	string	repeated	Queries to run before `query`. This can be useful for granting permissions.
postOperations	string	repeated	Queries to run after `query`.
protected	bool		If true, prevents the dataset from being rebuilt from scratch.
uniqueKey	string	repeated	If set, unique key represents a set of names of columns that will act as a the unique key. To enforce this, when updating the incremental table, sqlanvil merges rows with `uniqueKey` instead of appending them.
description	string		Description of the incremental table.
columns	ActionConfig.ColumnDescriptor	repeated	Descriptions of columns within the table.
partitionBy	string		The key by which to partition the table. Typically the name of a timestamp or the date column. See https://cloud.google.com/dataform/docs/partitions-clusters.
partitionExpirationDays	int32		The number of days for which BigQuery stores data in each partition. The setting applies to all partitions in a table, but is calculated independently for each partition based on the partition time.
requirePartitionFilter	bool		Declares whether the partitioned table requires a WHERE clause predicate filter that filters the partitioning column.
updatePartitionFilter	string		SQL-based filter for when incremental updates are applied.
clusterBy	string	repeated	The keys by which to cluster partitions by. See https://cloud.google.com/dataform/docs/partitions-clusters.
labels	ActionConfig.IncrementalTableConfig.LabelsEntry	repeated	Key-value pairs for BigQuery labels.
additionalOptions	ActionConfig.IncrementalTableConfig.AdditionalOptionsEntry	repeated	Key-value pairs of additional options to pass to the BigQuery API. Some options, for example, partitionExpirationDays, have dedicated type/validity checked fields. For such options, use the dedicated fields.
dependOnDependencyAssertions	bool		When set to true, assertions dependent upon any dependency will be add as dedpendency to this action
assertions	ActionConfig.TableAssertionsConfig		Assertions to be run on the dataset. If configured, relevant assertions will automatically be created and run as a dependency of this dataset.
hermetic	bool		If true, this indicates that the action only depends on data from explicitly-declared dependencies. Otherwise if false, it indicates that the action depends on data from a source which has not been declared as a dependency.
onSchemaChange	ActionConfig.OnSchemaChange		Defines the action behavior if the selected columns in the query don’t the match columns in the target table.
iceberg	ActionConfig.IcebergTableConfig		Configuration options for an Iceberg table.
metadata	ActionConfig.Metadata		Metadata for this incremental table.
reservation	string		Optional. The BigQuery reservation to use for execution. If unset, the value from workflow_settings.yaml is used. If neither is set, default BigQuery behavior applies. sqlanvil CLI only (GCP sqlanvil support pending).
postgres	PostgresOptions		Postgres and Supabase specific options.
supabase	SupabaseOptions
mysql	MysqlOptions		MySQL/MariaDB specific options (engine/charset/collation + indexes).

ActionConfig.IncrementalTableConfig.AdditionalOptionsEntry

Field	Type	Label	Description
key	string
value	string

ActionConfig.IncrementalTableConfig.LabelsEntry

Field	Type	Label	Description
key	string
value	string

ActionConfig.LoadModeConfig

Field	Type	Label	Description
mode	ActionConfig.LoadMode
incrementalColumn	string		Required when mode is MAXIMUM or UNIQUE
uniqueKey	string	repeated	required when mode is MERGE

ActionConfig.Metadata

Field	Type	Label	Description
overview	string		A detailed description of the data object.
extraProperties	google.protobuf.Struct		Extra properties of the data object.

ActionConfig.NotebookConfig

Field	Type	Label	Description
name	string		The name of the notebook.
location	string		The Google Cloud location of the notebook.
project	string		The Google Cloud project (database) of the notebook.
dependencyTargets	ActionConfig.Target	repeated	Targets of actions that this action is dependent on.
filename	string		Path to the source file that the contents of the action is loaded from.
tags	string	repeated	A list of user-defined tags with which the action should be labeled.
disabled	bool		If set to true, this action will not be executed. However, the action can still be depended upon. Useful for temporarily turning off broken actions.
description	string		Description of the notebook.
dependOnDependencyAssertions	bool		When set to true, assertions dependent upon any dependency will be add as dedpendency to this action

ActionConfig.OperationConfig

Field	Type	Label	Description
name	string		The name of the operation.
dataset	string		The dataset (schema) of the operation.
project	string		The Google Cloud project (database) of the operation.
dependencyTargets	ActionConfig.Target	repeated	Targets of actions that this action is dependent on.
filename	string		Path to the source file that the contents of the action is loaded from.
tags	string	repeated	A list of user-defined tags with which the action should be labeled.
disabled	bool		If set to true, this action will not be executed. However, the action can still be depended upon. Useful for temporarily turning off broken actions.
hasOutput	bool		Declares that this action creates a dataset which should be referenceable as a dependency target, for example by using the `ref` function.
description	string		Description of the operation.
columns	ActionConfig.ColumnDescriptor	repeated	Descriptions of columns within the operation. Can only be set if hasOutput is true.
dependOnDependencyAssertions	bool		When set to true, assertions dependent upon any dependency will be add as dedpendency to this action
hermetic	bool		If true, this indicates that the action only depends on data from explicitly-declared dependencies. Otherwise if false, it indicates that the action depends on data from a source which has not been declared as a dependency.
reservation	string		Optional. The BigQuery reservation to use for execution. If unset, the value from workflow_settings.yaml is used. If neither is set, default BigQuery behavior applies. sqlanvil CLI only (GCP sqlanvil support pending).

ActionConfig.RealtimePublicationConfig

Field	Type	Label
name	string
table	string
events	string	repeated
filename	string
dependencyTargets	ActionConfig.Target	repeated

ActionConfig.RlsPolicyConfig

Field	Type	Label
name	string
table	string
command	string
roles	string	repeated
using	string
withCheck	string
filename	string
dependencyTargets	ActionConfig.Target	repeated

ActionConfig.ScriptConfig

Configuration for a script action: an execution-time user script (Python in v1) that stages files / runs glue steps as a first-class DAG node. Language-neutral by design — language selects the per-language resolver in the CLI; the friendly python: actions.yaml key is sugar that normalizes to script: { language: "python", ... }. Scripts never receive warehouse credentials; they produce files, and type: "import" is the loading boundary.

Field	Type	Label	Description
name	string		The name of the script action (how downstream actions depend on it).
dataset	string		The dataset (schema) used to qualify the script’s target name.
project	string		The project (database) used to qualify the script’s target name.
dependencyTargets	ActionConfig.Target	repeated	Targets of actions that this action is dependent on.
filename	string		Path to the script file to execute, relative to the project root.
tags	string	repeated	A list of user-defined tags with which the action should be labeled.
disabled	bool		If set to true, this action will not be executed.
description	string		Description of the script action.
language	string		Script language (“python” in v1). Set automatically by the `python:` sugar.
args	string	repeated	Arguments passed verbatim to the script.
depsFile	string		Declared dependency manifest, relative to the project root (requirements.txt for python). sqlanvil validates the environment against it but never installs anything.
runtimeVersion	string		Runtime version specifier (a PEP 440 specifier for python, e.g. ”>=3.11”).
envRoot	string		Environment root directory, relative to the project root (a venv for python) — its interpreter runs the script. When unset, the interpreter is resolved from PATH.
timeoutMillis	int32		Per-action timeout in milliseconds. 0 means the default (30 minutes).

ActionConfig.TableAssertionsConfig

Options for shorthand specifying assertions, useable for some table-based action types.

Field	Type	Label	Description
uniqueKey	string	repeated	Column(s) which constitute the dataset’s unique key index. If set, the resulting assertion will fail if there is more than one row in the dataset with the same values for all of these column(s).
uniqueKeys	ActionConfig.TableAssertionsConfig.UniqueKey	repeated
nonNull	string	repeated	Column(s) which may never be `NULL`. If set, the resulting assertion will fail if any row contains `NULL` values for these column(s).
rowConditions	string	repeated	General condition(s) which should hold true for all rows in the dataset. If set, the resulting assertion will fail if any row violates any of these condition(s).

ActionConfig.TableAssertionsConfig.UniqueKey

Combinations of column(s), each of which should constitute a unique key index for the dataset. If set, the resulting assertion(s) will fail if there is more than one row in the dataset with the same values for all of the column(s) in the unique key(s).

Field	Type	Label	Description
uniqueKey	string	repeated

ActionConfig.TableConfig

Field	Type	Label	Description
name	string		The name of the table.
dataset	string		The dataset (schema) of the table.
project	string		The Google Cloud project (database) of the table.
dependencyTargets	ActionConfig.Target	repeated	Targets of actions that this action is dependent on.
filename	string		Path to the source file that the contents of the action is loaded from.
tags	string	repeated	A list of user-defined tags with which the action should be labeled.
disabled	bool		If set to true, this action will not be executed. However, the action can still be depended upon. Useful for temporarily turning off broken actions.
preOperations	string	repeated	Queries to run before `query`. This can be useful for granting permissions.
postOperations	string	repeated	Queries to run after `query`.
description	string		Description of the table.
columns	ActionConfig.ColumnDescriptor	repeated	Descriptions of columns within the table.
partitionBy	string		The key by which to partition the table. Typically the name of a timestamp or the date column. See https://cloud.google.com/dataform/docs/partitions-clusters.
partitionExpirationDays	int32		The number of days for which BigQuery stores data in each partition. The setting applies to all partitions in a table, but is calculated independently for each partition based on the partition time.
requirePartitionFilter	bool		Declares whether the partitioned table requires a WHERE clause predicate filter that filters the partitioning column.
clusterBy	string	repeated	The keys by which to cluster partitions by. See https://cloud.google.com/dataform/docs/partitions-clusters.
labels	ActionConfig.TableConfig.LabelsEntry	repeated	Key-value pairs for BigQuery labels.
additionalOptions	ActionConfig.TableConfig.AdditionalOptionsEntry	repeated	Key-value pairs of additional options to pass to the BigQuery API. Some options, for example, partitionExpirationDays, have dedicated type/validity checked fields. For such options, use the dedicated fields.
dependOnDependencyAssertions	bool		When set to true, assertions dependent upon any dependency will be add as dedpendency to this action
assertions	ActionConfig.TableAssertionsConfig		Assertions to be run on the dataset. If configured, relevant assertions will automatically be created and run as a dependency of this dataset.
hermetic	bool		If true, this indicates that the action only depends on data from explicitly-declared dependencies. Otherwise if false, it indicates that the action depends on data from a source which has not been declared as a dependency.
iceberg	ActionConfig.IcebergTableConfig		Configuration options for an Iceberg table.
metadata	ActionConfig.Metadata		Metadata for this table.
reservation	string		Optional. The BigQuery reservation to use for execution. If unset, the value from workflow_settings.yaml is used. If neither is set, default BigQuery behavior applies. sqlanvil CLI only (GCP sqlanvil support pending).
postgres	PostgresOptions		Postgres and Supabase specific options.
supabase	SupabaseOptions
mysql	MysqlOptions		MySQL/MariaDB specific options (engine/charset/collation + indexes).

ActionConfig.TableConfig.AdditionalOptionsEntry

Field	Type	Label	Description
key	string
value	string

ActionConfig.TableConfig.LabelsEntry

Field	Type	Label	Description
key	string
value	string

ActionConfig.Target

Target represents a unique action identifier.

Field	Type	Description
project	string	The Google Cloud project (database) of the action.
dataset	string	The dataset (schema) of the action. For notebooks, this is the location.
name	string	The name of the action.
includeDependentAssertions	bool	flag for when we want to add assertions of this dependency in dependency_targets as well.

ActionConfig.VectorIndexConfig

Field	Type	Label
name	string
table	string
column	string
dimensions	uint32
indexType	string
params	ActionConfig.VectorIndexConfig.ParamsEntry	repeated
filename	string
dependencyTargets	ActionConfig.Target	repeated

ActionConfig.VectorIndexConfig.ParamsEntry

Field	Type	Label	Description
key	string
value	string

ActionConfig.ViewConfig

Field	Type	Label	Description
name	string		The name of the view.
dataset	string		The dataset (schema) of the view.
project	string		The Google Cloud project (database) of the view.
dependencyTargets	ActionConfig.Target	repeated	Targets of actions that this action is dependent on.
filename	string		Path to the source file that the contents of the action is loaded from.
tags	string	repeated	A list of user-defined tags with which the action should be labeled.
disabled	bool		If set to true, this action will not be executed. However, the action can still be depended upon. Useful for temporarily turning off broken actions.
preOperations	string	repeated	Queries to run before `query`. This can be useful for granting permissions.
postOperations	string	repeated	Queries to run after `query`.
materialized	bool		Applies the materialized view optimization, see https://cloud.google.com/bigquery/docs/materialized-views-intro.
partitionBy	string		Optional. Applicable only to materialized view. The key by which to partition the materialized view. Typically the name of a timestamp or the date column. See https://cloud.google.com/bigquery/docs/materialized-views-create#partitioned_materialized_views.
clusterBy	string	repeated	Optional. Applicable only to materialized view. The keys by which to cluster partitions by. See https://cloud.google.com/bigquery/docs/materialized-views-create#cluster_materialized_views.
description	string		Description of the view.
columns	ActionConfig.ColumnDescriptor	repeated	Descriptions of columns within the table.
labels	ActionConfig.ViewConfig.LabelsEntry	repeated	Key-value pairs for BigQuery labels.
additionalOptions	ActionConfig.ViewConfig.AdditionalOptionsEntry	repeated	Key-value pairs of additional options to pass to the BigQuery API. Some options, for example, partitionExpirationDays, have dedicated type/validity checked fields. For such options, use the dedicated fields.
dependOnDependencyAssertions	bool		When set to true, assertions dependent upon any dependency will be add as dedpendency to this action
hermetic	bool		If true, this indicates that the action only depends on data from explicitly-declared dependencies. Otherwise if false, it indicates that the action depends on data from a source which has not been declared as a dependency.
assertions	ActionConfig.TableAssertionsConfig		Assertions to be run on the dataset. If configured, relevant assertions will automatically be created and run as a dependency of this dataset.
metadata	ActionConfig.Metadata		Metadata for this view.
reservation	string		Optional. The BigQuery reservation to use for execution. If unset, the value from workflow_settings.yaml is used. If neither is set, default BigQuery behavior applies. sqlanvil CLI only (GCP sqlanvil support pending).
postgres	PostgresOptions		Postgres-native options. For a materialized view (materialized: true), `no_data` (CREATE … WITH NO DATA) and `refresh_policy` (“on_dependency_change” → in-place REFRESH instead of drop+recreate) apply; indexes also apply to materialized views.
mysql	MysqlOptions		MySQL/MariaDB-native options. Only meaningful for a materialized view (materialized: true), which MySQL emulates as a refreshed table snapshot — engine/charset/collation/row_format and indexes apply to that table.

ActionConfig.ViewConfig.AdditionalOptionsEntry

Field	Type	Label	Description
key	string
value	string

ActionConfig.ViewConfig.LabelsEntry

Field	Type	Label	Description
key	string
value	string

ActionConfigs

Action configs defines the contents of actions.yaml configuration files.

Field	Type	Label	Description
actions	ActionConfig	repeated

BigQueryConnection

BigQueryConnection — connection params for warehouse.kind = “bigquery”. Mirrors the legacy flat fields on WorkflowSettings (default_project, default_location, default_dataset) but namespaced under warehouse.

Field	Type	Description
project	string	The Google Cloud project (database).
location	string	BigQuery location, e.g. “US”, “EU”, “europe-west4”.
defaultDataset	string	Default dataset (schema).

ConnectionConfig

A named connection: the warehouse (read/write target) or a read-only source.

Field	Type	Description
platform	string	Optional. One of “bigquery”, “postgres”, “supabase”, “mysql”. MySQL/MariaDB sources are runner-extract only (no Postgres FDW for MySQL): the CLI reads the source at run time and materializes it into the write warehouse. `sqlanvil introspect` scaffolds declarations (columnTypes) from a MySQL source.
project	string	Optional. BigQuery source default project.
dataset	string	Optional. BigQuery source default dataset.
saKeyId	string	Optional. Non-secret Vault secret id used in generated BigQuery FDW server DDL.
billingProject	string	Optional. BigQuery project that runs/bills the FDW query jobs. Defaults to `project`. Set this to your own project when `project` is a dataset you can read but not bill (e.g. `bigquery-public-data`): the FDW server’s `project_id` becomes the billing project and the foreign table reads the source via a full-FQN subquery.
host	string	Optional. Postgres/MySQL source host (non-secret; password lives in .df-credentials.json).
port	uint32	Optional. Postgres/MySQL source port.
database	string	Optional. Postgres/MySQL source database name.
defaultSchema	string	Optional. Postgres/Supabase default schema.
mode	string	Optional. Source read mode for a cross-warehouse source connection: “fdw” (default for bigquery/postgres/supabase) — a live foreign-data-wrapper table (needs the Vault secret + wrappers). “runner-extract” (default and only mode for mysql; also supported for bigquery) — the CLI reads the source directly at run time and materializes the rows into a plain table (keyless; no Vault secret, no wrappers/postgis on the branch). See the extract action.

DefaultIcebergConfig

Field	Type	Description
bucketName	string	Optional. Bucket name used to construct a storage URI when creating an Iceberg table.
tableFolderRoot	string	Optional. Table folder root used to construct a storage URI when creating an Iceberg table.
tableFolderSubpath	string	Optional. Table folder subpath used to construct a storage URI when creating an Iceberg table.
connection	string	Optional. The connection specifying the credentials to be used to read and write to external storage, such as Cloud Storage.

Environment

A named environment (dev/staging/prod). Holds only NON-SECRET overrides plus a pointer to a gitignored credentials file — never secrets themselves.

Field	Type	Label	Description
schemaSuffix	string
vars	Environment.VarsEntry	repeated
defaultDatabase	string
defaultLocation	string
credentials	string		Path (relative to the project dir) to this environment’s gitignored .df-credentials file.

Environment.VarsEntry

Field	Type	Label	Description
key	string
value	string

MysqlConnection

Field	Type	Description
host	string
port	uint32
database	string
user	string
password	string
sslMode	string	SSL mode: “disable"

MysqlOptions

MySQL/MariaDB table options, secondary indexes (plain/unique/fulltext/spatial, with optional per-column prefix lengths), row_format, and native partitioning.

Field	Type	Label	Description
engine	string		Storage engine, emitted as ENGINE=<engine> (e.g. “InnoDB”, “MyISAM”). Omit to use the server default.
charset	string		Default character set, emitted as DEFAULT CHARSET=<charset> (e.g. “utf8mb4”).
collation	string		Default collation, emitted as COLLATE=<collation> (e.g. “utf8mb4_unicode_ci”).
rowFormat	string		Row format, emitted as ROW_FORMAT=<row_format> (e.g. “DYNAMIC”, “COMPRESSED”, “COMPACT”, “REDUNDANT”). Omit to use the engine default.
indexes	MysqlOptions.Index	repeated
partition	MysqlOptions.Partition

MysqlOptions.Index

Field	Type	Label	Description
name	string		Optional; when omitted, derived as <table>_<cols>_idx (or _key if unique), capped at 63 chars.
columns	string	repeated	Column names. A column may carry a prefix length in MySQL’s own syntax — “description(50)” — emitted as `description`(50) (required to index BLOB/TEXT, optional elsewhere).
unique	bool
type	string		Index type: "" (default B-tree/engine default), “fulltext”, or “spatial”. Mutually exclusive with `unique`. NB a SPATIAL index requires a NOT NULL SRID-attributed geometry column — CTAS-created columns are nullable, so spatial usually needs an `ALTER TABLE ... MODIFY` pre-step in post_operations.

MysqlOptions.Partition

Native MySQL/MariaDB partitioning. NB: MySQL requires every column used in the partitioning expression to be part of every UNIQUE/PRIMARY key — so a partitioned incremental table’s uniqueKey must include the partition column(s).

Field	Type	Label	Description
kind	MysqlOptions.Partition.Kind
expression	string		The expression/columns inside `PARTITION BY <kind> (...)`, emitted verbatim. For RANGE/LIST: a column or expression (e.g. “id”, “YEAR(created_at)”). For HASH/KEY: a column list. (RANGE COLUMNS / LIST COLUMNS not modeled in v1.)
partitions	MysqlOptions.Partition.Bound	repeated
count	uint32		HASH/KEY partition count, emitted as `PARTITIONS <n>`. Ignored for RANGE/LIST.

MysqlOptions.Partition.Bound

RANGE/LIST child partitions. values is the raw clause body after the name, e.g. “VALUES LESS THAN (2024)” (range, use MAXVALUE for a catch-all) or “VALUES IN (‘us’, ‘ca’)” (list).

Field	Type	Label	Description
name	string
values	string

NotebookRuntimeOptionsConfig

Field	Type	Description
outputBucket	string	Storage bucket to output notebooks to after their execution.
runtimeTemplateName	string	Colab runtime template (https://cloud.google.com/colab/docs/runtimes), from which a runtime is created for notebook executions.
repositorySnapshotDestination	RepositorySnapshotDestinationConfig	Storage URI to upload the snapshot to. For empty URI it defaults to the provided output_bucket.

PostgresConnection

PostgresConnection — libpq-style connection params for warehouse.kind = “postgres”. Standard Postgres host/port/database/user.

Field	Type	Description
host	string
port	uint32
database	string
user	string
password	string
sslMode	string	SSL mode: “disable"
defaultSchema	string

PostgresOptions

PostgresOptions — Postgres-native table-level options. Mirrors what BigQueryOptions-style fields do in TableConfig but in idiomatic Postgres.

Used as a peer of the existing bigquery: {...} shape on action configs: publish(“daily_orders”, { postgres: { tablespace: “fast_ssd”, … } })

Field	Type	Label	Description
tablespace	string		Physical storage placement (CREATE TABLE … TABLESPACE <name>).
fillfactor	uint32		Storage parameter — fraction of each page to fill on insert (1-100).
unlogged	bool		CREATE UNLOGGED TABLE — faster writes, lost on crash. For staging/temp tables where durability isn’t required.
partition	PostgresOptions.Partition
indexes	PostgresOptions.Index	repeated
noData	bool		Materialized view: create WITH NO DATA (empty until first refresh). Default (false) is WITH DATA. Named for the non-default so proto3’s false default means the sensible WITH DATA.
refreshPolicy	string		Materialized view refresh on re-run: “on_dependency_change” refreshes an existing matview in place (REFRESH MATERIALIZED VIEW) instead of dropping + recreating. Default (unset) drops + recreates each run (safe — also picks up definition changes, which REFRESH does not).

PostgresOptions.Index

Indexes to create alongside the table.

Field	Type	Label	Description
name	string
columns	string	repeated
method	PostgresOptions.Index.Method
where	string		Partial index predicate (WHERE <expr>).
unique	bool
include	string	repeated	INCLUDE non-key columns for covering indexes.
opclass	string		Operator class applied to each indexed column, e.g. “gin_trgm_ops” (pg_trgm), “jsonb_path_ops”, or “vector_l2_ops” (pgvector). Required for gin/gist indexes on types without a default opclass.

PostgresOptions.Partition

Native Postgres declarative partitioning.

Field	Type	Label	Description
kind	PostgresOptions.Partition.Kind
columns	string	repeated
partitions	PostgresOptions.Partition.Bound	repeated
includeDefault	bool		Also create a catch-all DEFAULT partition so rows outside every bound still insert (recommended for declarative full-refresh loads).

PostgresOptions.Partition.Bound

Child partitions. values is the raw FOR VALUES clause body matching the kind, e.g. “FROM (‘2024-01-01’) TO (‘2025-01-01’)” (range), “IN (‘us’, ‘ca’)” (list), or “WITH (MODULUS 4, REMAINDER 0)” (hash).

Field	Type	Description
name	string
values	string
subPartition	PostgresOptions.Partition	Sub-partitioning: make this child a partitioned table in its own right (PARTITION BY …), with its own nested `partitions`. Omit for a leaf child that holds rows directly.

RepositorySnapshotDestinationConfig

Field	Type	Label	Description
repositorySnapshotUri	string		Storage URI to upload the repository snapshot to.

SupabaseConnection

SupabaseConnection — connection params for warehouse.kind = “supabase”. Supabase projects expose a Postgres connection via project_ref + service_role_key, or a direct connection string for bypassing PostgREST.

Field	Type	Description
projectRef	string	From the Supabase dashboard (project URL host before .supabase.co).
serviceRoleKey	string	Project service-role JWT. NEVER commit literally — use ${ENV_VAR} interpolation in workflow_settings.yaml.
defaultSchema	string
connectionString	string	Optional override — direct Postgres URL bypassing the PostgREST proxy. e.g. “postgresql://postgres:${PASSWORD}@db.<project_ref>.supabase.co:5432/postgres”. If set, takes precedence over project_ref + service_role_key for the direct DB connection. service_role_key is still used for RLS bypass.

SupabaseOptions

SupabaseOptions — Supabase-specific platform features layered on top of standard Postgres. Used as a peer of postgres: {...} for projects targeting warehouse: { kind: supabase }.

Field	Type	Label	Description
postgres	PostgresOptions		Standard Postgres options apply. Set these via `postgres:` directly or nest under `supabase.postgres:` — either is accepted.
publishToRealtime	bool		ALTER PUBLICATION supabase_realtime ADD TABLE <this>. Implicitly sets REPLICA IDENTITY appropriately.
enableRls	bool		ALTER TABLE <this> ENABLE ROW LEVEL SECURITY. Note: only enables RLS — policies are declared via the `rlsPolicy` action type (see Phase 5).
ownerRole	string		OWNER TO <role>. Typically “postgres” or “service_role”.
vectors	SupabaseOptions.VectorConfig	repeated

SupabaseOptions.VectorConfig

pgvector convenience config. Equivalent to declaring a PostgresOptions.Index with method=HNSW or method=GIST + ivfflat ops, but more ergonomic for RAG pipelines.

Field	Type	Label	Description
column	string
dimensions	uint32
indexType	SupabaseOptions.VectorConfig.IndexType
params	SupabaseOptions.VectorConfig.ParamsEntry	repeated	ivfflat: { lists }, hnsw: { m, ef_construction }.

SupabaseOptions.VectorConfig.ParamsEntry

Field	Type	Label	Description
key	string
value	string

WarehouseConfig

WarehouseConfig — discriminated union over connection variants. The kind: YAML tag selects which oneof arm is unmarshalled.

Example YAML: warehouse: kind: postgres host: db.example.com port: 5432 database: analytics user: sqlanvil_writer password: ${PG_PASSWORD} ssl_mode: require default_schema: public

Field	Type	Label	Description
bigquery	BigQueryConnection
postgres	PostgresConnection
supabase	SupabaseConnection
mysql	MysqlConnection

WorkflowSettings

Workflow Settings defines the contents of the workflow_settings.yaml configuration file.

Field	Type	Label	Description
sqlanvilCoreVersion	string		The desired sqlanvil core version to compile against.
defaultProject	string		Required. The default Google Cloud project (database).
defaultDataset	string		Required. The default dataset (schema).
defaultLocation	string		Required. The default BigQuery location to use. For more information on BigQuery locations, see https://cloud.google.com/bigquery/docs/locations.
defaultAssertionDataset	string		Required. The default dataset (schema) for assertions.
vars	WorkflowSettings.VarsEntry	repeated	Optional. User-defined variables that are made available to project code during compilation. An object containing a list of “key”: value pairs.
projectSuffix	string		Optional. The suffix to append to all Google Cloud project references.
datasetSuffix	string		Optional. The suffix to append to all dataset references.
namePrefix	string		Optional. The prefix to append to all action names.
defaultNotebookRuntimeOptions	NotebookRuntimeOptionsConfig		Optional. Default runtime options for Notebook actions.
builtinAssertionNamePrefix	string		Optional. The prefix to append to built-in assertion names.
defaultIcebergConfig	DefaultIcebergConfig		Optional. Default config options for Iceberg tables.
disableAssertions	bool		Optional. Disables all assertions including built-in assertions (uniqueKey, nonNull, rowConditions) and manual assertions (type: assertion). When true, assertions will still be compiled but marked as disabled.
defaultReservation	string		Optional. The default BigQuery reservation to use for execution. If unset, default BigQuery behavior applies. sqlanvil CLI only (GCP sqlanvil support pending).
extension	Extension		Optional. An external package that provides an extension.
includeTestsInCompiledGraph	bool		Optional. If set to true, unit tests will be included in the compiled graph.
warehouse	string		Optional. The database warehouse to use, e.g. “bigquery”, “postgres”, “supabase”.
connections	WorkflowSettings.ConnectionsEntry	repeated	Optional. Named connections (warehouse + read-only sources).
environments	WorkflowSettings.EnvironmentsEntry	repeated	Optional. Named environments (dev/staging/prod) selected with `--environment`.

WorkflowSettings.ConnectionsEntry

Field	Type	Label	Description
key	string
value	ConnectionConfig

WorkflowSettings.EnvironmentsEntry

Field	Type	Label	Description
key	string
value	Environment

WorkflowSettings.VarsEntry

Field	Type	Label	Description
key	string
value	string

ActionConfig.IcebergTableConfig.FileFormat

Supported file formats for BigQuery tables.

Name	Number	Description
FILE_FORMAT_UNSPECIFIED	0	Default value.
PARQUET	1	Apache Parquet format.

ActionConfig.LoadMode

Name	Number	Description
REPLACE_TABLE	0	Replace existing table (default).
APPEND	1	Insert into destination table.
MAXIMUM	2	Insert only records where the specified column value exceeds the existing maximum value in the destination table.
UNIQUE	3	Insert only records where the specified column value is not already present in the destination column values.
MERGE	4	Merge records into the destination table, deduplicating using 1+ unique keys

ActionConfig.OnSchemaChange

Name	Number	Description
IGNORE	0	Ignore any schema changes (default).
FAIL	1	Fails if the query would result in a new column(s) being added, deleted, or renamed.
EXTEND	2	Does not block any new column(s) from being added.
SYNCHRONIZE	3	Does not block any new column(s) from being added, deleted or renamed.

MysqlOptions.Partition.Kind

Name	Number	Description
RANGE	0
LIST	1
HASH	2
KEY	3

PostgresOptions.Index.Method

Name	Number	Description
BTREE	0
HASH	1
GIN	2
GIST	3
BRIN	4

PostgresOptions.Partition.Kind

Name	Number	Description
RANGE	0
LIST	1
HASH	2

SupabaseOptions.VectorConfig.IndexType

Name	Number	Description
IVFFLAT	0
HNSW	1

Scalar Value Types

.proto Type	Notes	C++	Java	Python	Go	C#	PHP	Ruby
double		double	double	float	float64	double	float	Float
float		float	float	float	float32	float	float	Float
int32	Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead.	int32	int	int	int32	int	integer	Bignum or Fixnum (as required)
int64	Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead.	int64	long	int/long	int64	long	integer/string	Bignum
uint32	Uses variable-length encoding.	uint32	int	int/long	uint32	uint	integer	Bignum or Fixnum (as required)
uint64	Uses variable-length encoding.	uint64	long	int/long	uint64	ulong	integer/string	Bignum or Fixnum (as required)
sint32	Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s.	int32	int	int	int32	int	integer	Bignum or Fixnum (as required)
sint64	Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s.	int64	long	int/long	int64	long	integer/string	Bignum
fixed32	Always four bytes. More efficient than uint32 if values are often greater than 2^28.	uint32	int	int	uint32	uint	integer	Bignum or Fixnum (as required)
fixed64	Always eight bytes. More efficient than uint64 if values are often greater than 2^56.	uint64	long	int/long	uint64	ulong	integer/string	Bignum
sfixed32	Always four bytes.	int32	int	int	int32	int	integer	Bignum or Fixnum (as required)
sfixed64	Always eight bytes.	int64	long	int/long	int64	long	integer/string	Bignum
bool		bool	boolean	boolean	bool	bool	boolean	TrueClass/FalseClass
string	A string must always contain UTF-8 encoded or 7-bit ASCII text.	string	String	str/unicode	string	string	string	String (UTF-8)
bytes	May contain any arbitrary sequence of bytes.	string	ByteString	str	[]byte	ByteString	string	String (ASCII-8BIT)