Concepts
Sources
Declare external data inputs for your pipeline.
Sources declare external data that your models depend on. They are defined in YAML files under sources/ in your project directory.
Table sources
Section titled “Table sources”Point at an existing table or view in your warehouse:
sources: - name: raw_events database: analytics schema: raw table: eventsReference it in models with __source("raw_events").
Expression sources
Section titled “Expression sources”Define source data inline as a SQL expression. No external tables or setup scripts needed:
sources: - name: raw__customers expression: | SELECT * FROM (VALUES (1, 'Leslie', 'Knope', 'leslie@pawnee.gov', TIMESTAMP '2026-01-15 09:00:00'), (2, 'Ron', 'Swanson', 'ron@pawnee.gov', TIMESTAMP '2026-02-01 08:00:00') ) AS raw__customers(id, first_name, last_name, email, created_at)Expression sources are resolved at compile time. They’re the escape hatch for anything the framework doesn’t natively model: external tables, warehouse-specific syntax, function calls, or any other relation type that doesn’t fit a standard table reference.
An inline source expression uses macros, constants, and enums available from its source definition
under sources/. Declarations limited to one source folder are not available in sibling folders.
See How Visibility Works.
Source audits
Section titled “Source audits”Sources support the same audit system as models. Audits attached to sources run before any dependent model is built:
sources: - name: raw_orders columns: - name: id audits: - not_null - unique audits: - expression_is_true: name: no future orders expression: "ordered_at <= CURRENT_TIMESTAMP"If a source audit with error severity fails, all downstream models that depend on that source are blocked.
Type enforcement
Section titled “Type enforcement”Type enforcement is implicit for sources. If any column declares a type, SQLBuild activates source type enforcement, casts that column during source resolution, and uses declared types for schema-change detection:
sources: - name: raw__customers expression: | SELECT 1 AS id, 'Leslie' AS first_name, 'Knope' AS last_name columns: - name: id type: INTEGER - name: first_name - name: last_nameIn this example, only id has a type declared, so only id is cast. Columns without a type are passed through unchanged.
For expression sources, SQLBuild probes the expression’s output columns and builds a projection that casts typed columns while preserving the rest. For table sources, it uses warehouse metadata to validate that declared column names exist and applies casts accordingly.
You can explicitly set type_enforcement: false on a source to disable casting even when column types are declared.
Managed sources (loaders)
Section titled “Managed sources (loaders)”Sources can be loaded by Python functions instead of pointing at existing tables or inline expressions. Set managed: true to bind a source to the @loader function of the same name, and SQLBuild will call it to populate the source table:
sources: - name: raw_customers managed: true write_strategy: table columns: - name: id type: INTEGER - name: name type: VARCHARManaged sources support incremental write strategies (table, append, delete_insert, merge), cursor-based loading, and concurrent execution.
See Loaders for the full guide on writing loader functions, write strategies, the loader context API, and auto-load behavior during builds.
Declarative integrations (no Python)
Section titled “Declarative integrations (no Python)”For common ingestion you can declare the source entirely in YAML, with no @loader function. SQLBuild generates the loader for you and runs it during the build:
- dlt - declare
dlt_sourcesforrest_api,sql_database, andfilesystemsources. - ingestr - add an
ingestrblock to a source to pull from 50+ sources.
Source freshness
Section titled “Source freshness”Source freshness lets SQLBuild observe whether a source’s data changed between runs. This feeds into planning and change detection: changed observations propagate to downstream models and can alter their planned action.
Configure freshness per source with a freshness: block:
sources: - name: raw_events schema: raw table: events freshness: strategy: column column: updated_at type: timestamp lag_tolerance: 15mStrategies
Section titled “Strategies”| Strategy | Description | Required fields |
|---|---|---|
adapter |
Uses warehouse metadata (e.g. Snowflake LAST_ALTERED). No query against the source table. |
None |
column |
Reads MAX(column) from the source table. |
column, type |
sql |
Runs a custom query that returns a single scalar value. | query, type |
adapter
Section titled “adapter”freshness: strategy: adapterUses adapter-level table metadata. Supported on Snowflake, BigQuery, Databricks, PostgreSQL, DuckDB, and SQL Server. Does not support type, column, or query.
column
Section titled “column”freshness: strategy: column column: updated_at type: timestampQueries MAX(column) from the source table. The column must be a plain column name (no expressions; use sql strategy for those). Requires type.
freshness: strategy: sql query: "SELECT MAX(version_id) FROM raw.events" type: integerRuns an arbitrary SQL query that returns a single scalar. Requires type. Does not support column.
The type field declares the value kind for comparison:
| Type | Description |
|---|---|
timestamp |
Datetime value. Supports lag_tolerance. |
integer |
Integer value. Change detected by exact comparison. |
string |
String value. Change detected by exact comparison. |
Lag tolerance
Section titled “Lag tolerance”lag_tolerance is optional and only valid with type: timestamp. It declares how much the observed timestamp can drift from the previous observation before being treated as a change:
freshness: strategy: column column: updated_at type: timestamp lag_tolerance: 2hAccepts positive durations: 15m (minutes), 2h (hours), 1d (days). If the current observation is within the tolerance of the previous one, the source is treated as unchanged.
Auto-observation
Section titled “Auto-observation”Sources without an explicit freshness: block are auto-observed using the adapter strategy if:
- The source has a physical table (not an expression source)
- The source is not managed
- The adapter supports table freshness metadata
This means most unmanaged table sources get freshness tracking automatically on adapters that support it, with no configuration needed.
Use sqb freshness to observe source freshness on demand without triggering a build. See Source freshness for how freshness feeds into change-aware builds.
Freshness config reference
Section titled “Freshness config reference”| Field | Description |
|---|---|
strategy |
Observation strategy: adapter, column, or sql |
type |
Value kind: timestamp, integer, or string |
column |
Column name for column strategy |
query |
SQL query for sql strategy |
lag_tolerance |
Duration tolerance for timestamp comparisons (e.g. 15m, 2h, 1d). Only valid with type: timestamp. |
Config reference
Section titled “Config reference”| Field | Description |
|---|---|
name |
Source name, used in __source("name") references |
database |
Target database (optional) |
schema |
Physical source schema (optional). For managed sources, overrides target loader_schema; otherwise managed loaders fall back to loader_schema, then target schema. |
table |
Target table name (defaults to name if omitted) |
expression |
Inline SQL expression (alternative to table reference) |
managed |
Set to true to bind the source to the @loader function of the same name (see Loaders) |
write_strategy |
How the loader writes data: table, append, delete_insert, or merge (requires managed: true) |
cursor_column |
Column for incremental cursor tracking (required for delete_insert and merge) |
unique_key |
Merge key column(s) (required for merge) |
freshness |
Source freshness observation config (see Source freshness) |
description |
Human-readable description |
type_enforcement |
Override implicit type enforcement (true/false). Defaults to true when any column declares a type. |
contract |
enforced or none. When enforced, downstream models validate configured column references against source columns. |
columns |
Column declarations with optional types and audits |
audits |
Source-level audits |