Skip to main content

Define virtual assets

When Snowflake manages your transformations — through Dynamic Tables, views, or other objects — Dagster should know they exist without trying to run them. The question is which of Dagster's two non-executable asset types is the right fit.

Both external assets and virtual assets appear in the lineage graph without being executed by Dagster. The difference is how automation treats them:

  • External assets are opaque to automation conditions. Dagster sees them in the graph, but evaluating whether a change should trigger a downstream run requires an explicit event.
  • Virtual assets are transparent. Automation conditions can look through them to their real upstream sources via resolve_through_virtual(), treating them like a synchronous view that is instantly consistent with its source.

Snowflake Dynamic Tables are a natural fit for virtual assets. They refresh automatically based on a target lag — Snowflake handles the computation, and Dagster should never run them. Declaring them with is_virtual=True models this correctly while keeping full lineage intact.

A Dynamic Table is not a view

The transparency described above is ideal for lineage, but be careful with it for automation. A Dynamic Table isn't instantly consistent with its source. It refreshes asynchronously on a target lag. Triggering downstream work the moment the source changes would run ahead of the data. The automation page covers why, and shows the correct trigger: fire on refresh completion, driven by the freshness sensor.

External source tables

The raw source tables — loaded by an external ETL pipeline — are represented as plain AssetSpec objects. Dagster tracks them for lineage but never executes them:

project_snowflake_dynamic_tables/defs/assets/sources.py
raw_orders = dg.AssetSpec(
key="raw_orders",
group_name="sources",
description="Raw order records loaded by the nightly ETL pipeline.",
metadata={
"snowflake_table": "ECOMMERCE.RAW.RAW_ORDERS",
"owner": "data-engineering-team",
},
kinds={"snowflake"},
)

raw_customers = dg.AssetSpec(
key="raw_customers",
group_name="sources",
description="Raw customer records loaded by the nightly ETL pipeline.",
metadata={
"snowflake_table": "ECOMMERCE.RAW.RAW_CUSTOMERS",
"owner": "data-engineering-team",
},
kinds={"snowflake"},
)

Snowflake Dynamic Tables as virtual assets

The Dynamic Tables are declared with is_virtual=True. They appear in the lineage graph and can be observed, but Dagster will never place them in a run:

project_snowflake_dynamic_tables/defs/assets/dynamic_tables.py
customer_lifetime_value = dg.AssetSpec(
key="customer_lifetime_value",
group_name="dynamic_tables",
description=(
"Snowflake Dynamic Table: per-customer lifetime value computed from orders.\n\n"
"TARGET_LAG = '1 minute', REFRESH_MODE = INCREMENTAL.\n"
"Snowflake refreshes this automatically — Dagster provides lineage and monitoring."
),
deps=["raw_orders", "raw_customers"],
is_virtual=True,
metadata={
"snowflake_object_type": "DYNAMIC TABLE",
"target_lag": "1 minute",
"refresh_mode": "INCREMENTAL",
"snowflake_table": "ECOMMERCE.ANALYTICS.CUSTOMER_LIFETIME_VALUE",
},
kinds={"snowflake"},
)

daily_revenue_rollup = dg.AssetSpec(
key="daily_revenue_rollup",
group_name="dynamic_tables",
description=(
"Snowflake Dynamic Table: daily revenue aggregated from raw orders.\n\n"
"TARGET_LAG = '1 hour', REFRESH_MODE = FULL.\n"
"Snowflake refreshes this automatically — Dagster provides lineage and monitoring."
),
deps=["raw_orders"],
is_virtual=True,
metadata={
"snowflake_object_type": "DYNAMIC TABLE",
"target_lag": "1 hour",
"refresh_mode": "FULL",
"snowflake_table": "ECOMMERCE.ANALYTICS.DAILY_REVENUE_ROLLUP",
},
kinds={"snowflake"},
)

Key properties on each virtual spec:

  • is_virtual=True — marks the asset as unexecutable
  • deps=[...] — preserves lineage so automation can traverse from source tables through the Dynamic Tables to downstream assets
  • metadata — stores Snowflake-specific properties (target lag, refresh mode, table name) visible in the asset catalog

Next steps

Continue this example by monitoring Dynamic Table freshness.