Bringing Snowflake App Runtime Into Your Data Lineage Graph: A Horizon Context Experiment

Introduction

Snowflake App Runtime lets you build and deploy a Next.js app directly on top of Snowflake, which is great for shipping internal tools fast. The catch is that once the app is live, it tends to disappear from your metadata. Snowsight's lineage graph stops at the table and view level, so if you want to know which app or API depends on a given column, you're stuck grepping through source code.

That bugged me enough to try fixing it. Snowflake's External Lineage API — which accepts lineage events in the open OpenLineage format and folds them straight into the lineage graph — has actually been generally available for a while now. What got me looking at it again is Horizon Context, a feature Snowflake recently put into preview that extends Horizon Catalog into a broader, pluggable metadata layer. If App Runtime's APIs and screens show up as regular nodes in that lineage graph, downstream of the tables they read from, a lot of the "what breaks if I change this column" guesswork goes away.

Heads up: Horizon Context itself was still in preview as of October 2026, so some of the framing around it may shift. The External Lineage mechanics this article relies on, though, are already GA. Check the official docs for anything that looks off by the time you read this.

Horizon Context release note

Here's the plan: take a small Next.js app deployed on App Runtime, automatically extract which tables its API routes query and which screens call those APIs, and push that as an OpenLineage event into Snowflake's lineage graph.

The Sample App

I skipped the app-building part, but here's what it looks like — a couple of screens on top of the dbt-labs jaffle_shop sample data, one for analytics and one for editing master data.

Charts built from mart-layer data

A tool for writing back to Snowflake's master data

The layout follows the standard Next.js App Router conventions: API routes with embedded SQL live under app/api/, and the screens live under app/ and components/.

jaffle-app/
├── app/
│   ├── page.tsx                 ← Dashboard screen (/)
│   ├── customers/page.tsx       ← Customers screen (/customers)
│   ├── orders/page.tsx          ← Orders screen (/orders)
│   └── api/
│       ├── dashboard/route.ts   ← GET: monthly revenue
│       ├── customers/route.ts   ← GET: customer list
│       ├── orders/route.ts      ← GET: order list
│       └── master-edit/route.ts ← PUT: update customer info
├── components/
│   ├── dashboard-view.tsx       ← calls fetch("/api/dashboard")
│   ├── customers-view.tsx       ← calls fetch("/api/customers"), fetch("/api/master-edit")
│   └── orders-view.tsx          ← calls fetch("/api/orders")
└── lib/
    └── snowflake.ts             ← Snowflake connection helper

To wire this into lineage, I pull the following out of the directory automatically:

Info	How it's extracted	Example
API endpoint name	File path under app/api/	app/api/customers/route.ts → /app/api/customers
HTTP method	`export function GET\	PUT` in the file
Tables referenced	SQL template literals, parsed with sqlglot	JAFFLE_SHOP.MARTS.CUSTOMERS
Screen node name	Page file path + app name	app/customers/page.tsx → jaffle-app/customers
Screen → API mapping	fetch("/api/...") calls inside components	customers-view.tsx → /api/customers, /api/master-edit

Once that's folded into the table and view lineage, you can see at a glance which API reads a given table and which screen calls that API.

The pipeline is three steps:

1. Extract — pull SQL template literals out of the TypeScript API route files with a regex
2. Analyze — parse the SQL with sqlglot: resolve table FQNs and trace column-level lineage
3. Send — build an OpenLineage COMPLETE event and push it to Snowflake's REST API

Step 1: Pulling SQL Out of TypeScript

Every App Runtime API route lives at app/api/<name>/route.ts, following the Next.js App Router convention, and the SQL inside each one is written as a template literal:

// app/api/customers/route.ts
const LIST_SQL = (db: string, schema: string) => `
  SELECT CUSTOMER_ID, CUSTOMER_NAME,
         COUNT_LIFETIME_ORDERS,
         LIFETIME_SPEND
  FROM ${db}.${schema}.CUSTOMERS
  WHERE CUSTOMER_NAME ILIKE ?
  ORDER BY LIFETIME_SPEND DESC
  LIMIT ? OFFSET ?
`

A regex pulls that out:

_TEMPLATE_LITERAL = re.compile(
    r'(?:const|let|var)\s+(\w+)\s*=\s*'
    r'(?:\([^)]*\)\s*(?::\s*\w+)?\s*=>\s*)?'  # the arrow function's argument list
    r'`(.*?)`',
    re.DOTALL,
)

It handles both the plain const NAME = \...form and the `const NAME = (db, schema) => \`...\ arrow-function form. The HTTP method (GET/PUT/POST/DELETE) is inferred from the exported function name and the variable name in the same file.

Step 2: Parsing SQL and Tracing Column Lineage with sqlglot

The extracted queries go through sqlglot. They still contain template placeholders like ${db} and Snowflake bind variables (:name, ?), so they need a bit of preprocessing before sqlglot can touch them:

def preprocess(sql: str) -> str:
    # ${VAR} -> actual value
    sql = re.sub(r'\$\{(\w+)\}', replacer, sql)
    # :name -> __BIND_name__ (skip the :: cast operator)
    sql = re.sub(r'(?<!:):(\w+)', r'__BIND_\1__', sql)
    return sql

Once preprocessed, sqlglot parses the SQL into an AST and pulls out every table reference from it. Its lineage() API goes further and traces, for each output column of a SELECT, exactly which table and column it was derived from.

Quick aside on what an AST actually is: it's a tree representation of the code — "this SELECT has a FROM clause, which has a table name" — and sqlglot builds one for every query it parses, then walks it to resolve table references and column dependencies.

from sqlglot.lineage import lineage

result = lineage("CUSTOMER_NAME", sql, schema=schema, dialect="snowflake")
# -> CUSTOMERS.CUSTOMER_NAME

Being able to pass dialect="snowflake" directly is a nice touch. It resolves correctly even through joins, CASE expressions, and aggregates — feed it something like this:

SELECT o.ORDER_ID,
       c.CUSTOMER_NAME,    -- comes from the right side of the LEFT JOIN
       CASE
         WHEN o.ORDER_TOTAL > 50 THEN 'HIGH'
         ELSE 'LOW'
       END AS VALUE_TIER   -- derived from the CASE condition column
FROM ORDERS o
JOIN CUSTOMERS c ON o.CUSTOMER_ID = c.CUSTOMER_ID

and it correctly attributes each output column back to its source, matching the comments above.

One catch: lineage() needs schema information (the tables' column definitions) to work. I fetched that ahead of time by connecting to Snowflake and running DESCRIBE TABLE.

Step 3: Building and Sending the OpenLineage Event

Two kinds of nodes get registered as lineage:

Layer	Event content	Purpose
Layer 1	table ↔ API endpoint	SQL input/output relationship, with column lineage
Layer 2	API endpoint → screen node	which screen calls which API

Screen nodes are named with the app name included, like jaffle-app/dashboard. The screen → API mapping is generated by regex-extracting fetch("/api/...") calls out of each frontend component and matching them back to the corresponding page.tsx file.

OpenLineage is an open format for data lineage: a job's inputs and outputs get recorded as a COMPLETE event. Snowflake accepts that format at the /api/v2/lineage/external-lineage endpoint and merges it straight into the lineage graph.

The read event for Layer 1 (table → API) carries OpenLineage's columnLineage facet. Facets are OpenLineage's mechanism for attaching extra information to a lineage event, but Snowflake currently only honors two of them: datasetType (controls how the node is labeled) and columnLineage (column-level lineage). Other facets like schema, documentation, or ownership won't error out, but they're silently ignored. I'd love to see documentation supported at some point — fingers crossed for a future update.

Here's what the assembled event looks like:

{
  "eventType": "COMPLETE",
  "job": { "namespace": "jaffle-app", "name": "api-app-api-orders" },
  "inputs": [
    { "namespace": "snowflake://<account-identifier>", "name": "JAFFLE_SHOP.MARTS.ORDERS" },
    { "namespace": "snowflake://<account-identifier>", "name": "JAFFLE_SHOP.MARTS.CUSTOMERS" }
  ],
  "outputs": [
    {
      "namespace": "app-runtime://jaffle-app",
      "name": "/app/api/orders",
      "facets": {
        "datasetType": { "datasetType": "API_ENDPOINT" },
        "columnLineage": {
          "fields": {
            "ORDER_ID": {
              "inputFields": [
                { "namespace": "snowflake://<account-identifier>",
                  "name": "JAFFLE_SHOP.MARTS.ORDERS", "field": "ORDER_ID" }
              ]
            },
            "CUSTOMER_NAME": {
              "inputFields": [
                { "namespace": "snowflake://<account-identifier>",
                  "name": "JAFFLE_SHOP.MARTS.CUSTOMERS", "field": "CUSTOMER_NAME" }
              ]
            },
            "VALUE_TIER": {
              "inputFields": [
                { "namespace": "snowflake://<account-identifier>",
                  "name": "JAFFLE_SHOP.MARTS.ORDERS", "field": "ORDER_TOTAL" }
              ]
            }
          }
        }
      }
    }
  ]
}

From there, it's a plain REST call to Snowflake, using the session token from the Python connector:

rest = connection._rest
token = rest.token

url = f"{rest._protocol}://{rest._host}:{rest._port}/api/v2/lineage/external-lineage"
headers = {"Authorization": f'Snowflake Token="{token}"'}

response = requests.post(url, json=event, headers=headers)

Results

Here's CUSTOMERS before any of this: Snowflake's native query-based lineage already traces it back through staging to the raw table, but that's where it stops. No app, no API, nothing downstream.

Before: lineage ends at the table level

Once the OpenLineage event above goes through, the app's APIs show up in the lineage of the tables and views they touch:

After: both the API and the screen that calls it now show up as connected nodes

Anything registered through OpenLineage shows up labeled as External — in this case, the first node is the API, and the second is the screen that calls it.

Write APIs (PUT/PATCH-style, like the master-data editor) show up as upstream of the table instead, since they write into it rather than read from it:

A write-path API shows up upstream of the table

Being able to see which screen calls which API, on top of the table relationship, is the part I like most here.

Putting the Lineage to Work

Say you want to change the definition of CUSTOMERS.CUSTOMER_NAME. Snowflake's built-in GET_LINEAGE function will tell you exactly what that touches:

SELECT SOURCE_OBJECT_NAME, SOURCE_COLUMN_NAME,
       TARGET_OBJECT_NAME, TARGET_COLUMN_NAME, DISTANCE
FROM TABLE(
  SNOWFLAKE.CORE.GET_LINEAGE(
    'JAFFLE_SHOP.MARTS.CUSTOMERS.CUSTOMER_NAME',
    'COLUMN', 'DOWNSTREAM', 3
  )
)

Source	Source column	Target	Target column	Distance
CUSTOMERS	CUSTOMER_NAME	/app/api/customers	CUSTOMER_NAME	1
CUSTOMERS	CUSTOMER_NAME	/app/api/orders	CUSTOMER_NAME	1

One query, and it's immediately obvious that changing CUSTOMER_NAME affects both the /api/customers and /api/orders endpoints.

The same path works from Snowflake's CoCo interface too — you can land on the same app-level nodes starting from a table.

Wrapping Up

Horizon Context's metadata connectors handle the easy cases — plugging in systems that already have a connector. OpenLineage is what you reach for when there isn't one: it let me register an app that would otherwise be invisible to Snowflake's lineage graph, API by API and screen by screen, with full column-level detail.

This is a small slice of what Horizon Context can do; there's a lot more in there worth exploring. But if you've ever had to go read source code just to figure out what a schema change would break, this is a solid place to start.