<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: jarred0214</title>
    <description>The latest articles on DEV Community by jarred0214 (@jarred0214).</description>
    <link>https://dev.to/jarred0214</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4251160%2Fd10948eb-609d-46b6-b56f-0f282c448e1e.png</url>
      <title>DEV Community: jarred0214</title>
      <link>https://dev.to/jarred0214</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/jarred0214"/>
    <language>en</language>
    <item>
      <title>Extracting Column Lineage from Flink SQL Without Running a Cluster</title>
      <dc:creator>jarred0214</dc:creator>
      <pubDate>Wed, 18 Nov 2026 06:34:19 +0000</pubDate>
      <link>https://dev.to/jarred0214/extracting-column-lineage-from-flink-sql-without-running-a-cluster-2ajp</link>
      <guid>https://dev.to/jarred0214/extracting-column-lineage-from-flink-sql-without-running-a-cluster-2ajp</guid>
      <description>&lt;p&gt;Before dropping a source column, I want to know which downstream columns depend on it. Searching SQL files helps find direct references, but a temporary view can hide the connection between the source column and the final write.&lt;/p&gt;

&lt;p&gt;I built &lt;a href="https://github.com/jarredhj0214/linesql" rel="noopener noreferrer"&gt;LineSQL&lt;/a&gt; to extract those relationships from SQL scripts. This Flink example runs locally with Java and curl.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl -fsSL \
  https://repo.maven.apache.org/maven2/io/github/jarredhj0214/linesql-cli/1.3.0/linesql-cli-1.3.0.jar \
  -o linesql.jar &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt;
java -jar linesql.jar --dialect FLINK &lt;span class="s2"&gt;"
  CREATE TEMPORARY VIEW active_users AS
  SELECT id FROM ods.users WHERE status = 'ACTIVE';

  INSERT INTO ads.user_summary (user_id)
  SELECT id FROM active_users;
"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The final INSERT directly references &lt;code&gt;active_users.id&lt;/code&gt;. By analyzing both statements together, LineSQL resolves that reference back to &lt;code&gt;ods.users.id&lt;/code&gt;. The JSON output for the INSERT reports three kinds of dependencies.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;What to check&lt;/th&gt;
&lt;th&gt;Result&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Which tables are involved?&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;ods.users&lt;/code&gt; → &lt;code&gt;ads.user_summary&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Where does &lt;code&gt;user_id&lt;/code&gt; get its value?&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;ods.users.id&lt;/code&gt; → &lt;code&gt;ads.user_summary.user_id&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;What controls which rows are written?&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;ods.users.status&lt;/code&gt; is used in &lt;code&gt;WHERE&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Those dependencies lead to different review steps. If you drop &lt;code&gt;id&lt;/code&gt;, you need a replacement source for &lt;code&gt;user_id&lt;/code&gt;. If you change what &lt;code&gt;status = 'ACTIVE'&lt;/code&gt; means, check which rows reach the destination. Table lineage points to the same table in both cases.&lt;/p&gt;

&lt;p&gt;The structured output can also give an AI assistant evidence for a review summary. Keep the parser diagnostics alongside it so reviewers can identify scripts that need manual inspection. For example, LineSQL needs schema metadata to expand &lt;code&gt;SELECT *&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Supported SQL dialects:&lt;/strong&gt; Spark SQL, Hive, Flink SQL, StarRocks, MySQL, Oracle, SQL Server, PostgreSQL, and OceanBase. Syntax and lineage coverage vary by dialect. When you know which engine runs the script, specify its dialect explicitly.&lt;/p&gt;

&lt;p&gt;If you have a Flink SQL script that’s hard to trace, I’d like to see a minimal anonymized example and the lineage you expect. &lt;a href="https://github.com/jarredhj0214/linesql" rel="noopener noreferrer"&gt;LineSQL on GitHub&lt;/a&gt;&lt;/p&gt;

</description>
      <category>sql</category>
    </item>
  </channel>
</rss>
