<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Sam Fisher</title>
    <description>The latest articles on DEV Community by Sam Fisher (@fisher_51).</description>
    <link>https://dev.to/fisher_51</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4211023%2F44d4195b-5f98-4a3d-adfd-8d11cbb100d7.jpg</url>
      <title>DEV Community: Sam Fisher</title>
      <link>https://dev.to/fisher_51</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/fisher_51"/>
    <language>en</language>
    <item>
      <title>Airflow retries are not a retry strategy</title>
      <dc:creator>Sam Fisher</dc:creator>
      <pubDate>Wed, 28 Oct 2026 14:16:22 +0000</pubDate>
      <link>https://dev.to/fisher_51/airflow-retries-are-not-a-retry-strategy-32n4</link>
      <guid>https://dev.to/fisher_51/airflow-retries-are-not-a-retry-strategy-32n4</guid>
      <description>&lt;p&gt;For a long time our answer to flaky pipelines was "just add retries". Three retries, then five. It made the failure rate look better and made everything else worse.&lt;/p&gt;

&lt;p&gt;What actually happened:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Real failures got buried. A task that failed because of a schema change looked identical to a task that failed because BigQuery hiccuped, and both eventually "succeeded" after a retry.&lt;/li&gt;
&lt;li&gt;Runtime crept up. Retries with exponential backoff on a slow dbt model turned a 40 minute DAG into a 90 minute one on bad days.&lt;/li&gt;
&lt;li&gt;The on-call runbook was useless. "It recovers on its own" is not a runbook.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;What we changed:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Classified failures. Timeouts, rate limits and transient network errors get retries. Data issues, schema changes and null explosions do not — they fail fast with a clear message.&lt;/li&gt;
&lt;li&gt;Added an explicit &lt;code&gt;retry_exponential_backoff&lt;/code&gt; and a max retry window, so a DAG can't quietly burn two hours.&lt;/li&gt;
&lt;li&gt;Wrote down the expected runtime per task and alert when it exceeds it. That single alert has caught more real problems than any retry ever did.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Retries are fine for flaky infrastructure. They are terrible as a substitute for knowing why something broke, and even worse when they hide the fact that your data arrived wrong three times in a row.&lt;/p&gt;

&lt;p&gt;One thing I still haven't figured out: where to draw the line for semi-transient errors like a query that times out once every few hours. Right now we retry it, but it feels like we're just postponing the investigation.&lt;/p&gt;

</description>
      <category>airflow</category>
      <category>data</category>
      <category>dbt</category>
      <category>devops</category>
    </item>
    <item>
      <title>dbt tests that actually catch bad data</title>
      <dc:creator>Sam Fisher</dc:creator>
      <pubDate>Tue, 27 Oct 2026 15:26:14 +0000</pubDate>
      <link>https://dev.to/fisher_51/dbt-tests-that-actually-catch-bad-data-a80</link>
      <guid>https://dev.to/fisher_51/dbt-tests-that-actually-catch-bad-data-a80</guid>
      <description>&lt;p&gt;Most dbt test suites I've inherited check &lt;code&gt;not_null&lt;/code&gt; and &lt;code&gt;unique&lt;/code&gt; and not much else. Those catch schema problems, not business problems.&lt;/p&gt;

&lt;p&gt;The tests that have saved us:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;accepted_values&lt;/code&gt; for status columns (catch typo'd statuses)&lt;/li&gt;
&lt;li&gt;A &lt;code&gt;relationships&lt;/code&gt; test on every foreign key&lt;/li&gt;
&lt;li&gt;A custom test for "row count per day should be within 30% of last week"&lt;/li&gt;
&lt;li&gt;Freshness checks on source tables&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The custom tests took an afternoon and have caught two silent pipeline failures.&lt;/p&gt;

&lt;p&gt;I'm learning GraphQL next, mostly because our analytics API is moving that direction. Data people: how are you testing your marts?&lt;/p&gt;

</description>
      <category>dataengineering</category>
      <category>dbt</category>
      <category>testing</category>
    </item>
  </channel>
</rss>
