<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Arnold M S</title>
    <description>The latest articles on DEV Community by Arnold M S (@epten08).</description>
    <link>https://dev.to/epten08</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3904499%2F4344221c-b44b-4aff-a51b-448ff9506627.jpeg</url>
      <title>DEV Community: Arnold M S</title>
      <link>https://dev.to/epten08</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/epten08"/>
    <language>en</language>
    <item>
      <title>Open Source OWASP API Security Scanner with AI-Assisted Testing</title>
      <dc:creator>Arnold M S</dc:creator>
      <pubDate>Thu, 30 Apr 2026 10:06:49 +0000</pubDate>
      <link>https://dev.to/epten08/open-source-owasp-api-security-scanner-with-ai-assisted-testing-27ge</link>
      <guid>https://dev.to/epten08/open-source-owasp-api-security-scanner-with-ai-assisted-testing-27ge</guid>
      <description>&lt;p&gt;Most security scanners produce a list of vulnerabilities ranked by severity and leave the remediation work to you. After working on projects where that list grew long and the question "can someone actually exploit this right now?" remained unanswered, I built something different.&lt;/p&gt;

&lt;p&gt;The result is &lt;strong&gt;Breach Gate&lt;/strong&gt;, an open source CLI tool that combines static analysis, container scanning, dynamic API testing, and AI-assisted behavioral testing into a single pipeline. It outputs one clear answer: &lt;strong&gt;SAFE&lt;/strong&gt;, &lt;strong&gt;UNSAFE&lt;/strong&gt;, or &lt;strong&gt;REVIEW REQUIRED&lt;/strong&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Core Problem
&lt;/h2&gt;

&lt;p&gt;Traditional scanners answer: &lt;em&gt;"What vulnerabilities exist?"&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Breach Gate answers: &lt;em&gt;"Can an attacker actually compromise the system right now?"&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;The distinction matters in CI pipelines. A list of medium-severity findings does not tell you whether to block a deployment. A confirmed exploit does.&lt;/p&gt;

&lt;p&gt;Breach Gate scores every finding using a multiplicative formula:&lt;br&gt;
Risk = Reachability x Exploitability x Impact x Confidence&lt;/p&gt;

&lt;p&gt;A vulnerability that is hard to reach, has no working proof-of-concept, and low confidence stays at a low risk score. A confirmed exploit with a working payload gets boosted to critical regardless of how the individual factors score.&lt;/p&gt;
&lt;h2&gt;
  
  
  What It Tests
&lt;/h2&gt;
&lt;h3&gt;
  
  
  AI-Assisted Behavioral Testing
&lt;/h3&gt;

&lt;p&gt;The scanner generates OWASP-based test cases per endpoint and executes them against your live API. Two mechanisms keep false positives low:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Baseline diffing&lt;/strong&gt; -- a benign request is sent to each endpoint before any attack probes. Response tokens that appear in the baseline are filtered from vulnerability indicators, eliminating a large class of false positives where generic words like "error" or "id" triggered matches.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Time-based blind injection&lt;/strong&gt; -- responses delayed more than 3 seconds AND more than 3 times the baseline timing are flagged as potential blind SQL or command injection, which cannot be detected from response bodies alone.&lt;/p&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Attack categories covered out of the box:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Category&lt;/th&gt;
&lt;th&gt;Detection Method&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;SQL Injection&lt;/td&gt;
&lt;td&gt;Response body, error text, blind timing&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Command Injection&lt;/td&gt;
&lt;td&gt;Response body, blind timing&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;XSS&lt;/td&gt;
&lt;td&gt;Reflected probe in response&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Broken Access Control&lt;/td&gt;
&lt;td&gt;Status code shift vs baseline&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;SSRF&lt;/td&gt;
&lt;td&gt;Cloud metadata endpoint probing&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Mass Assignment&lt;/td&gt;
&lt;td&gt;Privilege field echo in response&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;JWT Attacks&lt;/td&gt;
&lt;td&gt;Algorithm confusion, claim tampering, expired token&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Path Traversal&lt;/td&gt;
&lt;td&gt;File content indicators in response&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;
&lt;h3&gt;
  
  
  Static Analysis via Trivy
&lt;/h3&gt;

&lt;p&gt;Scans your source code and dependencies for known CVEs, exposed secrets, and misconfigurations. Results feed into the same scoring pipeline as dynamic findings.&lt;/p&gt;
&lt;h3&gt;
  
  
  Container Scanning
&lt;/h3&gt;

&lt;p&gt;Pulls your Docker image and runs Trivy against the filesystem and OS packages. Findings are correlated with the API endpoint they affect where possible.&lt;/p&gt;
&lt;h3&gt;
  
  
  GraphQL Security Probing
&lt;/h3&gt;

&lt;p&gt;For GraphQL APIs, Breach Gate runs five dedicated probes: introspection exposure, depth-limit denial of service, field suggestion enumeration, variable injection, and IDOR by ID enumeration.&lt;/p&gt;
&lt;h3&gt;
  
  
  Dynamic Testing via OWASP ZAP
&lt;/h3&gt;

&lt;p&gt;When ZAP is available (local or Docker), the scanner runs an active API scan and merges the results with findings from other scanners.&lt;/p&gt;
&lt;h2&gt;
  
  
  The Output
&lt;/h2&gt;

&lt;p&gt;SECURITY VERDICT:&lt;br&gt;
╔════════════════════════════════════════════════════════╗&lt;br&gt;
║            UNSAFE TO DEPLOY                            ║&lt;br&gt;
╚════════════════════════════════════════════════════════╝&lt;br&gt;
Reason: Confirmed exploitation: SQL Injection, Command Injection.&lt;br&gt;
Active attacks succeeded during testing.&lt;br&gt;
2 CONFIRMED EXPLOITS:&lt;br&gt;
SQL Injection on POST /api/data&lt;br&gt;
Command Injection on POST /api/execute&lt;br&gt;
Attack Surface (by endpoint):&lt;br&gt;
POST /api/execute&lt;br&gt;
Risk: 95%&lt;br&gt;
Command Injection&lt;br&gt;
Attack chain: Command Injection -&amp;gt; Full System Compromise&lt;br&gt;
POST /api/data&lt;br&gt;
Risk: 90%&lt;br&gt;
SQL Injection&lt;br&gt;
Attack chain: Injection -&amp;gt; System Compromise&lt;/p&gt;

&lt;p&gt;Reports are generated in JSON, Markdown, SARIF, and HTML. The HTML report includes a category filter bar and one-click evidence copy.&lt;/p&gt;
&lt;h2&gt;
  
  
  CI Integration
&lt;/h2&gt;

&lt;p&gt;Breach Gate is published to the GitHub Marketplace as a composite action:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;Run Breach Gate&lt;/span&gt;
  &lt;span class="na"&gt;uses&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;epten08/breach-gate@v1&lt;/span&gt;
  &lt;span class="na"&gt;with&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;target&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;${{ vars.STAGING_API_URL }}&lt;/span&gt;
    &lt;span class="na"&gt;anthropic-api-key&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;${{ secrets.ANTHROPIC_API_KEY }}&lt;/span&gt;
    &lt;span class="na"&gt;format&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;json,markdown,sarif&lt;/span&gt;
    &lt;span class="na"&gt;output&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;security-reports&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The action outputs a &lt;code&gt;verdict&lt;/code&gt; value (&lt;code&gt;PASS&lt;/code&gt; or &lt;code&gt;FAIL&lt;/code&gt;) that downstream steps can consume, and the SARIF report integrates directly with GitHub Code Scanning.&lt;/p&gt;

&lt;p&gt;For teams not on GitHub, the same scan runs via npm:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npx breach-gate scan &lt;span class="nt"&gt;--target&lt;/span&gt; https://staging.api.example.com &lt;span class="nt"&gt;--ci&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The &lt;code&gt;--ci&lt;/code&gt; flag sets a non-zero exit code on &lt;code&gt;UNSAFE&lt;/code&gt; verdicts, which blocks the deployment step in any CI system.&lt;/p&gt;

&lt;h2&gt;
  
  
  Watch Mode
&lt;/h2&gt;

&lt;p&gt;For continuous environments, a watch command runs scans on a configurable interval and diffs findings between runs:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;breach-gate watch &lt;span class="nt"&gt;--target&lt;/span&gt; http://localhost:3000 &lt;span class="nt"&gt;--interval&lt;/span&gt; 300
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;New findings are logged as warnings. Resolved findings are logged as informational. This is useful for staging environments that receive frequent deployments.&lt;/p&gt;

&lt;h2&gt;
  
  
  Suppressing Known Findings
&lt;/h2&gt;

&lt;p&gt;Teams working on legacy APIs often have accepted known issues that are tracked. A &lt;code&gt;.breachgateignore&lt;/code&gt; file prevents those from blocking pipelines:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;suppress&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;id&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;finding-abc123"&lt;/span&gt;
    &lt;span class="na"&gt;reason&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Tracked&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;in&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;JIRA-456,&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;fix&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;scheduled&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;for&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;next&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;sprint"&lt;/span&gt;
    &lt;span class="na"&gt;expires&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;2026-06-01"&lt;/span&gt;

  &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;pattern&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Missing&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;security&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;header"&lt;/span&gt;
    &lt;span class="na"&gt;endpoint&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;/api/health"&lt;/span&gt;
    &lt;span class="na"&gt;reason&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Health&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;check&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;endpoint,&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;intentionally&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;minimal&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;headers"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Rules with an &lt;code&gt;expires&lt;/code&gt; date automatically stop suppressing after that date, which prevents forgotten suppressions from masking real regressions.&lt;/p&gt;

&lt;h2&gt;
  
  
  Getting Started
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Install globally&lt;/span&gt;
npm &lt;span class="nb"&gt;install&lt;/span&gt; &lt;span class="nt"&gt;-g&lt;/span&gt; breach-gate

&lt;span class="c"&gt;# Run against your API&lt;/span&gt;
breach-gate scan &lt;span class="nt"&gt;--target&lt;/span&gt; http://localhost:3000

&lt;span class="c"&gt;# Run the built-in demo to see a full vulnerable API scan&lt;/span&gt;
git clone https://github.com/epten08/breach-gate
&lt;span class="nb"&gt;cd &lt;/span&gt;breach-gate
npm &lt;span class="nb"&gt;install
&lt;/span&gt;npm run demo        &lt;span class="c"&gt;# starts a deliberately vulnerable API&lt;/span&gt;
npm run scan        &lt;span class="c"&gt;# scans it&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;An OpenAPI spec can be passed to give the scanner full endpoint coverage:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;breach-gate scan &lt;span class="nt"&gt;--target&lt;/span&gt; http://localhost:3000 &lt;span class="nt"&gt;--openapi&lt;/span&gt; ./openapi.yml
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Without a spec, the scanner infers common endpoint patterns and uses them as a starting point.&lt;/p&gt;

&lt;h2&gt;
  
  
  Lessons Learned
&lt;/h2&gt;

&lt;p&gt;Reducing the false positive rate was more challenging than building the detection logic. Early versions flagged nearly everything because words like "error", "id", and "success" appeared in every API response. Combining baseline diffing with restricting body matches to 2xx responses brought the false positive rate to a manageable level.&lt;/p&gt;

&lt;p&gt;Prompt design for the Anthropic API also required careful iteration. Prompts using direct offensive language were blocked by content filtering. Reframing the same tests as "authorized penetration testing" and "OWASP-based assessment probes" passed the filter while generating identical test cases.&lt;/p&gt;

&lt;h2&gt;
  
  
  Links
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;GitHub: &lt;a href="https://github.com/epten08/breachgate" rel="noopener noreferrer"&gt;https://github.com/epten08/breachgate&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;npm: &lt;a href="https://www.npmjs.com/package/breach-gate" rel="noopener noreferrer"&gt;https://www.npmjs.com/package/breach-gate&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;GitHub Marketplace: &lt;a href="https://github.com/marketplace/actions/breach-gate" rel="noopener noreferrer"&gt;https://github.com/marketplace/actions/breach-gate&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Contributions, bug reports, and false positive reports are welcome. The contributing guide covers how to add new attack categories and scanners.&lt;/p&gt;

</description>
      <category>security</category>
      <category>devsecops</category>
      <category>opensource</category>
      <category>api</category>
    </item>
    <item>
      <title>What K6 Load Testing Found in My POS SaaS API (Before It Hit Production)</title>
      <dc:creator>Arnold M S</dc:creator>
      <pubDate>Thu, 30 Apr 2026 08:33:32 +0000</pubDate>
      <link>https://dev.to/epten08/what-k6-load-testing-found-in-my-pos-saas-api-before-it-hit-production-58b0</link>
      <guid>https://dev.to/epten08/what-k6-load-testing-found-in-my-pos-saas-api-before-it-hit-production-58b0</guid>
      <description>&lt;h2&gt;
  
  
  Background
&lt;/h2&gt;

&lt;p&gt;I'm a solo developer building a multi-tenant Point of Sale SaaS system,the kind where dozens of shops might be processing sales at the same time through a shared Laravel API. My unit tests passed. My Postman collection passed. Everything worked perfectly when I tested it by hand.&lt;/p&gt;

&lt;p&gt;So naturally, I decided to see what happened when I threw 50 simultaneous virtual users at it.&lt;/p&gt;

&lt;p&gt;The answer: chaos.&lt;/p&gt;

&lt;p&gt;This post walks through exactly what broke, the root causes, and the fixes. The patterns here deadlocks, N+1 queries, race conditions on unique values, synchronous work on hot paths — show up in almost every database-backed API. I just didn't know they were hiding in mine until I looked.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Setup
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Stack:&lt;/strong&gt; Laravel 11 + MySQL + Redis queues, running in Docker Compose&lt;br&gt;
&lt;strong&gt;Test tool:&lt;/strong&gt; &lt;a href="https://k6.io/" rel="noopener noreferrer"&gt;K6&lt;/a&gt; v0.53.0&lt;br&gt;
&lt;strong&gt;Scenarios:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;multi-tenant.js&lt;/code&gt; — multiple tenants each with concurrent cashiers&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;busy-shop.js&lt;/code&gt; — one shop, 50 virtual users (VUs) ramping up over 10 minutes&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;end-of-day.js&lt;/code&gt; - simultaneous cash-ups across all tenants (not yet run)
The critical path I was testing: &lt;code&gt;POST /api/sales&lt;/code&gt; — the sale transaction that validates items, reduces inventory, records payment, creates journal entries, and updates the till balance.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Before I could even run a test, I had to fix a long list of test harness issues (wrong Docker volume paths, K6 SharedArray type mismatches, Windows Git Bash path mangling, tenant soft-delete not cleaning up properly). If you're setting up K6 with Docker on Windows, budget extra time for that part.I'm considering publishing my Docker Compose + K6 configuration separately if there's interest.&lt;/p&gt;


&lt;h2&gt;
  
  
  Bug 1: MySQL Deadlock on &lt;code&gt;tills.current_balance&lt;/code&gt; — Critical
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Error rate at 50 VUs:&lt;/strong&gt; 2.76% of all sales&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;SQLSTATE&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;40001&lt;/span&gt;&lt;span class="p"&gt;]:&lt;/span&gt; &lt;span class="n"&gt;Serialization&lt;/span&gt; &lt;span class="n"&gt;failure&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;1213&lt;/span&gt; &lt;span class="n"&gt;Deadlock&lt;/span&gt; &lt;span class="k"&gt;found&lt;/span&gt; &lt;span class="k"&gt;when&lt;/span&gt; &lt;span class="n"&gt;trying&lt;/span&gt; &lt;span class="k"&gt;to&lt;/span&gt;
&lt;span class="k"&gt;get&lt;/span&gt; &lt;span class="k"&gt;lock&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="n"&gt;try&lt;/span&gt; &lt;span class="n"&gt;restarting&lt;/span&gt; &lt;span class="n"&gt;transaction&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Every cashier hitting "Complete Sale" had about a 1-in-36 chance of getting an error. The sale would silently roll back and the cashier would have to retry.&lt;/p&gt;

&lt;h3&gt;
  
  
  Why it happened
&lt;/h3&gt;

&lt;p&gt;The sale transaction went roughly:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;code&gt;Sale::create()&lt;/code&gt; → triggers &lt;code&gt;SaleObserver::created()&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;Observer immediately increments &lt;code&gt;tills.current_balance&lt;/code&gt; (acquires exclusive lock on &lt;code&gt;tills&lt;/code&gt; row)&lt;/li&gt;
&lt;li&gt;Loop through items → &lt;code&gt;InventoryRepository::reduceStock()&lt;/code&gt; → &lt;code&gt;SELECT ... FOR UPDATE&lt;/code&gt; on &lt;code&gt;inventories&lt;/code&gt; rows
Under concurrent load with 50 VUs sharing 3 tills, this created a classic circular dependency:&lt;/li&gt;
&lt;/ol&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Transaction A&lt;/strong&gt; (cash, products [1, 3]): holds &lt;code&gt;tills[1]&lt;/code&gt; lock → waiting for &lt;code&gt;inventories[product_3]&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Transaction B&lt;/strong&gt; (cash, products [3, 1]): holds &lt;code&gt;inventories[product_3]&lt;/code&gt; lock → waiting for &lt;code&gt;tills[1]&lt;/code&gt;
MySQL breaks the cycle by rolling back one transaction. The "victim" returns HTTP 400 to the cashier.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The lock on &lt;code&gt;tills&lt;/code&gt; was being held for ~500ms  the entire duration of inventory processing because it was acquired inside the open transaction, before the inventory loop.&lt;/p&gt;

&lt;h3&gt;
  
  
  The fix
&lt;/h3&gt;

&lt;p&gt;Defer the till balance update to after the transaction commits:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight php"&gt;&lt;code&gt;&lt;span class="c1"&gt;// app/Observers/SaleObserver.php&lt;/span&gt;
&lt;span class="k"&gt;public&lt;/span&gt; &lt;span class="k"&gt;function&lt;/span&gt; &lt;span class="n"&gt;created&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kt"&gt;Sale&lt;/span&gt; &lt;span class="nv"&gt;$sale&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt; &lt;span class="kt"&gt;void&lt;/span&gt;
&lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$sale&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;isCompleted&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="nv"&gt;$sale&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;payment_method&lt;/span&gt; &lt;span class="o"&gt;===&lt;/span&gt; &lt;span class="s1"&gt;'cash'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="no"&gt;DB&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;afterCommit&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;function&lt;/span&gt; &lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="k"&gt;use&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$sale&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="nv"&gt;$sale&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;till&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;increment&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'current_balance'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nv"&gt;$sale&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;total&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
        &lt;span class="p"&gt;});&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;DB::afterCommit()&lt;/code&gt; fires after &lt;code&gt;DB::commit()&lt;/code&gt; completes. The &lt;code&gt;tills&lt;/code&gt; lock now lasts ~1ms (a single &lt;code&gt;UPDATE&lt;/code&gt; outside any transaction) instead of ~500ms.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Result:&lt;/strong&gt; 0 deadlocks at 50 VUs.&lt;/p&gt;




&lt;h2&gt;
  
  
  Bug 2: N+1 Queries in the Sale Service — High
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Before fix:&lt;/strong&gt; sale latency minimum 377ms, p(95) = 1.74s at 6 VUs&lt;/p&gt;

&lt;p&gt;This one wasn't crashing anything it was just quietly making every sale slower and slower as load increased.&lt;/p&gt;

&lt;h3&gt;
  
  
  Why it happened
&lt;/h3&gt;

&lt;p&gt;&lt;code&gt;createSale()&lt;/code&gt; had two separate loops over the sale items. Each loop called &lt;code&gt;productRepository-&amp;gt;find($item['product_id'])&lt;/code&gt; independently — so a 4-item sale triggered 8 product queries. Additionally, &lt;code&gt;inventoryRepository-&amp;gt;getQuantity()&lt;/code&gt; was called once in the validation loop and again in the deduction loop, doubling the inventory queries.&lt;/p&gt;

&lt;p&gt;With 50 concurrent users each processing 3–5 item sales, the database was doing 8–10 queries per sale where 1 was needed.&lt;/p&gt;

&lt;h3&gt;
  
  
  The fix
&lt;/h3&gt;

&lt;p&gt;Batch-load all products before the loops, and cache inventory quantities from the validation pass:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight php"&gt;&lt;code&gt;&lt;span class="c1"&gt;// One query for all products in this sale&lt;/span&gt;
&lt;span class="nv"&gt;$productMap&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Product&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;whereIn&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'id'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nb"&gt;array_column&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$data&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'items'&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="s1"&gt;'product_id'&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
    &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;keyBy&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'id'&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="nv"&gt;$stockMap&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[];&lt;/span&gt;

&lt;span class="c1"&gt;// Validation loop: cache stock alongside the check&lt;/span&gt;
&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$product&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;track_inventory&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nv"&gt;$availableStock&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nv"&gt;$this&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;inventoryRepository&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;getQuantity&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mf"&gt;...&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="nv"&gt;$stockMap&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nv"&gt;$item&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'product_id'&lt;/span&gt;&lt;span class="p"&gt;]]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nv"&gt;$availableStock&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="c1"&gt;// Deduction loop: no DB hits&lt;/span&gt;
&lt;span class="nv"&gt;$product&lt;/span&gt;        &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nv"&gt;$productMap&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$item&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'product_id'&lt;/span&gt;&lt;span class="p"&gt;]);&lt;/span&gt;
&lt;span class="nv"&gt;$quantityBefore&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nv"&gt;$stockMap&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nv"&gt;$item&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'product_id'&lt;/span&gt;&lt;span class="p"&gt;]]&lt;/span&gt; &lt;span class="o"&gt;??&lt;/span&gt; &lt;span class="mf"&gt;...&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Result:&lt;/strong&gt; Sale minimum latency: 377ms → 253ms (−33%). p(95) at 6 VUs: 1.74s → 814ms (−53%).&lt;/p&gt;




&lt;h2&gt;
  
  
  Bug 3: Synchronous Accounting on the HTTP Hot Path — High
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;120–140ms added to every single sale response&lt;/strong&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Why it happened
&lt;/h3&gt;

&lt;p&gt;After the sale committed, &lt;code&gt;SaleService&lt;/code&gt; called &lt;code&gt;saleAccountingService-&amp;gt;recordSaleEntry($sale)&lt;/code&gt; synchronously on the same HTTP worker thread that the cashier's request was waiting on. That method made 4–6 separate &lt;code&gt;ChartOfAccount&lt;/code&gt; lookups, inserted &lt;code&gt;JournalEntry&lt;/code&gt; and &lt;code&gt;JournalEntryLine&lt;/code&gt; rows, and did all of this while blocking the response.&lt;/p&gt;

&lt;p&gt;The cashier doesn't need to wait for the accounting ledger to update before the "Sale complete" screen appears.&lt;/p&gt;

&lt;h3&gt;
  
  
  The fix
&lt;/h3&gt;

&lt;p&gt;Dispatch a queued job instead:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight php"&gt;&lt;code&gt;&lt;span class="c1"&gt;// Before (blocking):&lt;/span&gt;
&lt;span class="nv"&gt;$this&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;saleAccountingService&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;recordSaleEntry&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$sale&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

&lt;span class="c1"&gt;// After (async):&lt;/span&gt;
&lt;span class="nc"&gt;RecordSaleEntryJob&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;dispatch&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$sale&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;id&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The existing Redis queue worker picks it up in ~100–140ms. Journal entries are still written correctly verified by inspecting the database after a 60-sale test run. The cashier just doesn't have to wait for it.&lt;/p&gt;

&lt;p&gt;One thing to consider with this approach: if the job fails, the sale exists but the journal entries don't. I handle this with Laravel's built-in job retry mechanism (&lt;code&gt;tries = 3&lt;/code&gt; with exponential backoff) and a failed job alert that notifies me via the existing monitoring. In practice, journal entry creation is simple enough that transient failures (brief DB hiccups) resolve on retry, and permanent failures (code bugs) get caught in development.&lt;/p&gt;

&lt;p&gt;I also added an instance-level &lt;code&gt;$accountCache&lt;/code&gt; to &lt;code&gt;SaleAccountingService&lt;/code&gt; to avoid redundant &lt;code&gt;ChartOfAccount&lt;/code&gt; lookups within a single job execution.&lt;/p&gt;




&lt;h2&gt;
  
  
  Bug 4: Duplicate &lt;code&gt;sale_number&lt;/code&gt; Under Concurrency — Medium
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;SQLSTATE&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;23000&lt;/span&gt;&lt;span class="p"&gt;]:&lt;/span&gt; &lt;span class="n"&gt;Integrity&lt;/span&gt; &lt;span class="k"&gt;constraint&lt;/span&gt; &lt;span class="n"&gt;violation&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;1062&lt;/span&gt; &lt;span class="n"&gt;Duplicate&lt;/span&gt; &lt;span class="n"&gt;entry&lt;/span&gt; &lt;span class="s1"&gt;'SAL-...'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Why it happened
&lt;/h3&gt;

&lt;p&gt;&lt;code&gt;generateSaleNumber()&lt;/code&gt; used PHP's &lt;code&gt;uniqid()&lt;/code&gt;, which is time-based. Under 50 concurrent VUs, multiple calls within the same microsecond returned identical values.&lt;/p&gt;

&lt;h3&gt;
  
  
  The fix
&lt;/h3&gt;

&lt;p&gt;Replace with cryptographically random bytes and rely on the database's UNIQUE constraint as the true guarantee:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight php"&gt;&lt;code&gt;&lt;span class="k"&gt;protected&lt;/span&gt; &lt;span class="k"&gt;function&lt;/span&gt; &lt;span class="n"&gt;generateSaleNumber&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt; &lt;span class="kt"&gt;string&lt;/span&gt;
&lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nv"&gt;$prefix&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;'SAL'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="nv"&gt;$date&lt;/span&gt;   &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;date&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'Ymd'&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="nv"&gt;$maxAttempts&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$i&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="nv"&gt;$i&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="nv"&gt;$maxAttempts&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="nv"&gt;$i&lt;/span&gt;&lt;span class="o"&gt;++&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="k"&gt;try&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="nv"&gt;$random&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;strtoupper&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;substr&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;bin2hex&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;random_bytes&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;)),&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;6&lt;/span&gt;&lt;span class="p"&gt;));&lt;/span&gt;
            &lt;span class="nv"&gt;$number&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="nv"&gt;$prefix&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;-&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="nv"&gt;$date&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;-&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="nv"&gt;$random&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

            &lt;span class="c1"&gt;// The UNIQUE constraint on sale_number is the real safety net.&lt;/span&gt;
            &lt;span class="c1"&gt;// This existence check is an optimistic pre-filter to avoid&lt;/span&gt;
            &lt;span class="c1"&gt;// hitting the constraint in the common case.&lt;/span&gt;
            &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="nc"&gt;Sale&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;where&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'sale_number'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nv"&gt;$number&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="nf"&gt;exists&lt;/span&gt;&lt;span class="p"&gt;())&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
                &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nv"&gt;$number&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
            &lt;span class="p"&gt;}&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;catch&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;QueryException&lt;/span&gt; &lt;span class="nv"&gt;$e&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$i&lt;/span&gt; &lt;span class="o"&gt;===&lt;/span&gt; &lt;span class="nv"&gt;$maxAttempts&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;throw&lt;/span&gt; &lt;span class="nv"&gt;$e&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
            &lt;span class="c1"&gt;// Duplicate key → retry with a new random value&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="k"&gt;throw&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;\RuntimeException&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'Failed to generate unique sale number'&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;6 hex characters = 16.7 million combinations per day. The probability of a collision across 4,000 daily sales is ~0.05%. The &lt;code&gt;exists()&lt;/code&gt; check handles the common case, but under true concurrency, two transactions can pass the check simultaneously that's why the UNIQUE constraint and the retry-on-exception are there as the actual guarantee.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Lesson:&lt;/strong&gt; &lt;code&gt;uniqid()&lt;/code&gt; is not unique under concurrency. Any identifier you generate at the application layer and store in a UNIQUE column needs either a random component with enough entropy, or a database-side sequence/auto-increment. And the database constraint, not an application-level check, must be your source of truth.&lt;/p&gt;




&lt;h2&gt;
  
  
  Bug 5: Duplicate &lt;code&gt;inventory_adjustments.reference_number&lt;/code&gt; — Medium
&lt;/h2&gt;

&lt;p&gt;This one had two causes that had to be fixed separately.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Cause A:&lt;/strong&gt; The reference was built as &lt;code&gt;'SALE-' . $sale-&amp;gt;sale_number . '-' . $item['product_id']&lt;/code&gt;. If the same product appeared twice in a sale, two items produced an identical reference.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Fix A:&lt;/strong&gt; Add a 1-based index suffix: &lt;code&gt;'SALE-' . $sale-&amp;gt;sale_number . '-' . $product_id . '-' . ($idx + 1)&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Cause B:&lt;/strong&gt; The K6 test payload builder was picking products randomly with replacement, so it could include the same product twice which exposed the above bug. This was a test harness issue, not an application bug, but it was testing a valid edge case.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Fix B:&lt;/strong&gt; Deduplicate product selection in K6:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;usedIds&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;Set&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
&lt;span class="k"&gt;do&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nx"&gt;product&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;randomProduct&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;products&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;while &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;usedIds&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;has&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;product&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;id&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="nx"&gt;attempts&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="mi"&gt;20&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="nx"&gt;usedIds&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;add&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;product&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;id&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  Bug 6: Inventory Lock Ordering — Low (Latent)
&lt;/h2&gt;

&lt;p&gt;This one didn't cause observable failures during the test because Bug 1 was the active deadlock source. But it was sitting there waiting.&lt;/p&gt;

&lt;h3&gt;
  
  
  Why it was a risk
&lt;/h3&gt;

&lt;p&gt;&lt;code&gt;SELECT ... FOR UPDATE&lt;/code&gt; locks inventory rows in the order items appear in the request payload which is random. Two concurrent transactions with overlapping products in reverse order create a textbook circular-wait deadlock.&lt;/p&gt;

&lt;h3&gt;
  
  
  The fix
&lt;/h3&gt;

&lt;p&gt;Sort items by &lt;code&gt;product_id&lt;/code&gt; before both loops:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight php"&gt;&lt;code&gt;&lt;span class="nb"&gt;usort&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$data&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'items'&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="k"&gt;fn&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$a&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nv"&gt;$b&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="nv"&gt;$a&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'product_id'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;=&amp;gt;&lt;/span&gt; &lt;span class="nv"&gt;$b&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'product_id'&lt;/span&gt;&lt;span class="p"&gt;]);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;All transactions now acquire inventory locks in ascending &lt;code&gt;product_id&lt;/code&gt; order. Circular waits on the inventory dimension are impossible.&lt;/p&gt;

&lt;p&gt;This is a standard database technique: if multiple transactions need locks on the same set of rows, they must acquire them in a consistent order.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Numbers
&lt;/h2&gt;

&lt;p&gt;All comparisons below are at &lt;strong&gt;50 VUs over a 10-minute run&lt;/strong&gt; to keep the baseline and post-fix numbers directly comparable.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Metric&lt;/th&gt;
&lt;th&gt;Before&lt;/th&gt;
&lt;th&gt;After&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;http_req_failed&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;1.44%&lt;/td&gt;
&lt;td&gt;0.00%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Sale error rate&lt;/td&gt;
&lt;td&gt;2.76%&lt;/td&gt;
&lt;td&gt;0.00%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;All checks passed&lt;/td&gt;
&lt;td&gt;98.55%&lt;/td&gt;
&lt;td&gt;100.00%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Sale p(95)&lt;/td&gt;
&lt;td&gt;3.32s&lt;/td&gt;
&lt;td&gt;2.07s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Sale minimum&lt;/td&gt;
&lt;td&gt;377ms&lt;/td&gt;
&lt;td&gt;253ms&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Deadlock errors&lt;/td&gt;
&lt;td&gt;~68 / 2,457 sales&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;




&lt;h2&gt;
  
  
  What I Learned
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Unit tests cannot find concurrency bugs.&lt;/strong&gt; Every one of these bugs was invisible to my test suite because tests run sequentially. Deadlocks, race conditions on unique values, and lock ordering issues only appear when multiple transactions run simultaneously.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Load testing reveals the real hot path.&lt;/strong&gt; Profiling under load showed that synchronous accounting, redundant queries, and observer side effects were all adding up on a code path that runs on every single sale. None of those costs were obvious from reading the code.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;uniqid()&lt;/code&gt; is not unique.&lt;/strong&gt; At any meaningful concurrency, &lt;code&gt;uniqid()&lt;/code&gt; generates collisions. Use &lt;code&gt;random_bytes()&lt;/code&gt; for application-layer identifiers stored in unique-constrained columns,and let the database constraint be the final enforcer, not an application-level &lt;code&gt;exists()&lt;/code&gt; check.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Observers that do DB writes inside a transaction are a trap.&lt;/strong&gt; Eloquent observers fire during the wrapping transaction. Any row lock acquired in an observer is held for the full transaction duration,including any slower work that happens after. Use &lt;code&gt;DB::afterCommit()&lt;/code&gt; for side effects that don't need to be part of the main transaction.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Consistent lock ordering prevents deadlocks.&lt;/strong&gt; If multiple transactions acquire locks on the same rows, sorting by primary key before processing guarantees they'll always request locks in the same order, making circular waits impossible.&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  6. &lt;strong&gt;The test harness itself has bugs.&lt;/strong&gt; About half of my debugging time was spent on the test infrastructure (Docker volumes, Windows path issues, tenant cleanup, K6 API quirks) before I could even get a clean test run. Budget for this.
&lt;/h2&gt;

&lt;h2&gt;
  
  
  What's Still Open
&lt;/h2&gt;

&lt;p&gt;The &lt;code&gt;end-of-day.js&lt;/code&gt; scenario,simultaneous cash-ups across all tenants hasn't run yet. End-of-day is the highest-risk moment: all shops closing, all GL accounts being touched at once. That's next.&lt;/p&gt;

&lt;p&gt;There's also a latent &lt;code&gt;entry_number&lt;/code&gt; collision in the accounting service that's going to need the same &lt;code&gt;random_bytes&lt;/code&gt; treatment as the sale number fix. Found it in the queue worker logs; haven't fixed it yet.&lt;/p&gt;




&lt;p&gt;If you're building any kind of transactional API and haven't run a load test, I'd strongly recommend it. The bugs above weren't edge cases, they were waiting to hit real customers on any moderately busy day.&lt;/p&gt;




</description>
      <category>k6</category>
      <category>testing</category>
      <category>performance</category>
      <category>saas</category>
    </item>
  </channel>
</rss>
