DEV Community: Denis Morozov

Your Retry Budget Is Not a Safety Net

Denis Morozov — Tue, 18 Aug 2026 21:40:12 +0000

*Second in a series on The Factory. Previously: The Factory That Merged 37 Tasks. The harness is at github.com/frozer/factory.

The public description of my task harness ends on a claim: a packet that's wrong about the world fails identically on every retry.

That sentence cost me four dead tasks and nine commits spent repairing task definitions instead of writing code. It reads like something you'd arrive at by thinking. I arrived at it by watching the same failure scroll past three times in a row.

What three attempts is actually good for

max_attempts = 3 felt like obvious hygiene. Models are stochastic. Sometimes a run goes sideways for no reason you can name — a bad turn, a truncated response, a tool call that gets refused. Retry it and it works. That's real, and a retry budget handles it well.

The strength is exactly the constraint. A retry budget assumes the next attempt will differ from the last one. It buys you a second sample from a distribution.

But a retry doesn't hand the model a fresh situation. It hands it the same packet back. Same file, same claims, same instructions. If the packet says a file lives at a path where no file lives, attempt three fails precisely where attempt one did, and the only thing three attempts bought was three times the bill.

Failure without variance isn't flakiness. It's a specification defect wearing a reliability costume.

Nobody had opened the files

Here's what that looked like in practice.

B03 was a loader for a national census dataset. Three attempts, all burned, all against a file shape that existed nowhere: wrong directory, wrong filenames, and a Data / Valor JSON envelope that appears nowhere in the actual data tree. Every attempt produced a parser for a document that doesn't exist.

The packet was the defect, not the model. Nobody had opened the actual files before cutting it.

Rewritten from the real JSON, the truth was a flat metadata/data envelope, four files — one of which shouldn't be loaded at all — and a long-format pivot with genuine missing rows. Nothing about that is inferable from a design document. It's only knowable by opening the file.

The same check applied to two sibling packets turned up a schema mismatch, a missed pivot, and a real design collision: two source files sharing five of six category labels verbatim, with no column to tell them apart. Loading both, as the packet instructed, would have silently overwritten data. Also, the packet's confident claim about Spanish decimal commas was simply false for that data — zero commas in any total column, across all three files.

A11 is the cleaner case, because it removes the model from the argument entirely. Its invariant required two fields — a source SRID and an attribution string — routed through a typed config. Those fields did not exist on the type. And the file that defines the type was outside A11's editable set.

So no diff the implementer was permitted to produce could satisfy that invariant. Not a hard task. An impossible one, three times, at full price.

C09 was the expensive kind. Its steps read build → validate → write, which is a sensible order and reads perfectly. But the build call persisted internally before returning — so build had already written. All three attempts validated after the write, and dutifully logged the failure that the invariant existed to prevent.

Nothing in the worktree tells a model any of this. It fails, gets handed the same packet, and fails the same way.

The rule

A retry budget is for variance, not for error.

Before you raise max_attempts, read the three failures side by side and ask one question: did they differ?

If attempt three failed differently from attempt one, retries are doing their job. If all three failed identically, you are not looking at an unreliable model. You are looking at an instruction that cannot be followed, and you are paying full price to have that confirmed repeatedly.

The retry loop is the last place to fix this and the first place everyone looks.

What it forced

The fix wasn't a better implementer prompt. It was admitting that the cheapest place to catch a defective packet is before any model has cost anything.

So the planning step got its own two agents. One cuts packets from the spec. A second one — with no ability to edit anything, because a run that can edit what it's judging will eventually repair a defect quietly instead of reporting it — assesses each packet and rules on every checkable claim in it: held, violated, or unverifiable, each with a path:line or a command's output. The queue now refuses any packet without a passing assessment pinned to that packet's exact hash.

The cutter works from a ground-truth ladder: files on disk first, then the code the task will call, then what earlier packets produce, then the toolchain — and the spec ranks below all of them, as evidence of intent rather than of fact.

That ordering is the whole lesson. A spec tells you what someone wanted. Only the file tells you what's there.

The part I didn't expect

I backtested the assessor against those nine repair commits as a labelled set. All five known-bad packets came back recut, each naming the same defect the human fix had found. The two I'd already repaired came back ready.

Three packets that had shipped came back recut — for true reasons.

And assessing A11 surfaced a live bug in merged code: a loader calling a snapshot_id property on a directory path, where the property reads a single file's bytes. That code had passed the linter, passed the tests, passed review, and merged.

No test could have caught it. The dataset isn't in this checkout, so the offline suite never reaches the call.

A green test suite told me that code was fine. A model reading the packet next to the code found it in one pass.

The Factory That Merged 37 Tasks: What Happens When the Harness Stops Believing the Agent

Denis Morozov — Wed, 12 Aug 2026 12:17:13 +0000

Back in March I wrote that in an AI-native company, processes stop being folklore and become measurable artifacts — versioned, evaluated, improvable. That was a claim. This is the receipt.

Seven days in August. A harness I built drained a queue of 37 task packets. Each one implemented by a model in its own git worktree, verified by tests the harness ran itself, reviewed against invariants the packet declared by name, squash-merged onto the feature branch only if it survived all three.

172 files changed. 29,360 lines added. The test suite it was building went from 81 passing tests to 422. Direct cost: $3.89.

Thirty-seven packets, thirty-seven merges, and I did not read a single diff before it landed.

Where this came from

This didn't start as an idea. It started as a habit I'd been demoing to my team for the better part of a year.

The workflow was manual and it was simple: plan first, read the plan yourself, ask for the implementation, then open a fresh model with no memory of writing the code and ask it to assess what changed. Separately, git worktree so several agents could work different changes in the same project without stepping on each other. Nothing clever. It just worked better than letting one long conversation both write the code and reassure me about it.

I ran that by hand for months. Then the honest question: if the loop is always plan → implement → assess-with-a-clean-model, and I'm the one clicking through it forty times, what exactly am I contributing?

The answer was git worktree, a SQLite file, and about a week.

What a packet is

The unit of work isn't a prompt. It's a file in git.

+++
id = "B01"
slug = "adrh-loader"
goal = "add ingest/adrh_load.py — the ADRH income CSV, all levels, all years"
tier = "standard"
gate = "auto"
surface = "api"
spec_commit = "504abb9dca9e050fda5e85a665f5bd2b66c2944b"
spec_path = "docs/IMPLEMENTATION-S3-reporting-baseline.md"
requires = ["A08", "A10"]
max_attempts = 3
forbidden_paths = ["api/ingest/vintage.py", "api/tests/conftest.py", ...]

[[invariants]]
id = "thousands-separator"
critical = true
assert = "`16.429` loads as 16429, not 16.429. The dot is a thousands separator."

[[invariants]]
id = "empty-total-loads-as-null"
critical = true
assert = "An empty `Total` cell loads as NULL, never 0 and never a skipped row. 15,480 rows in the source have one."
+++

Two fields do most of the work.

spec_commit pins the packet to the exact revision of the design document it was cut from. If the spec moves, the packet is stale, and the harness refuses to run it rather than building something the design stopped asking for.

[[invariants]] matters more. Each one has an id, and the reviewer must return a verdict on every id — held, violated, or unverifiable — with a file:line behind it. A review that quietly skips one is rejected by the contract. Silence is not a pass.

Now look at what those two invariants actually say. Not "parse the CSV correctly." A specific dot in a specific number. A count — 15,480 rows — that somebody went and measured in the real file.

That precision isn't style. It turned out to be the entire difference between packets that worked and packets that burned every attempt they had. More on that next time.

The loop

ready ──claim──▶ running ──▶ gating ──green──▶ reviewing ──accept──▶ merging ──▶ done
  ▲                            │                    │                    │
  └────── needs_work ◀─────────┴── red ──────┴── revise ──┘         awaiting_human
                 │
                 └── attempts exhausted ──▶ blocked ──▶ the loop halts

Per task: claim it, check the spec hasn't moved, create a worktree, measure the test baseline, plant the supplied test files and commit them, implement, gate, review, merge.

Two exits are not done.

blocked halts the whole queue instead of skipping ahead. A task that failed three times has usually revealed something wrong with its packet, and running the next twenty on the same misunderstanding wastes more than it saves.

awaiting_human is where a packet goes when I marked it gate = "human" at cut time. Five of the 37 carried that flag, on one rule: the failure is silent and the blast radius is the whole stage. A migration that would need undoing on live data. A cache that never hits — every assertion still passes, only latency changes. A chart that draws backwards while the numbers printed next to it stay correct.

An automated green gate cannot certify an invariant that fails green. Pretending otherwise just merges it.

The one idea

Everything above is mechanism. The actual claim is one sentence:

The harness never believes the agent.

The implementer returns a verification block saying its tests pass. That block is recorded and ignored. ruff check, pytest, npm run lint and npm run build are run by the harness, in the worktree, and its parse of the output decides whether the work is even eligible for review.

Same principle at every layer. Test files are planted into the worktree and committed before the model starts, then declared off-limits — so git diff against that commit proves the model made the test pass instead of making it agree. The reviewer's accept is downgraded automatically over a red gate, a violated critical invariant, or a blocker finding, and the reason is written to a column instead of living in my head.

That's the difference between letting a model write your code and having a pipeline. Not the model. Not the prompt. Refusing to accept any actor's report of its own work as evidence about that work.

The first real run

Then I ran it for the first time, end to end, on one small task.

The model did its job. All five invariants held, 96 tests passing. And the harness could not get the work into git.

The first run turned up a cluster of bugs, every one of them mine. My favourite: the loop regenerates its status board after each merge and leaves the file modified — and the loop refuses to start on a dirty tree. It was dirtying its own working copy. It would have blocked its own next invocation.

A week building a governance system for other people's agents, and the first thing it governed into a corner was itself.

What it cost me

The work didn't disappear. It moved.

The harness is 5,741 lines of stdlib-only Python, plus 2,168 lines of tests for itself — 161 of them. No framework, no dependencies, runs on a bare python3. That's a real week of engineering to avoid reading 37 diffs.

Which is a terrible trade for 37 diffs. It's a good trade for a process that states its own failure conditions, records what every run cost, and can be pointed at the next thirty-seven.

And I got the economics wrong going in. My assumption was that a cheap model does the work and a stronger one checks it, so the checking is the rounding error. The first task I measured properly: $0.11 to implement, $0.18 to review. The review was the expensive half. I had built the whole thing on the opposite premise and found out on task one.

It's public now, Apache 2.0: github.com/frozer/factory

One caveat before someone finds it for me. It was built for — and has only ever run against — a repository with exactly two surfaces, an api/ and a webapp/. Point it anywhere else and it reports an environment fault. The toolchain is narrow and the README says so.

But the toolchain was never the interesting part. What generalises is the failure modes: what goes wrong when you let a model plan work for other models, and why the obvious safety net — just retry it — catches none of them.

I gave every task three attempts, because three attempts felt like obvious hygiene.

Four tasks in that queue burned all three. Every one of them failed identically each time.

The retry budget caught nothing.

The Redis That Timed Out: A Docker Subnet Overlap Hiding in Plain Sight

Denis Morozov — Mon, 13 Jul 2026 17:58:57 +0000

Postgres worked. Redis didn't. Same host, same Docker, same iptables, two containers started four minutes apart — and one of them refused to answer. This is the story of a TimeoutError that had nothing to do with Redis, and the one-line routing fact that explained everything.

The setup

My dev stack for a trading-desk API lives on an Alpine Linux VM under Windows 11 + Hyper-V. Inside Alpine, Docker runs two containers: Postgres and Redis, both port-published, both reachable from the host at 192.168.240.2. The Python app talks to Postgres happily. Then I wired up Redis for cache-aside and got:

redis.exceptions.TimeoutError: Timeout connecting to server

Not connection refused. Timeout. That distinction is the whole plot.

Refused = the packet arrived and something said "no." Wrong port, dead service.
Timeout = the packet left and nothing ever came back. It's being black-holed.

So the SYN was going somewhere it shouldn't. But where? Redis was up (docker exec redis redis-cli ping → PONG), the DNAT rule looked identical to the Postgres one, and Postgres — same host, same IP — was fine.

The clue was in `ifconfig`

Two interfaces on the Docker host, side by side:

docker0   inet addr:172.17.0.1   Mask:255.255.0.0     -> 172.17.0.0/16   (Redis lives here)
eth0      inet addr:172.17.6.90   Mask:255.255.240.0   -> 172.17.0.0/20   (Hyper-V network)

Read those two subnets again. docker0 claims 172.17.0.0/16. The Hyper-V-assigned eth0 sits on 172.17.0.0/20. They overlap. And Redis's container IP, 172.17.0.2, falls inside both.

When a client hit 192.168.240.2:6379, iptables DNAT'd the packet to 172.17.0.2 — correctly. Then the kernel had to route to 172.17.0.2, and it found two matching routes:

172.17.0.0/20 via eth0
172.17.0.0/16 via docker0

The kernel picks the longest prefix. /20 is more specific than /16, so eth0 wins. The packet was shoved out onto the Hyper-V LAN, hunting for a 172.17.0.2 that doesn't exist out there — instead of down docker0 to the container two hops away. Black hole. Timeout.

One command nailed it:

# ip route get 172.17.0.2
172.17.0.2 dev eth0  src 172.17.6.90

dev eth0. Not docker0. There's your bug, printed in one line.

Why Postgres got away with it

Here's the part that makes it feel like magic until you see it: Postgres never did anything special. It just happened to be started with docker compose, and Redis with a bare docker run.

Compose silently creates a dedicated user-defined bridge network per project, and those are allocated from Docker's address pool starting at 172.18.0.0/16 — the 172.17 range is reserved for the built-in docker0. So Postgres landed on 172.18.x, which collides with nothing. Redis, via docker run, fell onto docker0/172.17 — straight into the Hyper-V overlap.

Nobody configured this. The outcome hinged entirely on docker compose vs docker run. It's a latent trap that only springs when your host network and Docker's defaults happen to share a prefix — and Hyper-V's Default Switch re-rolls its subnet on almost every reboot, drawing from exactly the ranges Docker likes: 172.16.0.0/12 and 192.168.0.0/16.

The fix: evict Docker from the danger zone

The robust move isn't to dodge one subnet — it's to relocate all of Docker into the one big private block Hyper-V never touches: 10.0.0.0/8.

1. Pin Docker's pools in /etc/docker/daemon.json:

{
  "bip": "10.200.0.1/24",
  "default-address-pools": [
    { "base": "10.201.0.0/16", "size": 24 }
  ]
}

bip moves the default docker0 bridge; default-address-pools moves every Compose/user network. Nothing in 10.200/10.201 can ever share a prefix with a 172.x/192.168.x Hyper-V interface — even after the next reboot re-roll.

2. Make it deterministic by folding Redis into the same Compose file with a fixed subnet, so it stops being the odd docker run container that started this mess:

services:
  db:
    image: postgres:17
    networks: [opendesk]
    # ...
  redis:
    image: redis:7
    ports: ["6379:6379"]
    networks: [opendesk]

networks:
  opendesk:
    name: opendesk
    driver: bridge
    ipam:
      config:
        - subnet: 10.201.10.0/24

3. Restart and rebuild (existing networks keep their old subnets until recreated):

rc-service docker restart
docker compose down && docker compose up -d

4. Verify the route moved — the exact test that exposed the bug:

ip route get $(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' redis)
#  -> ... dev br-xxxxxxxx    (fixed)   instead of   dev eth0   (broken)

And the client:

Connected successfully to Async Redis!

The takeaway

Two things I'm keeping:

Timeout vs refused is a routing hint, not a service hint. When a service is provably up but the client times out, stop poking the service and start asking where the packet actually goes. ip route get <ip> is the fastest question you can ask.
Docker's defaults live inside Hyper-V's range. On any Windows/Hyper-V host, pin daemon.json to 10.x on day one. Don't wait for the overlap to bite — it only bites after a reboot re-rolls the subnet, which is the worst possible time to be surprised.

The docker exec ... PONG lied to me for a good ten minutes. It was answering over the container's own loopback, a network path the outside world never uses. The truth was one ip route get away the whole time.

The Skill Writes Itself: Building Complex Claude Skills From Real Work

Denis Morozov — Fri, 15 May 2026 21:35:20 +0000

You know how it goes. You're deep in some code, implementing a new feature or reading about another cool thing you can do with your brain and AI power, when — BUZZ — the work chat blows up. 15 seconds later, you're on an emergency call with the top dogs asking what we can do about it.

Sometimes it happens so fast that users never even notice the issue, chalking it up to a "network error" or "overheated device" — while deep in the Kibana logs, in the silence of traces and spans, the issue grows until it's finally caught by an alert and lands in your inbox as "This isn't an emergency, but it looks suspicious...". Some of you may have a first-class aggregation system that does the job for you, delivering the full picture along with your morning cappuccino. In my case, I have to read Kibana logs across different Kibana instances, check traces in Datadog, dig through the Wiki, JIRA, or even GitHub, and finally tie all the strings together.

One day, filling out yet another investigation report — two-thirds of which were log excerpts showing the same broken workflow — it hit me: "Why am I still doing this?" The investigation and deduction are the fun part. The tedious part is logging into every resource, defining the same time interval five times, copying and combining logs, and tracking down trace IDs (which can mutate as a message travels through different systems).

The trick: don't design the SKILL, discover it

Here's what I want you to take away: don't sit down to design a complex SKILL from scratch. Do the work with the AI, then ask it to codify the workflow into a SKILL afterwards. The conversation is the spec.

This works because a SKILL is just a program written at a higher level of abstraction. And the easiest way to write that program is to run it manually first, with the AI watching.

How it played out

I already had a Swiss Army knife of skills I use in regular work:

a git reader (curl-based)
an Atlassian Wiki skill (read/write Wiki pages)
a JIRA skill (read/write tickets)
a Datadog reader (query traces)

So the next time an alert landed, instead of opening five tabs, I just started asking:

"Check the logs for this time interval for %USER%."
"Analyze the Datadog traces for error responses and cross-reference them against the collected logs — find matches, highlight suspicious patterns."
"Build an issue timeline and list the systems involved."

And here's the kicker — the GitHub skill. Once you know the application version, you can pull the sources, find the exact module producing the error, point right to it in the report, and even run a simulation against the affected app to nail down the root cause.

When the report was ready, instead of clearing the context, I asked: "Create a new SKILL based on this conversation." Review, save, done. That's issue-investigator.

From the first alert to a complete report: 5 minutes on a simple issue, a few hours on the gnarly PROD ones — end-to-end, with me reading the result instead of assembling it.

Keep your skills sharp

A SKILL built from one real investigation isn't finished — it's a v1. Every time you reuse it and spot a rough edge, polish it. That's how a SKILL stays sharp.

And once it is sharp — share it with your team. If you cut down the time you spend on tedious work, you boost your own efficiency. Spread that tool across your team and you boost everyone's. That's the multiplier you don't get from solving the problem alone.

And when every team works this way, SKILLs stop being personal productivity hacks — they become the operating system of the company.

Nuxt 3 Deployment: Automating Environment-Specific Builds with GitHub Actions

Denis Morozov — Sun, 23 Feb 2025 22:39:28 +0000

Introduction

Hi All,

Recently, I needed to prepare a deployment procedure for a Nuxt-based frontend. Based on my experience, I initially expected to create a single build that could be used across all environments — QA, UAT, and PROD. However, I discovered that I had to generate separate builds for each environment, each with its own .env file defining the Strapi URL and other parameters. This limitation arose due to the @nuxt/image library which does not support runtime configuration option for its plugins (we're using Strapi to store images).

Different teams have different workflows—some prefer a single build deployed across all environments (Build-Once, Deploy Everywhere), while others go with separate builds per environment. Both approaches have their pros and cons.

I prefer the first approach due to the following advantages:

Program behavior remains unchanged, ensuring consistency. If only configuration files or databases change, you can guarantee that something working in preproduction will work in production simply by ensuring those configurations match.
It's slightly more robust. You maintain only one "gold" build, reducing the risk of mistakenly deploying a dev version to QA, and so on.

However, due to the limitation I mentioned above, I couldn't use this approach and had to venture into the unfamiliar waters of per-environment builds. Unfortunately, Nuxt’s documentation on this topic is unclear (see Nuxt Deployment Guide), focusing more on exact deployment or testing procedures rather than workflow guidance for development teams.

Environment Setup

To address this issue, I explored and documented a structured approach to handling environment-specific builds in Nuxt, aiming to bridge this gap in the official documentation.

Technology Stack

Nuxt 3
Strapi as the backend
Node.js 18 for building and running
GitHub for storing source code and artifacts

Requirements

Two environments: UAT and PROD
A changelog for each release
The ability to initiate a release by assigning a tag to a commit
The @nuxt/image plugin requires a statically injected (.env) STRAPI_URL, which cannot be placed in runtimeConfig (See the Nuxt Runtime Config Docs)

Workflow

This workflow automates the build and release process by generating environment-specific builds and creating a changelog for each release.

How It Works

Triggering the Workflow: The build and release process is triggered when a tag matching v* is pushed. To deploy to UAT, for example, we assign a tag to the respective commit. The first tag (e.g., v1.0.0) requires a manual CHANGELOG.md generation via:

   git log %YOUR_FIRST_COMMIT%..${{ env.TAG_NAME }} --pretty=format:"%h %s by %an"

Please refer to the Git documentation for a detailed explanation of the output format.

Repository Permissions: The workflow requires write access to create releases and attach artifacts.
Changelog Generation: The difference between the current and previous tags is used to generate a meaningful changelog.
Environment-Specific Configuration: GitHub Action Secrets store the Strapi backend URL for each environment, ensuring secure and correct configurations.
Release Artifacts: Each release includes two build artifacts, one for UAT and one for PROD, ensuring proper separation.

GitHub Actions Workflow:

on:
  push:
    tags:
      - 'v*'

permissions:
  contents: write

env:
  artifact_package_name: project61-nuxt

jobs:
  build:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        node-version: [18.x]
        environment: [UAT, PROD]
    steps:
      - uses: actions/checkout@v4
      - name: Save tag as environment variable
        run: echo "TAG_NAME=${GITHUB_REF#refs/tags/}" >> $GITHUB_ENV
      - name: Store artifact package name as environment variable
        run: echo ARTIFACT_NAME=${{ env.artifact_package_name }}-${{ env.TAG_NAME }}-${{ matrix.environment }}.tar.gz >> $GITHUB_ENV
      - name: Use Node.js ${{ matrix.node-version }}
        uses: actions/setup-node@v3
        with:
          node-version: ${{ matrix.node-version }}
      - run: |
          yarn install --immutable --check-cache --non-interactive
      - name: Load Environment Variables
        run: |
          echo "STRAPI_URL=${{ secrets[format('{0}_STRAPI_URL', matrix.environment)] }}" > .env
      - name: Build Nuxt App
        run: |
          npx nuxt build
      - name: Pack build artifacts
        run: >
          tar zcvf ${{ env.ARTIFACT_NAME }} .output
      - name: Upload Artifact
        uses: actions/upload-artifact@v4
        with:
          name: ${{ env.ARTIFACT_NAME }}
          path: ${{ env.ARTIFACT_NAME }}

  release:
    needs: build
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Download All Artifacts
        uses: actions/download-artifact@v4
        with:
          path: dist
          merge-multiple: true
      - name: Save tag as environment variable
        run: echo "TAG_NAME=${GITHUB_REF#refs/tags/}" >> $GITHUB_ENV
      - name: Store previous tag
        run: echo "PREV_TAG_NAME=$(git describe --abbrev=0 --tags $(git rev-list --tags --skip=1 --max-count=1))" >> $GITHUB_ENV
      - name: Create changelog
        run: |
          echo ${{ env.PREV_TAG_NAME }}..${{ env.TAG_NAME }}   
          git log ${{ env.PREV_TAG_NAME }}..${{ env.TAG_NAME }} --pretty=format:"%h %s by %an" > CHANGELOG.md
          ls -l dist/
      - name: Create Release
        run: gh release create ${{ github.ref_name }} dist/*.tar.gz --title "Release ${{ github.ref_name }}" --notes-file CHANGELOG.md
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Conclusion

This workflow streamlines the deployment process by automating build creation, versioning, and changelog generation. Each environment gets its own dedicated build, ensuring proper separation and accurate testing. By using GitHub Action Secrets, environment-specific configurations remain secure, and every release includes artifacts for both UAT and PROD.

Oh, and one final discovery—while generating changelogs, we realized that some developers might need a refresher on writing meaningful commit messages. Turns out, "fix stuff" and "update" aren't the most helpful descriptions when trying to track changes! 😅

Async Cache Service - you don't need "isLoading" anymore!

Denis Morozov — Tue, 23 May 2023 02:07:27 +0000

What I built

Simple Javascript library to manage asynchronous cache/storage in app/service.

Category Submission:

The best category matches my project is Wacky Wildcards

App Link

You can find library page and package here - Async Cache Service on NPM

Screenshots

Description

The asynchronous cache service is designed to handle the storage and retrieval of data in a non-blocking manner. When data is being loaded into the cache, the service returns a Promise object to the requester. This Promise serves as a placeholder or notification that the data is currently in the process of being fetched.

Once the data loading process is complete and the data is successfully retrieved, the cache service resolves all previously issued Promises associated with that specific data. By resolving the Promises, the cache service effectively provides the updated and complete data to all subscribers or requesters who were waiting for it.

How to use

Typescript Example:

import { AsyncCacheService } from 'async-cache-service';

// by default - the cache is never expire
const cacheService = new AsyncCacheService<string>();

// with 5min expiration
const cacheService = new AsyncCacheService<string>(300_000);

// as a dependency with 15min expiration
export class SomeDataService {
  constructor(private cacheService = new AsyncCacheService<string>(900_000)) {}
}

Javascript Example:

import {AsyncCacheService} from 'async-cache-service';

class DataService {
  // cache expires in 15min
  constructor(cacheService = new AsyncCacheService(15 * 60 * 1000)) {
    this.cacheService = cacheService;
  }

  async getData(id) {
    if (this.cacheService.isExpired(id) {
      this.cacheService.refreshItem(id);

      try {
        // retrieve data from somewhere
        const data = await axios.get('/some/resource');
        this.cacheService.setItem(id, data);
      } catch (e) {
        this.cacheService.flushItem(id);
      }
    }

    return this.cacheService.getItem(id);
  }
}

When the getData method of DataService is called, it returns a Promise, and DataService initiates the data load. During this process, all incoming calls to getData will receive Promises that resolve after the data load. In other words, we eliminate the necessity to have a well-known isLoading flag in our DataService and provide an elegant way to handle multiple data requests.

Link to Source Code

You can find library sources here - Async Cache Service on Github

Permissive License

This project is under MIT License

Background (What made you decide to build this particular app? What inspired you?)

In the preceding article Vanilla JS data cache service, I emphasized the issue of asynchronous data cache and outlined a potential solution. However, after thoughtful consideration, I opted to integrate the suggested approach into a compact Javascript library, accompanied by comprehensive unit tests and leveraging Github Actions for complete CI/CD workflow - encompassing build -> test -> package -> deploy. The resultant library, named "async-data-service", introduces an efficient method to manage asynchronous cache/storage operations with minimal developer overhead.

How I built it (How did you utilize GitHub Actions or GitHub Codespaces? Did you learn something new along the way? Pick up a new skill?)

Well, I used TypeScript (love it!) and Rollup for bundling. For my projects, I prefer to use the Nx build systemit provides me with a lot of pre-defined tools and allows me to focus on what I do. For unit tests, I used the Jest framework. The real challenge for me in this project was to utilize GitHub Actions for NPM publishing.

The most available resources describe the publishing process in a pretty straightforward way, like (taken from official :

name: Publish Package to npmjs
on:
  release:
    types: [published]
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      # Setup .npmrc file to publish to npm
      - uses: actions/setup-node@v3
        with:
          node-version: '16.x'
          registry-url: 'https://registry.npmjs.org'
      - run: npm ci
      - run: npm publish
        env:
          NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

Which isn't really helpful. How does "npm ci" relate to "npm publish"? What is the difference between a "published" release and a "created" release? And finally, where are my built artifacts?

So, I need to find answers. The first thing I resolved was the question regarding the difference between "created" and "published" events for releases. Actually, the "created" event is emitted only for draft releases, while the "published" event is emitted for non-draft releases. Therefore, I will use the "published" event in my Github Actions.

Next, I have to consider that Nx puts build artifacts into the "dist/packages/async-cache-service" folder. Therefore, I need to run "npm publish" from there. As a result of my GitHub Actions efforts, I implemented two actions - "push" and "publish".

The "push" action is used to handle the push event of a pull request and performs the build and test of the proposed changes:

name: Node.js CI

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

jobs:
  build:

    runs-on: ubuntu-latest

    strategy:
      matrix:
        node-version: [18.x]

    steps:
      - uses: actions/checkout@v3
      - name: Use Node.js ${{ matrix.node-version }}
        uses: actions/setup-node@v3
        with:
          node-version: ${{ matrix.node-version }}
      - run: npm ci
      - run: npm run build --if-present
      - run: npm test

The "publish" action is used to handle the release publish event and performs the build, test, and publish steps for the release in two steps:

name: Publish Package to npmjs
on:
  release:
    types: [published]
jobs:
  build:
    runs-on: ubuntu-latest

    strategy:
      matrix:
        node-version: [18.x]

    steps:
      - uses: actions/checkout@v3
      - name: Use Node.js ${{ matrix.node-version }}
        uses: actions/setup-node@v3
        with:
          node-version: ${{ matrix.node-version }}
      - run: |
          npm ci
          npm run build --if-present
          npm test 
      - name: Archive production artifacts
        uses: actions/upload-artifact@v3
        with:
          name: artifact
          path: dist/packages/async-cache-service

  download:
    needs: build
    runs-on: ubuntu-latest
    steps:
      - name: Download build step artifact
        uses: actions/download-artifact@v3
        with:
          name: artifact
      - uses: actions/setup-node@v3
        with:
          node-version: '18.x'
          registry-url: 'https://registry.npmjs.org'
      - run: npm publish
        env:
          NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

As you can see, this action contains two steps - build and publish. The build step performs the build, test, and stores the build artifacts into the Github storage. The publish step retrieves the just created build from the build node and publishes it into the NPM registry.

In summary, I find myself very interested in using GitHub Actions for my current and future projects. I love the way it works. My next step is to adopt GitHub Actions for another project I'm working on, written in C.

Additional Resources/Info

Demystifying Nx's "dependsOn" configuration

Denis Morozov — Sun, 07 May 2023 02:07:18 +0000

We're using Nx in our multirepos to build our Node/Nest apps, as well as Angular. It was a long story how we migrated from Angular Workspace repo for web apps, and single repos for NodeJS/Nest, but in this article I'd like to focus on "dependsOn" property of project.json and how we treat it.

Nx documentation contains a good portion of "dependsOn" description - https://nx.dev/reference/project-configuration#dependson but as for me it still not clear how does "dependsOn" inside "targetDefaults" interact with project-level "dependsOn".

In our project we're utilizing some shared libraries which can be used for Angular apps and for NodeJS/Nest apps as well, i.e. some models definitions and so on. Thus, it is important that those libraries should be built before other apps. That's why we included libraries definitions into "targetDefaults". From other hand we have got some script which need to be run before build, so we create a separate task inside project.json to describe it.

 "injectBuildVersion": {
   "executor": "nx/run-script",
   "options": {
     "script": "inject-build-version some-app"
   }
 }
 "targetDefaults": {
   "build": {
     "dependsOn": "some-lib"
   }
 },
 "dependsOn": ["injectBuildVersion"]

as a result the application was failing to build, because library wasn't built yet. But we noticed that "injectBuildVersion" was executing correctly.

The reason of such behavior is that project-level "dependsOn" overrides "targetDefaults". Moreover, this behavior already described somewhere in Nx issues :-) To make it work, we have to include a special dependency "^build" into project-level "dependsOn" property, like this:

 "injectBuildVersion": {
   "executor": "nx/run-script",
   "options": {
     "script": "inject-build-version some-app"
   }
 }
 "targetDefaults": {
   "build": {
     "dependsOn": "some-lib"
   }
 },
 "dependsOn": ["injectBuildVersion", "^build"]

Now, everything works as expected, build version value (which comes from Jenkins) injects into environment.ts file, and then Nx performs the build with all necessary dependencies. Good job, Nx!

Singletons in JS And How To Use Them

Denis Morozov — Wed, 07 Sep 2022 02:20:18 +0000

Hi Team,

Today, I'd like to tell about singletons and how we are using them on daily-basis.
So, singleton pattern allows us to create single instance of particular class. For which purposes can it be helpful? We are using them for:

data interchange services, which take care about data transferring to/from underlying services
data processing services, which can be instantiated only once on application start
cache services
testing purposes

For example, we decided to continue with previously described Cache Service (https://dev.to/frozer/vanilla-js-data-cache-service-1ei2), for some reasons we don't need multiple
cache services in our app, so how we are going to create a singletone?

Let's start with very basic code to create a singletone:

class SingletoneService {
  static instance;

  getInstance(args) {
    if (!SingletoneService.instance) {
      SingletoneService.instance = new SingletoneService(...args);
    }

    return SingletoneService.instance;
  }

  constructor(args) {
    // do something with args
  }

  doSomething() {
    // do something
  }
}

So, how does it work? In the main program we can create instance of singleton by calling this getInstance method with arguments (if we need them):

function main() {
  const instance = SingletoneService.getInstance();

  instance.doSomething();
}
main();

By calling the getInstance method it checks for static instance field value of class SingletoneService, and then instantiate a new object based on that class using the new keyword. If it exists, it simply returns the existing instance.

Now, once the draft implementation is done, try to move this functionality into our CacheService:

class SomeServiceWithDataCache {
  static instance;

  getInstance(args) {
    if (!SomeServiceWithDataCache.instance) {
      SomeServiceWithDataCache.instance = new SomeServiceWithDataCache(...args);
    }

    return SomeServiceWithDataCache.instance;
  }

  constructor() {
    this.cache = {
      isLoading: false,
      expire: 0,
      data: null
    };
    this.cacheSubscriptions = [];
  }
  ...
}

Also, you can use singletones to inject dependency to your class:

class DataProcessingService {
  // use existing DataCacheService instance by default
  constructor(cacheService = DataCacheService.getInstance()) {
    // do something in constructor)
  }
  ...
}

Which allows you to implement code isolation and Open-Closed principle.

Vanilla JS data cache service

Denis Morozov — Mon, 11 Jul 2022 01:35:09 +0000

Hi Team,

Some time ago I faced an interesting task to solve, and today I'd like to share what I found. Let's imagine that we have a service which needs to load some data (sounds familiar, huh?),

Also we've got several "customers" which utilize the data retrieval method from this service, and for first look all works fine, but when I opened the Developers Console, I was wondering how much duplicate data requests we got there!

Here is our initial implementation of that service (I rewrote it to be more general):

// let's assume we have some network slowness there...
const delayedDataFetch = () => new Promise((resolve) =>
  setTimeout(() => resolve([1, 2, 3]), 2000)
);

class SomeService {
  async getData() {
    return await delayedDataFetch();
  }
}

// ...
// emulation of parallel requests for the same data
function main() {
  const service = new SomeService();
  await Promise.all([
    service.getData(1),
    service.getData(2)
  ]).then(
    ([res1, res2]) => {
      // receives correct data
      console.log(`1. ${JSON.stringify(res1)}`);
      // receives correct data once again, 
      // from the second call to delayedDataFetch
      // and I want to get rid of that second data call to server
      console.log(`2. ${JSON.stringify(res2)}`);
    }
  );
}

Well, my first thought was to implement a some little cache for such data:

const CACHE_EXPIRATION_TIME_MS = 5000;

class SomeService {
  constructor() {
    this.cache = {
      isLoading: false,
      data: null,
      // current timestamp plus expiration delay
      expire: 0
    };
  }

  // added sequenceId for more clarity
  async getData(sequenceId) {
    console.log(`Received ${sequenceId} request`);

    // if cache is expired and isLoading is false 
    // - initiate data update from server
    if (this.cache.expire < new Date().getTime() && !this.cache.isLoading) {
      this.cache.isLoading = true;

      this.cache = {
        isLoading: false,
        data: await delayedDataFetch(),
        expire: new Date().getTime() + CACHE_EXPIRATION_TIME_MS
      };
    }

    console.log(`Response ${sequenceId} with data`);
    return this.cache.data;
  }
}

It works! At least, as I can see from Network Tab the number of networks requests reduced to one. Looks like I can submit this code to repo.

But... hold on, I just started to receive errors from other parts of applications, and the main issue here, that they are different from time to time. Quick searching gives me a new problem - the new code returns null data from time to time.

So, this simple approach didn't solve the problem completely. Let's look closely to this code of getData method:

  // a first request switch isLoading to true, and...
  if (this.cache.expired < new Date().getTime() && !this.cache.isLoading) {
    this.cache.isLoading = true;
    // ... do something
  }

  // the second one simply receives empty data... 
  console.log(`Response ${sequenceId} with data`);
  return this.cache.data;
}

Gotcha! I found! But how can we say our "customers" that they need to wait? Of course, I can switch to RxJS f.e., but I don't want to bubble up the size or our bundle, and I was eager to reach the goal using more vanilla approach.

I started about to return an another Promise to the second response:

  // a first request switch isLoading to true, and...
  if (this.cache.expire < new Date().getTime() && !this.cache.isLoading) {
    this.cache.isLoading = true;
    // ... do something
  }

  if (this.cache.isLoading) {
    // it should return something to client in such cases, 
    // which "something" should be resolved back to refreshed data
    console.log(`Response ${sequenceId} with unresolved promise`);
    return new Promise(resolve => ???);
  }

  // the second one simply receives empty data... 
  console.log(`Response ${sequenceId} with data`);
  return this.cache.data;
}

I was thinking about how I can resolve it, and a flash brighten my brain :-) - I need a simple PUB/SUB!
I decided to add an additional subscribers pool into SomeService class and renamed it to SomeServiceWithDataCache:

class SomeServiceWithDataCache {
  constructor() {
    this.cache = {
      isLoading: false,
      expire: 0,
      data: null
    };
    this.cacheSubscriptions = [];
  }

and change the code of getData method accordingly:

  async getData(sequenceId) {
    console.log(`Received ${sequenceId} request`);
    if (this.cache.expire < new Date().getTime() && !this.cache.isLoading) {
      this.cache.isLoading = true;

      this.cache = {
        isLoading: false,
        data: await delayedDataFetch(),
        expire: new Date().getTime() + CACHE_EXPIRATION_TIME_MS
      };

      // once we receive data - iterate over subscribers pool, 
      // and resolve each with received data, and finally drop
      // subscriptions
      await Promise.all(
        this.cacheSubscriptions.map((res) => res(this.cache.data))
      ).then(() => (this.cacheSubscriptions = []));
    }

    if (this.cache.isLoading) {
      console.log(`Response ${sequenceId} with unresolved promise`);
      // push the promise resolve into subscribers pool
      return new Promise((resolve) => this.cacheSubscriptions.push(resolve));
    }

    console.log(`Response ${sequenceId} with data`);
    return this.cache.data;
  }
}

Well, how it works - it receives the first data request from client, founds that internal data cache is expired, and initiates data loading. For the second request it returns the unresolved Promise, and put the resolver function into the subscribers pool. Once it receives data from server it iterates through the subscribers pool and call resolver with an actual data. That's it.

You can find complete code for the solution on CodeSandBox - https://codesandbox.io/s/simple-data-caching-service-cds83d

SCRUM Retrospective meeting is for what?

Denis Morozov — Sat, 19 Sep 2020 18:19:03 +0000

Hi Devs,

Most of us tackled with SCRUM at least at once. As for me, I do my job about five years using the SCRUM methology. It helpful, it allows us to deliver features and fixes more predictive and transparent, and sometimes, in time :-) you know what I mean. I was a SCRUM master, team leader, ordinal developer, chief... and know what? I still can't understand why we need retrospective.

Last Friday we had one, and our SCRUM master opened a Word document with empty three-column table and asked us the question - "John/Sam/Dan, what you think about last sprint, what was good, what we need to change and what is ugly?". You know that question and I always in trouble what I should to answer. Look, I have a lot of JIRA's, I communicated well with others (BAs, QAs) to get them to solve, and I solved them. Nothing to say - ALL GOOD! And everyone of us said nothing, because - all works well. Right after the meeting, our SCRUM master said that my team-mates "aren't evolved into product". Hey dude, these guys deliver features/fixes every two week in-time and mostly error-free, you can't judge them for the silence on the Retrospective meeting!

If I can't communicate with someone, have hardware problem, or my IDE license has been expired - I don't need to wait until end of the sprint, to tell about it on the retrospective, otherwise I won't do my job well.

As SCRUM methology said - "discuss problems and circumstances with other team-mates..." Cool, but I have time to discuss about on the daily meetings.

So, what do you think about Retrospective? How do we need to cook it well?

DEV Community: Denis Morozov

Your Retry Budget Is Not a Safety Net

*Second in a series on The Factory. Previously: The Factory That Merged 37 Tasks. The harness is at github.com/frozer/factory.

What three attempts is actually good for

Nobody had opened the files

The rule

What it forced

The part I didn't expect

The Factory That Merged 37 Tasks: What Happens When the Harness Stops Believing the Agent

Where this came from

What a packet is

The loop

The one idea

The first real run

What it cost me

The Redis That Timed Out: A Docker Subnet Overlap Hiding in Plain Sight

The setup

The clue was in ifconfig

Why Postgres got away with it

The fix: evict Docker from the danger zone

The takeaway

The Skill Writes Itself: Building Complex Claude Skills From Real Work

The trick: don't design the SKILL, discover it

How it played out

Keep your skills sharp

Nuxt 3 Deployment: Automating Environment-Specific Builds with GitHub Actions

Introduction

Environment Setup

Technology Stack

Requirements

Workflow

How It Works

Conclusion

Async Cache Service - you don't need "isLoading" anymore!

What I built

Category Submission:

App Link

Screenshots

Description

How to use

Link to Source Code

Permissive License

Background (What made you decide to build this particular app? What inspired you?)

How I built it (How did you utilize GitHub Actions or GitHub Codespaces? Did you learn something new along the way? Pick up a new skill?)

Additional Resources/Info

Demystifying Nx's "dependsOn" configuration

Singletons in JS And How To Use Them

Vanilla JS data cache service

SCRUM Retrospective meeting is for what?

The clue was in `ifconfig`