DEV Community: Anton Yakutovich

Polly wants a transcript: giving agents ears and a voice, on your own machine

Anton Yakutovich — Sun, 31 May 2026 13:11:25 +0000

Half the messages I send my coding agents these days start life as a voice note. I'm walking the dog, an idea lands, I mumble it into my phone, and later something turns it into text an agent can actually act on. It's a great workflow — right up until you notice where the audio goes to become text.

Because the default answer to "transcribe this" is still: ship it to someone's cloud. Whisper API, AWS Transcribe, Deepgram, Google Speech-to-Text. Your voice — which is about as personal as data gets — leaves the building, runs through their model, on their meter, under a privacy policy they can rewrite on a Tuesday. And when you want the round trip — text back to speech — it's the same story: AWS Polly, ElevenLabs, another key, another bill.

Meanwhile my laptop has a Neural Engine sitting mostly idle. Whisper-class models run locally just fine now. So why is the audio leaving at all?

That itch turned into Kesha Voice Kit 🦜 — and Polly can keep her transcript.

What it is

Kesha (yes, named after the cartoon parrot — "Свободу попугаям!" is literally the demo clip) is a local-first voice toolkit: speech-to-text and back, no cloud, no account, no API key. The whole thing is one ~20 MB Rust binary — no Python, no ffmpeg, no native Node addons to babysit.

Transcribe in 25 languages — up to ~19× faster than Whisper on Apple Silicon, ~2.5× on CPU
Speak back in 9 languages, auto-picking a voice from the text's language
On a Mac it runs on the Apple Neural Engine via CoreML; everywhere else it falls back to ONNX
Models are never auto-downloaded — you ask for them once, explicitly, and every weight is pinned by SHA-256 so a mirror can't quietly swap it

The CLI is a thin Bun wrapper; the engine is the Rust binary it shells out to. Pipe-friendly by design — transcript on stdout, errors on stderr.

Quick start

bun add -g @drakulavich/kesha-voice-kit
kesha install                 # downloads engine + models (explicit — never automatic)

kesha audio.ogg               # → transcript to stdout

Want it to talk?

kesha install --tts           # opt-in voices (~990 MB)
kesha say "Свободу попугаям!" > freedom.wav

That's it. No OPENAI_API_KEY, no region to pick, no spend alert to set up.

What's new in 1.22.0

The release I just cut adds two things I'd wanted for a while.

Multilingual voices. Text-to-speech used to be English + Russian and not much else. 1.22.0 wires up the multilingual Kokoro voices on Apple Silicon, so kesha say now covers Spanish, French, Italian, Portuguese and more — all on the Neural Engine, all offline.

Stable error codes everywhere. Every failure path now prints a machine-readable line:

error [E_MODEL_MISSING]: TTS models not installed. Run: kesha install --tts

kesha --error-codes-json dumps the whole taxonomy. If you're driving Kesha from a script or an agent, you no longer have to grep prose to find out why it bailed. There's even a test that fails CI if a code exists in the binary but not in the docs — drift is a build break, not a surprise.

A gotcha I'll save you from (and why I made it louder)

Here's the kind of thing that only shows up once real users point it at real text.

I added the multilingual voices, ran kesha say on a line of Hindi… and got noise. Not wrong words — actual garbage audio. No error, no warning. The most confident kind of broken.

The root cause is buried a layer down. The on-device Kokoro path phonemizes text with an English-only grapheme-to-phoneme model. Feed it Latin script and it's happy. Feed it Devanagari, kana, or Han characters and it doesn't fail — it just produces phonemes that mean nothing, and the model dutifully sings them.

I had three options:

Option	Verdict
Emit the garbage audio	No. Confidently wrong is the worst failure mode there is.
Quietly transliterate to Latin and guess	Fragile, surprising, hides the real gap
Refuse with a clear, coded error	✅

So now non-Latin text aimed at a Latin-only voice stops immediately:

error [E_SCRIPT_UNSUPPORTED]: voice 'hi' cannot phonemize Devanagari text;
it only supports Latin-script input. Romanize the text, or use a voice
whose engine supports Devanagari.

Exit code 4, a stable code, an actionable hint. Fail fast beats fail quietly, every single time — especially for an agent that can't hear that the WAV it got back is nonsense. Real multilingual G2P for those scripts is tracked as an open issue; until it lands, the tool tells you the truth instead of humming gibberish.

Plugging it into agents

The reason any of this exists is that I wanted my agents to hear and speak without phoning home. So Kesha speaks the protocols they do:

MCP — kesha mcp exposes transcribe / synthesize / list-voices as tools to any MCP client (Claude, Cursor, Codex, Gemini)
OpenClaw — drop-in skill so your agent grows ears
Hermes — local STT/TTS through command providers
Raycast on macOS, and a programmatic @drakulavich/kesha-voice-kit/core API if you'd rather call it from a Bun program

A voice note in, a transcript out, an answer spoken back — and the audio never left the laptop.

Honest about the edges

It's not magic. Diarization and the multilingual voices are Apple-Silicon-only today (Linux/Windows get a clear error, not a crash). The first TTS download is ~990 MB — local models aren't free, they're just yours. And as above, true G2P for non-Latin scripts isn't here yet. I'd rather ship the limitation with a loud error than paper over it.

It's MIT, it's on GitHub and npm, and bun add -g @drakulavich/kesha-voice-kit is the whole install.

What does your local-first setup look like these days — and what's still quietly phoning home in your stack that you wish wasn't? 🦜

Awk! Awk! Add a diagram!: Greptile-style PR diagrams, minus the SaaS

Anton Yakutovich — Sun, 31 May 2026 12:18:58 +0000

I review a lot of pull requests with Claude Code and Codex these days. The /review command is genuinely good — it reads the diff, spots the sharp edges, writes it up. But the output is always a wall of prose. On a PR that touches a request path across three services, I'd find myself drawing the call flow on a napkin to convince myself the review was right.

Meanwhile, the hosted reviewers — Greptile, CodeRabbit — just draw it for you. Every PR gets a tidy Mermaid sequence or flow diagram at the top of the review. It's lovely. I wanted that.

So I looked at what it would take.

What's notable here?

The hosted tools render those diagrams server-side: your diff goes to a third party, runs through their LLM, on their subscription. That's a lot of machinery for "turn this diff into a picture" — especially when my /review is already being written by a perfectly capable LLM running on my machine. The model that just understood the change well enough to critique it can obviously draw it.

The diagram doesn't need a SaaS. It needs about thirty lines of glue.

The idea: let the agent draw, let `gh` post

That's the whole design of Iago 🦜 — a skill, not a service:

The host agent draws the diagram. Iago's SKILL.md tells your agent which diagram type fits the diff and how to write valid Mermaid. No provider SDK, no API key — it uses the LLM you're already paying for.
A tiny helper posts it. A single TypeScript file (post.ts, run by bun) finds your /review comment and appends the diagram via authenticated gh. That's the entire runtime: bun + gh.
It's idempotent. Re-run it and it replaces its own block instead of stacking duplicates, using a pair of HTML markers.

No server. No key. No diff leaves your machine except through the same gh you already trust.

What it actually posts

Iago auto-detects the diagram type from the diff — and you can always override it:

The diff looks like…	You get
migrations / `*.sql` / ORM models	an entity-relation diagram
new classes / interfaces across files	a class diagram
a request hopping handler → client → worker	a sequence diagram
branching / state-machine logic	a flowchart
a docs or dependency bump	nothing — it abstains

See an example:

GitHub renders that inline, right on top of the review. (dev.to may show the source above; paste it into a PR to see it drawn.)

Quick start

Runtime is just bun and an authenticated gh. Install the skill into whatever agent you use:

bunx @drakulavich/iago install --force

On Claude Code you can grab the plugin instead:

/plugin marketplace add drakulavich/iago
/plugin install iago@iago-marketplace

Then, on any PR:

/review        # your agent writes the review
/iago          # Iago perches on top of it and squawks a diagram

/squawk is an alias, because of course it is.

A gotcha I'll save you from

Here's the kind of thing that only bites you in production. My sequence-diagram example stopped rendering on GitHub one day, with no useful error. The culprit:

participant Loop

loop is a reserved keyword in Mermaid's sequence grammar — along with alt, opt, par, note, end, and activate. Name a participant Loop and GitHub's renderer silently chokes. Renaming it to Reader fixed it instantly.

So Iago's templates steer the agent away from those reserved ids, and the helper validates with mermaid.parse() before anything gets posted. You shouldn't ever hit it — but if you write Mermaid by hand, now you know where the body is buried. 💀

"Okay, but how do you know the diagrams are any good?"

Fair question, and I'll be honest: there's no benchmark. Diagram quality is the kind of thing a number doesn't capture well, so I validate it by dogfooding — running /iago on real PRs whenever I touch the selection rubric, and looking at what comes out. A formal eval harness felt like overkill for a tool whose whole pitch is "small and dependency-free." I'd rather keep it that way.

Try it on your next PR

If your /review workflow ends in a wall of text, this is a thirty-second add-on that puts a picture on top of it — without signing up for anything. It's MIT, it's on GitHub and npm.

What does your AI code-review setup look like these days — and what would you want drawn automatically on every PR? Would love to hear what you're running. 💎

Fast and reliable end-to-end tests with Playwright on GitHub Actions

Anton Yakutovich — Tue, 24 Jan 2023 17:48:30 +0000

Playwright is a powerful web testing tool supporting Chromium, Firefox and WebKit engines. The project has excellent documentation and many examples.

I use Playwright for E2E tests on the DEV environment. Today I'm going to show how you can integrate Playwright on GitHub Actions more efficiently.

The documentation suggests using the following snippet to run Playwright inside GitHub Actions:

steps:
  - uses: actions/checkout@v3
  - uses: actions/setup-node@v3
    with:
      node-version: '18'
  - name: Install dependencies
    run: npm ci
  - name: Install Playwright
    run: npx playwright install --with-deps
  - name: Run your tests
    run: npx playwright test
  - name: Upload test results
    if: always()
    uses: actions/upload-artifact@v3
    with:
      name: playwright-report
      path: playwright-report

What is notable here?

It's supposed to run all tests on the same type of GitHub Runner. ubuntu-latest by default.
All supported browser engines (chromium, firefox, WebKit) will be installed on the Linux runner.
Playwright will execute tests for all browser engines from the same runner.
The last step will prepare an archive with an HTML report.

After several months of experiments I come up with the following configuration:

# ...

jobs:
  test:
    name: 🧪 ${{ matrix.project }} E2E Tests
    runs-on: ${{ matrix.os }}
    timeout-minutes: 20
    strategy:
      fail-fast: false
      matrix:
        include:
          - project: chromium
            os: ubuntu-latest
            cache_dir: ~/.cache/ms-playwright
          - project: firefox
            os: ubuntu-latest
            cache_dir: ~/.cache/ms-playwright
          - project: webkit
            os: macos-12
            cache_dir: ~/Library/Caches/ms-playwright
    steps:

    - uses: actions/checkout@v3

    - uses: actions/setup-node@v3
      with:
        node-version-file: '.nvmrc'
        cache: 'npm'

    - name: ⚡️ Cache NPM dependencies
      uses: actions/cache@v3
      id: cache-primes
      with:
        path: node_modules
        key: ${{ runner.os }}-npm-${{ hashFiles('package-lock.json') }}

    - name: ⚡️ Cache playwright binaries
      uses: actions/cache@v3
      id: playwright-cache
      with:
        path: ${{ matrix.cache_dir }}
        key: ${{ runner.os }}-${{ matrix.project }}-pw-${{ hashFiles('**/.playwright-version') }}

    - name: 📥 Install deps
      if: steps.cache-primes.outputs.cache-hit != 'true'
      run: make install

    - name: 📥 Install ${{ matrix.project }} with Playwright
      if: steps.playwright-cache.outputs.cache-hit != 'true'
      run: npx playwright install --with-deps ${{ matrix.project }}

    - name: 🎭 Playwright tests
      run: npx playwright test --project=${{ matrix.project }}
      env:
        DEBUG: pw:api,pw:browser*

    - name: 📊 Upload test results
      if: failure()
      uses: actions/upload-artifact@v3
      with:
        name: playwright-report-${{ matrix.project }}
        path: playwright-report

What I've changed in GitHub Workflow for Playwright tests?

I use a matrix with three browser engines to run the jobs simultaneously. Instead of ubuntu-latest, I run WebKit tests on macos-12 runner. I had to make the change to avoid fragile tests. I noticed that the page.goto() API works differently on Linux and macOS for the WebKit engine. WebKit didn't wait for load event, which causes failures from time to time. At the same time, I've never caught such a problem locally on my MacBook. If you're interested, the ticket is still open in the Playwright bug tracker.
The fail-fast parameter is set to false to wait for all testing jobs to complete, even if one of them fails for some reason.
I believe it is essential to specify timeout-minutes for E2E tests so that the tests will crash if the infrastructure hangs or behaves incorrectly.
I use the .nvmrc file inside the repository to specify the NodeJS version. DRY all over the place!
Configured aggressive cache for npm dependencies and binary files with browsers. I added a special target to Makefile to calculate the Playwright browsers hash:

bump-playwright:
  npm install --save-dev @playwright/test
  npx playwright --version > .github/.playwright-version

After executing make bump-playwright command, the latest PW version will be installed and the value of the installed release will be written to the .github/.playwright-version file. GitHub Actions calculate the hash for the cache from the file content. Example file:

$ cat .github/.playwright-version
Version 1.29.2

Each runner installs only one browser and it's pretty handy cause the HTML report will have the results for that browser — easy to navigate and spot the issues. This distinction also allows you to have an idea of the test execution speed in each browser and to notice flickering tests specific to a particular engine.
Playwright tests run with enabled DEBUG, so that in case of a crash, you can quickly get additional information for diagnostics.
In contrast to the recommended configuration from the documentation, I prefer to upload artifacts with the report only in case of test failures. I don't see the point in storing those artifacts on the cloud, given how often the tests are run.

I shared my findings for configuring E2E playwright tests in GitHub Actions. Feel free to ask questions about the example GitHub workflow.

What Playwright tricks do you use in your daily work? Would love to hear about your hidden gems 💎.

Cover photo by Denny Müller.

Aggressive dependency caching in GitHub Actions

Anton Yakutovich — Tue, 18 Oct 2022 15:54:52 +0000

There are three things you can watch forever: fire burning, water falling, and how the build passes the stages in Pipeline after the next commit. To make the wait less tedious, it's best to take care of the CI setup from the beginning.

GitHub Actions has a cache that gets to the runner's virtual machine in seconds. In this article I'd like to share examples of how to set up aggressive dependency caching. Why did I call this approach "aggressive caching"? Because we will be caching not only the packages archives but also the state of the environment after installation.

For Node.js it will be the node_modules directory, and for Python it will be the virtualenv directory with installed dependencies.

Node.js Example

Let's take the typical setup for dependency caching example mentioned in the documentation. If you don't need any exotics, you can use the standard actions/setup-node action, specifying a package manager.

steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v3
  with:
    node-version: 16
    cache: 'npm'
- run: npm ci
- run: npm test

This will save the .npm directory with the global package cache. Sounds great! Remember that if we have several workflow jobs requiring npm ci inside, this will also be time-consuming.

Let's imagine a pipelining with several jobs:

                         │
          Create         │                        Reuse
          Dependencies   │                        Dependencies
          Cache          │
                         │  ┌────────────────────┐
                ┌────────┼──►     Lint  Job      ├────────────────────┐
                │        │  └────────────────────┘                    │
                │        │                                            │
                │        │                                            │
┌───────────────┴────┐   │  ┌────────────────────┐                 ┌──▼─────────────────┐
│     Build Job      ├───┼──►     Test  Job      ├─────────────────►     Deploy Job     │
└───────────────┬────┘   │  └────────────────────┘                 └──▲─────────────────┘
                │        │                                            │
                │        │                                            │
                │        │  ┌────────────────────┐                    │
                └────────┼──►     E2E   Job      ├────────────────────┘
                         │  └────────────────────┘
                         │
                         │
                         │
                         │

Ideally, we want to install dependencies only in the first job and get a state with available dependencies in all subsequent jobs.
I'll show how to achieve it using a sample repo — redux-react-realworld-example-app.

The first (build) job might look like this:

steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v3
  with:
    node-version-file: '.nvmrc' # (1)
    cache: 'npm'

- name: Cache NPM dependencies # (2)
  uses: actions/cache@v3
  id: cache-primes
  with:
    path: node_modules
    key: ${{ runner.os }}-node-${{ hashFiles('package-lock.json') }}

- name: Install dependencies # (3)
  if: steps.cache-primes.outputs.cache-hit != 'true'
  run: npm ci

- name: Build
  run: npm run build

Line #1 specifies the node version using .nvmrc file. That's the alternative way to specify the version and it helps follow the DRY - Don't Repeat Yourself principle.

In line #2 we use actions/cache to cache the node_modules directory. We use the hash from the package-lock.json file as the key.

In line #3 we only install dependencies if the cache is invalidated.

To automatically retrieve node_modules in subsequent jobs, you must declare actions/cache with the same key. For example the test job can be configured as:

steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v3
  with:
    node-version-file: '.nvmrc'
    cache: 'npm'

- name: Cache NPM dependencies
  uses: actions/cache@v3
  with:
    path: node_modules
    key: ${{ runner.os }}-node-${{ hashFiles('package-lock.json') }} # (1)

- name: Tests
  run: npm run test # (2)

Line #1 specifies the cache key. The key must be the same as in the build job. After the actions/cache step we consider that the dependencies are installed and run the tests in line #2.

Check out the complete workflow on GitHub.

Python Example

Standard scenario from actions/setup-python docs:

steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
  with:
    python-version: '3.9'
    cache: 'pip' # caching pip dependencies
- run: pip install -r requirements.txt

This workflow will cache pip packages in ~/.cache/pip, but the installation step will always be performed, as in the previous example with npm ci.

Let's see how we can optimize the installation of dependencies. I'll use the Django-based education-backend repo.

Let's dive into the build job:

steps:
- uses: actions/checkout@v3

- uses: actions/setup-python@v4
  id: setup-python
  with:
    python-version-file: '.python-version'

- uses: actions/cache@v3
  with:
    path: venv
    key: ${{ runner.os }}-venv-${{ steps.setup-python.outputs.python-version }}-${{ hashFiles('**/*requirements.txt') }} # (1)

- name: Install dependencies # (2)
  if: steps.cache-primes.outputs.cache-hit != 'true'
  run: |
    python -m venv venv
    . venv/bin/activate
    pip install --upgrade pip pip-tools
    pip-sync requirements.txt dev-requirements.txt

- name: Run the linter
  run: |
    . venv/bin/activate # (3)
    cp src/app/.env.ci src/app/.env
    make lint

As you can see, we use the same idea for caching as for the Node.js project. There are some minor changes which are quite important. We need to specify the cache key for each python version involved in the workflow. Line #1 has steps.setup-python.outputs.python-version variable exactly for this purpose.

Dependencies installation from line #2 is tricky. For python we use a virtual environment created with the module venv. The environment directory venv will be cached. You can think about it as node_modules for node.

Line #3 has one more trick. After the cache is warmed up it's necessary to initialize virtualenv in the future steps. Otherwise, the python interpreter will not be able to detect the necessary libraries to import.

The simplified test job may look as follows:

- uses: actions/checkout@v3

- uses: actions/setup-python@v4
  id: setup-python
  with:
    python-version-file: '.python-version'

- uses: actions/cache@v3
  with:
    path: venv
    key: ${{ runner.os }}-venv-${{ steps.setup-python.outputs.python-version }}-${{ hashFiles('**/*requirements.txt') }} # (1)

- name: Run the tests
  run: |
    . venv/bin/activate # (2)
    cp src/app/.env.ci src/app/.env
    make test

Ensure you use the same key for the caching step (line #1) and remember to activate the virtual environment before running the tests (line #2).

Check out the complete workflow on GitHub.

Summary

We've practiced "aggressive caching" with Node.js and Python examples. As far as you have a significant number of dependencies the changes can speed up your GitHub workflow sensibly. I recommend trying to set up workflows for your project using the references I've mentioned:

If you still have questions about caching in GitHub Actions, don't hesitate to ask in the comments. I'll try to help.
I would be grateful if you share your tips on how to speed up workflows on GitHub too.

Operation Pact or: How I Learned to Stop Worrying and Love Contract Testing

Anton Yakutovich — Tue, 20 Sep 2022 08:44:37 +0000

I heard about Contract Tests in detail at the Ministry of Testing Meetup. Abhi Nandan made an excellent introduction talk Contract testing in Polyglot Microservices.

I liked the idea and decided to try this approach on my current project. I must say that the devil is in the details. In theory, everything is simple until you try the integration in practice on an existing project.
It is the same with algorithms: when you are watching how the developer on YouTube solves the task, everything seems straightforward. You immediately hit a snag while sitting down to solve the problem alone.

Today I want to focus on practical examples, as much as possible close to real-world conditions. I encourage you to try it on your own to gain experience and see if these approaches are worth applying to your projects. We will use the realword project on GitHub, an Exemplary Medium.com clone powered by different frameworks and tools.

I like using this project for the demo because it's much more complex than HelloWorld, where everything works most of the time. True-to-life problems give your extensive experience.

What is so unique in contract tests?

Contract tests assert that inter-application messages conform to a shared understanding documented in a contract. Without contract testing, the only way to ensure that applications will work correctly together is by using expensive and brittle integration tests.

You can hear opinions: Contract tests often make sense when you have a microservice architecture with 10-50 services.
I'm afraid I have to disagree with that. If you have at least one Provider (back-end with API) and Consumer (front-end), it already makes sense to try this testing approach.

I suggest starting exploration with bi-directional contract tests.

Bi-Directional Contract Testing is a type of static contract testing.
Teams generate a consumer contract from a mocking tool (such as Pact or Wiremock) and API providers verify a provider contract (such as an OAS) using a functional API testing tool (such as Postman). Pactflow then statically compares the contracts down to the field level to ensure they remain compatible.

Why is it better to start with BDCT? Bi-Directional Contract Testing (BDCT) allows you to use existing Provider Open API. If your current project has a back-end service with Open API specification, you are lucky to leverage it as a Provider contract.

1. Provider Contract

Let's configure publishing of OAS spec from realworld app to pactflow.

Requirements:

Create a free account on Pactflow.io;
Install Docker on your machine.

We want to be able to publish Open API spec to Pactflow locally and from CI/CD. To do that we will need the following: API key from Pactflow, docker and make commands installed.

Consider the Makefile:

# Makefile
PACTICIPANT ?= "realworld-openapi-spec"

## ====================
## Pactflow Provider Publishing
## ====================
PACT_CLI="docker run --rm -v ${PWD}:/app -w "/app" -e PACT_BROKER_BASE_URL -e PACT_BROKER_TOKEN pactfoundation/pact-cli"
OAS_FILE_PATH?=api/openapi.yml
OAS_FILE_CONTENT_TYPE?=application/yaml
REPORT_FILE_PATH?=api/README.md
REPORT_FILE_CONTENT_TYPE?=text/markdown
VERIFIER_TOOL?=newman

# Export all variable to sub-make if .env exists
ifneq (,$(wildcard ./.env))
    include .env
    export
endif

default:
    cat ./Makefile

ci: ci-test publish_pacts can_i_deploy

# Run the ci target from a developer machine with the environment variables
# set as if it was on CI.
# Use this for quick feedback when playing around with your workflows.
fake_ci:
    @CI=true \
    GIT_COMMIT=`git rev-parse --short HEAD` \
    GIT_BRANCH=`git rev-parse --abbrev-ref HEAD` \
    make ci

## =====================
## Build/test tasks
## =====================

ci-test:
    @echo "\n========== STAGE: CI Tests ==========\n"

## =====================
## Pact tasks
## =====================

publish_pacts:
    @echo "\n========== STAGE: publish provider contract (spec + results) - success ==========\n"
    PACTICIPANT=${PACTICIPANT} \
    "${PACT_CLI}" pactflow publish-provider-contract \
    /app/${OAS_FILE_PATH} \
    --provider ${PACTICIPANT} \
    --provider-app-version ${GIT_COMMIT} \
    --branch ${GIT_BRANCH} \
    --content-type ${OAS_FILE_CONTENT_TYPE} \
    --verification-exit-code=0 \
    --verification-results /app/${REPORT_FILE_PATH} \
    --verification-results-content-type ${REPORT_FILE_CONTENT_TYPE} \
    --verifier ${VERIFIER_TOOL}

deploy: deploy_app record_deployment

can_i_deploy:
    @echo "\n========== STAGE: can-i-deploy? 🌉 ==========\n"
    "${PACT_CLI}" broker can-i-deploy --pacticipant ${PACTICIPANT} --version ${GIT_COMMIT} --to-environment test

deploy_app:
    @echo "\n========== STAGE: deploy ==========\n"
    @echo "Deploying to test"

record_deployment:
    "${PACT_CLI}" broker record-deployment --pacticipant ${PACTICIPANT} --version ${GIT_COMMIT} --environment test

## =====================
## Misc
## =====================

.env:
    cp -n .env.example .env || true

Run make .env to create .env file. You will need to update two variables:

PACT_BROKER_BASE_URL
PACT_BROKER_TOKEN

[Optional] We may try to publish the contract from the local machine using make fake-ci. This target under the hood executes the following stages:

ci-test runs functional tests (skipped in our case)
publish_pacts uses pactflow docker image to publish current OAS to Pactflow broker. GIT_COMMIT and GIT_BRANCH environment variables are used to set the version tag and the branch on the broker side.
can_i_deploy verifies that the published contract is compatible with dependent consumers. Hence we are safe to deploy.

And the last piece. Let's create a GitHub workflow to publish OAS on each push to the repository:

# .github/workflows/pact.yml

name: Pact CI

on:
  pull_request:
  push:
    branches:
      - main
    paths-ignore:
      - '*.md'

concurrency:
  # For pull requests, cancel all currently-running jobs for this workflow
  # https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#concurrency
  group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
  cancel-in-progress: true

env:
  PACT_BROKER_BASE_URL: https://drakulavich.pactflow.io
  PACT_BROKER_TOKEN: ${{ secrets.PACT_BROKER_TOKEN }}
  GIT_COMMIT: ${{ github.sha }}

jobs:
  verify:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3

    - name: Inject slug/short variables
      uses: rlespinasse/github-slug-action@v4

    - name: Tests
      run: GIT_BRANCH=${GITHUB_REF_NAME_SLUG} make ci

  deploy:
    runs-on: ubuntu-latest
    needs:
      - verify
    if: ${{ github.ref == 'refs/heads/main' }}
    steps:
      - uses: actions/checkout@v3

      - name: Inject slug/short variables
        uses: rlespinasse/github-slug-action@v4

      - name: 🚀 Record deployment on Pactflow
        run: GIT_BRANCH=${GITHUB_REF_NAME_SLUG} make deploy

GitHub will use the same Makefile. Please, have a look at the Deploy job from the workflow above. We are executing make deploy to explicitly mark the deployed version for the test environment on the Pactflow broker.

We will provide the environment variable to access Pactflow differently. Don't forget to add the secret PACT_BROKER_TOKEN in Settings — Secrets — New repository secret.

Ok. We are ready to commit the files! If everything is fine, the pipeline should be green.

Open your Pactflow workspace and check out the Provider Contract. You should be able to open Open API Swagger spec.

So, we finished with the first part. We have CI/CD for Provider Contract. Let's move on to the second part.

2. Consumer contract

We will use ts-redux-react-realworld-example-app project which implements the front-end in TS for realword app using React/Redux.

If you want to follow the same approach, I suggest you to check pact-workshop-js for better understanding.

1) We'll start by installing @pact-foundation/pact dependency:

npm install --save-dev @pact-foundation/pact

2) Create directory src/services/consumer/
3) Create src/services/consumer/apiPactProvider.ts
Here we'll describe the pact specification format and name of our pacticipants (participants).

// src/services/consumer/apiPactProvider.ts

import path from 'path';
import { PactV3, MatchersV3, SpecificationVersion } from '@pact-foundation/pact';

export const { eachLike, like } = MatchersV3;

export const provider = new PactV3({
  consumer: 'ts-redux-react-realworld-example-app',
  provider: 'realworld-openapi-spec',
  logLevel: 'warn',
  dir: path.resolve(process.cwd(), 'pacts'),
  spec: SpecificationVersion.SPECIFICATION_VERSION_V2,
});

4) Create src/services/consumer/api.pact.spec.ts

Let's start with something tiny and achievable. We would like to check /tags endpoint.

// src/services/consumer/api.pact.spec.ts

describe('API Pact tests', () => {
  describe('getting all tags', () => {
    test('tags exist', async () => {
      // set up Pact interactions

      // Execute provider interaction
    });
  });
});

From pact-workshop-js you will learn that each Pact test contains two parts:

Setup Pact interactions.
Execute the interaction using the API client.

The first part might look like:

// src/services/consumer/api.pact.spec.ts
// ...
      const tagsResponse = {
        tags: ['reactjs', 'angularjs'],
      };

      // set up Pact interactions
      await provider.addInteraction({
        states: [{ description: 'tags exist' }],
        uponReceiving: 'get all tags',
        withRequest: {
          method: 'GET',
          path: '/tags',
        },
        willRespondWith: {
          status: 200,
          headers: {
            'Content-Type': 'application/json',
          },
          body: like(tagsResponse),
        },
      });
// ...

We created a stub for the response tagsResponse. Then we added interaction to the provider object from src/services/consumer/apiPactProvider.ts. The essential parts relate to the expected request and how the provider will reply.

In the second part we will add the test execution:

// src/services/consumer/api.pact.spec.ts
// ...

      await provider.executeTest(async (mockService) => {
        axios.defaults.baseURL = mockService.url;

        // make request to Pact mock server
        const tags = await getTags();

        expect(tags).toStrictEqual(tagsResponse);
      });
// ...

Ultimately we need to change the URL for the API client and execute the request after it. In our case we have to alter axios baseURL, because all API interactions explicitly use axios. For example, getTags() implementation:

export async function getTags(): Promise<{ tags: string[] }> {
  return guard(object({ tags: array(string) }))((await axios.get('tags')).data);
}

We want to verify that response is the same as a prepared stub. After collecting all parts inside the test file we will get the following:

// src/services/consumer/api.pact.spec.ts

import axios from 'axios';
import { provider, like } from './apiPactProvider';
import { getTags } from '../conduit';

describe('API Pact tests', () => {
  describe('getting all tags', () => {
    test('tags exist', async () => {
      const tagsResponse = {
        tags: ['reactjs', 'angularjs'],
      };

      // set up Pact interactions
      await provider.addInteraction({
        states: [{ description: 'tags exist' }],
        uponReceiving: 'get all tags',
        withRequest: {
          method: 'GET',
          path: '/tags',
        },
        willRespondWith: {
          status: 200,
          headers: {
            'Content-Type': 'application/json',
          },
          body: like(tagsResponse),
        },
      });

      await provider.executeTest(async (mockService) => {
        axios.defaults.baseURL = mockService.url;

        // make request to Pact mock server
        const tags = await getTags();

        expect(tags).toStrictEqual(tagsResponse);
      });
    });
  });
});

Let's run make fake-ci to check how is going:

Computer says yes \o/ 

CONSUMER                             | C.VERSION          | PROVIDER               | P.VERSION  | SUCCESS? | RESULT#
-------------------------------------|--------------------|------------------------|------------|----------|--------
ts-redux-react-realworld-example-app | d981433+1663657371 | realworld-openapi-spec | 95dbd23... | true     | 1

Hooray! It's a huge step towards contract testing.
Now let's try to test another endpoint GET /user. We will start with the same ideas: create a stub, prepare interactions and execute API client call:

// src/services/consumer/api.pact.spec.ts
// ...

  describe('getting current user', () => {
    test('user exists', async () => {
      const tUser: User = {
        email: 'jake@jake.jake',
        token: 'jwt.token.here',
        username: 'jake',
        bio: 'I work at statefarm',
        image: 'https://i.stack.imgur.com/xHWG8.jpg',
      };

      const userResponse = {
        user: tUser,
      };
      // set up Pact interactions
      await provider.addInteraction({
        states: [{ description: 'user has logged in' }],
        uponReceiving: 'get user',
        withRequest: {
          method: 'GET',
          path: '/user',
        },
        willRespondWith: {
          status: 200,
          headers: {
            'Content-Type': 'application/json',
          },
          body: like(userResponse),
        },
      });

      await provider.executeTest(async (mockService) => {
        axios.defaults.baseURL = mockService.url;

        // make request to Pact mock server
        const user = await getUser();

        expect(user).toStrictEqual(tUser);
      });
    });
  });

// ...

If you run make fake-ci, you will get an error:

Computer says no ¯_(ツ)_/¯

Follow the generated report link and you will see the details:

Request Authorization header is missing but is required by the spec file

We missed Authorization header that is required to check logged in user. Let's add the stub (1), headers to the expected request (2) and axios defaults (3):

// src/services/consumer/api.pact.spec.ts

      // (1)
      const authToken = {
        Authorization: 'Token xxxxxx.yyyyyyy.zzzzzz',
      };

// ...

        uponReceiving: 'get user',
        withRequest: {
          method: 'GET',
          path: '/user',
          headers: authToken, // (2)
        },
// ...
      await provider.executeTest(async (mockService) => {
        axios.defaults.baseURL = mockService.url;
        axios.defaults.headers.Authorization = authToken.Authorization; // (3)

After next make fake-ci attempt you will get the new error:

FAIL src/services/consumer/api.pact.spec.ts
  ● Console

    console.error
      Error: Cross origin http://localhost forbidden

I found the fix for it on StackOverflow. We need to add the following line on the top of the test file:

axios.defaults.adapter = require('axios/lib/adapters/http');

Next make fake-ci should be successful.

Summary

We have learned how to configure Provider and Consumer for bi-directional contract testing. We used RealWorld project to practice integration with Pactflow broker. And finally, we wrote a couple of consumer tests for ts-redux-react-realworld-example-app. We also learned how to debug the tests using make fake-ci command. All of the examples are available on GitHub. Feel free to try and post your questions if you face the issues.

Materials

RealWorld repo with Open API spec (provider contract)
ts-redux-react-realworld-example-app with pact tests (consumer)
pact js workshop

GitLab CI: Cache and Artifacts explained by example

Anton Yakutovich — Wed, 04 Aug 2021 10:13:46 +0000

Hi, DEV Community! I've been working in the software testing field for more than eight years. Apart from web services testing, I maintain CI/CD Pipelines in our team's GitLab.

Let's discuss the difference between GitLab cache and artifacts. I'll show how to configure the Pipeline for the Node.js app in a pragmatic way to achieve good performance and resource utilization.

There are three things you can watch forever: fire burning, water falling, and the build is passing after your next commit. Nobody wants to wait for the CI completion too much, it's better to set up all the tweaks to avoid long waiting between the commit the build status. Cache and artifacts to the rescue! They help reduce the time it takes to run a Pipeline drastically.

People are confused when they have to choose between cache and artifacts. GitLab has bright documentation, but the Node.js app with cache example and the Pipeline template for Node.js contradict each other.

Let's see what the Pipeline in GitLab terms means. The Pipeline is a set of stages and each stage can have one or more jobs. Jobs work on a distributed farm of runners. When we start a Pipeline, a random runner with free resources executes the needed job. The GitLab-runner is the agent that can run jobs. For simplicity, let's consider Docker as an executor for all runners.

Each job starts with a clean slate and doesn't know the results of the previous one. If you don't use cache and artifacts, the runner will have to go to the internet or local registry and download the necessary packages when installing project dependencies.

What is cache?

It's a set of files that a job can download before running and upload after execution. By default, the cache is stored in the same place where GitLab Runner is installed. If the distributed cache is configured, S3 works as storage.

Let's suppose you run a Pipeline for the first time with a local cache. The job will not find the cache but will upload one after the execution to runner01. The second job will execute on runner02, it won't find the cache on it either and will work without it. The result will be saved to runner02. Lint, the third job, will find the cache on runner01 and use it (pull). After execution, it will upload the cache back (push).

What are artifacts?

Artifacts are files stored on the GitLab server after a job is executed. Subsequent jobs will download the artifact before script execution.

Build job creates a DEF artifact and saves it on the server. The second job, Test, downloads the artifact from the server before running the commands. The third job, Lint, similarly downloads the artifact from the server.

To compare the artifact is created in the first job and is used in the following ones. The cache is created within each job.

Consider the CI template example for Node.js recommended by GitLab:

image: node:latest # (1)

# This folder is cached between builds
cache:
  paths:
    - node_modules/ # (2)

test_async:
  script:
    - npm install # (3)
    - node ./specs/start.js ./specs/async.spec.js

test_db:
  script:
    - npm install # (4)
    - node ./specs/start.js ./specs/db-postgres.spec.js

Line #1 specifies the docker image, which will be used in all jobs. The first problem is the latest tag. This tag ruins the reproducibility of the builds. It always points to the latest release of Node.js. If the GitLab runner caches docker images, the first run will download the image, and all subsequent runs will use the locally available image. So, even if a node is upgraded from version XX to YY, our Pipeline will know nothing about it. Therefore, I suggest specifying the version of the image. And not just the release branch (node:14), but the full version tag (node:14.2.5).

Line #2 is related to lines 3 and 4. The node_modules directory is specified for caching, the installation of packages (npm install) is performed for every job. The installation should be faster because packages are available inside node_modules. Since no key is specified for the cache, the word default will be used as a key. It means that the cache will be permanent, shared between all git branches.

Let me remind you, the main goal is to keep the pipeline reproducible. The Pipeline launched today should work the same way in a year.

NPM stores dependencies in two files — package.json and package-lock.json. If you use package.json, the build is not reproducible. When you run npm install the package manager puts the last minor release for not strict dependencies. To fix the dependency tree, we use the package-lock.json file. All versions of packages are strictly specified there.

But there is another problem, npm install rewrites package-lock.json, and this is not what we expect. Therefore, we use the special command npm ci which:

removes the node_modules directory;
installs packages from package-lock.json.

What shall we do if node_modules will be deleted every time? We can specify NPM cache using the environment variable npm_config_cache.

And the last thing, the config does not explicitly specify the stage where jobs are executed. By default, the job runs inside the test stage. It turns out that both jobs will run in parallel. Perfect! Let's add jobs stages and fix all the issues we found.

What we got after the first iteration:

image: node: 16.3.0 # (1)

stages:
  - test

variables:
  npm_config_cache: "$CI_PROJECT_DIR/.npm" (5)

# This folder is cached between builds
cache:
  key:
    files:
      - package-lock.json (6)
  paths:
    - .npm # (2)

test_async:
  stage: test
  script:
    - npm ci # (3)
    - node ./specs/start.js ./specs/async.spec.js

test_db:
  stage: test
  script:
    - npm ci # (4)
    - node ./specs/start.js ./specs/db-postgres.spec.js

We improved Pipeline and make it reproducible. There are two drawbacks left. First, the cache is shared. Every job will pull the cache and push the new version after executing the job. It's a good practice to update cache only once inside Pipeline. Second, every job installs the package dependencies and wastes time.

To fix the first problem we describe the cache management explicitly. Let's add a "hidden" job and enable only pull policy (download cache without updating):

# Define a hidden job to be used with extends
# Better than default to avoid activating cache for all jobs
.dependencies_cache:
  cache:
    key:
      files:
        - package-lock.json
    paths:
      - .npm
    policy: pull

To connect the cache you need to inherit the job via extends keyword.

...
extends: .dependencies_cache
...

To fix the second issue we use artifacts. Let's create the job that archives package dependencies and passes the artifact with node_modules further. Subsequent jobs will run tests from the spot.

setup:
  stage: setup
  script:
    - npm ci
  extends: .dependencies_cache
  cache:
    policy: pull-push
  artifacts:
    expire_in: 1h
    paths:
      - node_modules

We install the npm dependencies and use the cache described in the hidden dependencies_cache job. Then we specify how to update the cache via a pull-push policy. A short lifetime (1 hour) helps to save space for the artifacts. There is no need to keep node_modules artifact for a long time on the GitLab server.

The full config after the changes:

image: node: 16.3.0 # (1)

stages:
  - setup
  - test

variables:
  npm_config_cache: "$CI_PROJECT_DIR/.npm" (5)

# Define a hidden job to be used with extends
# Better than default to avoid activating cache for all jobs
.dependencies_cache:
  cache:
    key:
      files:
        - package-lock.json
    paths:
      - .npm
    policy: pull

setup:
  stage: setup
  script:
    - npm ci
  extends: .dependencies_cache
  cache:
    policy: pull-push
  artifacts:
    expire_in: 1h
    paths:
      - node_modules

test_async:
  stage: test
  script:
    - node ./specs/start.js ./specs/async.spec.js

test_db:
  stage: test
  script:
    - node ./specs/start.js ./specs/db-postgres.spec.js

We learned what's the difference between cache and artifacts. We built a reproducible Pipeline that works predictably and uses resources efficiently. This article shows some common mistakes and how to avoid them when you are setting up CI in GitLab.
I wish you green builds and fast pipelines. Would appreciate your feedback in the comments!

DEV Community: Anton Yakutovich

Polly wants a transcript: giving agents ears and a voice, on your own machine

What it is

Quick start

What's new in 1.22.0

A gotcha I'll save you from (and why I made it louder)

Plugging it into agents

Honest about the edges

Awk! Awk! Add a diagram!: Greptile-style PR diagrams, minus the SaaS

What's notable here?

The idea: let the agent draw, let gh post

What it actually posts

Quick start

A gotcha I'll save you from

"Okay, but how do you know the diagrams are any good?"

Try it on your next PR

Fast and reliable end-to-end tests with Playwright on GitHub Actions

Aggressive dependency caching in GitHub Actions

Node.js Example

Python Example

Summary

Operation Pact or: How I Learned to Stop Worrying and Love Contract Testing

What is so unique in contract tests?

1. Provider Contract

2. Consumer contract

Summary

Materials

GitLab CI: Cache and Artifacts explained by example

What is cache?

What are artifacts?

Links

The idea: let the agent draw, let `gh` post