DEV Community: Anish Karandikar

Code coverage for a running Deno server

Anish Karandikar — Thu, 27 Oct 2022 01:12:57 +0000

Deno (a modern Rust based TypeScript runtime) comes with code coverage measurement built in. Unfortunately, right now, its only supported for deno test but not for deno run.

This seems unnecessarily limiting, since it makes measuring coverage in a running Deno server by running API tests against it convoluted - but not impossible. This blog outlines a workaround that lets you achieve just this. Thanks to an idea from zachauten.

Let's say you had a bunch of Postman API tests written up for testing a Deno HTTP REST API server that you execute using the excellent newman CLI collection runner. Further, lets say you wanted to measure code coverage from running these API tests.

You would think you would be able to do something like this:

# Start Deno HTTP server and put it in background
deno run --coverage ./main.ts &
export PID=$!

# Run Postman API tests using newman
npx newman run ./api-tests.postman.json  

# Kill server to generate coverage
kill $PID

Unfortunately, the above does not work because deno run --coverage is not supported.

Instead, the trick is to do all these steps (start server, run API tests, stop server) inside a Deno test and then use deno test --coverage. Sigh 😞. Seems labyrinthine but here is how to do it.

Step 1: Make your server a testable module

Make sure that your server is in a form that can be consumed from a Deno test and started and stopped there. For example a simple server can look like:

// server.ts
export function requestHandler(_req: Request): Promise<Response> {
  return Promise.resolve(new Response("Hello, world!"));
}

This server can also be consumed from your main entry point thus:

// main.ts
import { serve } from "std/http/server.ts";
import { requestHandler } from "./server.ts";

serve(requestHandler);

Step 2: Write a test that calls the server

Now write a test that does the following:

Start server with abort controller
Shell out to run newman API tests
Stop server using abort controller

For example:

// server_test.ts
import { assert } from "std/testing/asserts.ts";
import { serve } from "std/http/server.ts";
import { requestHandler } from "./server.ts";

Deno.test("API Tests", async () => {
  // Start server (with abort controller)
  const controller = new AbortController();
  const { signal } = controller;
  const server = serve(requestHandler, { signal });

  // Run API tests
  const p = Deno.run({
    cmd: ["npx", "newman", "run", "./api-tests.postman.json"],
  });
  const { success } = await p.status();
  p.close();
  assert(success);

  // Stop server (using abort controller)
  controller.abort();

  // Wait for server to be closed
  await server;
});

Step 3: Run test in coverage mode

Finally, we can run the server test with the --coverage flag.

deno test --allow-net --allow-run --coverage=coverage ./server_test.ts

This will run the outer server_test.ts in coverage mode, generating code coverage in the coverage folder which can be further processed. See help on Deno coverage.

Conclusion

Hopefully, in the long run, Deno will introduce a deno run --coverage option. Till then, this workaround will have to do.

For a more fleshed out version of the code in this post, please see:

anishkny / deno-api-tests-coverage

Measure code coverage of a Deno server when running Postman (newman) API tests

Deno API Tests Coverage

Demonstrates code coverage measurement of a Deno server when running Postman (newman) API tests.

See accompanying blog post for more details.

Requirements

Deno v1.26 (or higher) installed and available on path.

Usage

Clone this repository and execute:

deno task test

This runs the test file ./test/server_test.ts and collects resulting code coverage.

The test file takes the following steps:

Start the Deno server to be tested (src/server.ts).
Run Postman API tests using newman in a subprocess.
Stop the server.

Background

Deno lacks a way to measure code coverage of a server when running API tests. See this discussion on the Deno repo.

Ideally, there would be a way to start a server thus:

deno run --coverage=coverage src/server.ts # Doesn't work
# Run API tests
# Kill server

Instead you have to start the server, run API tests by shelling out to a process…

View on GitHub

Code coverage for a Next.js app using Playwright tests

Anish Karandikar — Tue, 05 Apr 2022 18:58:40 +0000

Playwright is a remarkable tool that lets you author cross-browser cross-platform end-to-end tests for your web app. It even lets you generate test code as you navigate your app (but that's probably a topic for another blog post)!

Unfortunately, as of now, out of the box, it does not support measuring code coverage for the app being tested (see #7030 and #9208). This post demonstrates a workaround.

Specifically, this post shows you how to instrument a Next.js app using a custom babel.config.js and how to measure code coverage using the playwright-test-coverage plugin. If you want to jump straight into the demo, you can do that here:

anishkny / playwright-test-coverage-demo

Demonstrates usage of playwright-test-coverage - a Playwright extension that collects code coverage from running end-to-end tests

Playwright Test Coverage Demo

Demonstrates usage of playwright-test-coverage - a Playwright extension that collects code coverage from running end-to-end tests.

Usage

npm install
npm test

Code coverage should be generated in coverage/ folder.

Sample

License

MIT

View on GitHub

Step 1: Instrument your code

If you want to measure code coverage, first you will want to instrument your code (i.e. insert counters without causing any side-effects) using istanbul. Most modern web apps (React, Vue etc), use babel in their toolchain (which converts modern ES2015+ code into backwards compatible code to support older browsers). The recommended way to instrument code in this scenario is using babel-plugin-istanbul:



npm i -D babel-plugin-istanbul

Now, we need to instruct babel to use this plugin. The details may vary by framework, but for a Next.js app, create a file named babel.config.js in the root folder:



// babel.config.js
const plugins = [];

// Instrument for code coverage in development mode
if (process.env.NODE_ENV === "development") {
  console.log(
    "Detected development environment. Instrumenting code for coverage."
  );
  plugins.push("istanbul");
}

module.exports = {
  presets: ["next/babel"],
  plugins,
};

This will instrument code only in development mode e.g. when running next dev. That's good because we do not want instrumentation cluttering our production code.

Step 2: Write tests using the `playwright-test-coverage` extension

Now that your code is instrumented, you are ready to measure code coverage. For this you will need to install this Playwright plugin:



npm i -D playwright-test-coverage

This package exports overrides for Playwright's test and expect methods which handle code coverage measurement correctly.



// tests/app.spec.js
const { test, expect } = require("playwright-test-coverage");

test("basic test", async ({ page }) => {
  // Use Playwright as usual
  await page.goto("http://localhost:3000");
  await expect(title).toHaveText("Title");

  // ...
});

If you have existing Playwright tests, simply drop in the following replacement near the top:




// Replace this:
const { test, expect } = require("@playwright/test");

// With this:
const { test, expect } = require("playwright-test-coverage");

Step 3: Measure code coverage

Finally, you are ready to measure code coverage.

Start the Next.js dev server using the Istanbul command line tool nyc - which enables code coverage measurement:



npx nyc next dev

Then, in another terminal window, run the playwright tests as usual:



npx playwright test

If all goes well, you should see a code coverage report similar to:



Running 1 test using 1 worker

  ✓  tests/app.spec.js:5:1 › basic test (1s)


  1 passed (2s)
----------|---------|----------|---------|---------|-------------------
File      | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
----------|---------|----------|---------|---------|-------------------
All files |     100 |      100 |     100 |     100 |
 index.js |     100 |      100 |     100 |     100 |
----------|---------|----------|---------|---------|-------------------

Wrap up

Hopefully you now have some understanding of how to measure code coverage for a Next.js (or any Babel based) app when running Playwright tests.

You can check out the full source code for the demo here:

anishkny / playwright-test-coverage-demo

Demonstrates usage of playwright-test-coverage - a Playwright extension that collects code coverage from running end-to-end tests

Playwright Test Coverage Demo

Demonstrates usage of playwright-test-coverage - a Playwright extension that collects code coverage from running end-to-end tests.

Usage

npm install
npm test

Code coverage should be generated in coverage/ folder.

Sample

License

MIT

View on GitHub

The extension code is available here and contributions/issues are welcome:

anishkny / playwright-test-coverage

Extends Playwright test to measure code coverage

Playwright Test Coverage

A Playwright extension that collects code coverage from running end-to-end tests. Assumes that code has been instrumented with babel-plugin-istanbul during the build process.

Prerequisites

Playwright test framework
babel-plugin-istanbul plugin
nyc for running tests

npm i -D @playwright/test babel-plugin-istanbul nyc

Installation

npm i -D playwright-test-coverage

Usage

Write your Playwright tests as usual, except require test and expect from this package as follows:

// tests/foo.spec.js
const { test, expect } = require("playwright-test-coverage");

// Use test and expect as usual
test("basic test", async ({ page }) => {
  await page.goto("https://playwright.dev/");
  const title = page.locator(".navbar__inner .navbar__title");
  await expect(title).toHaveText("Playwright");
});

Then, instrument your front end source code for coverage using the babel-plugin-istanbul plugin.

Finally, run your server via nyc…

View on GitHub

Cheers!

PS: Using with fixtures

Playwright lets you define custom fixtures in accordance with the Page Object Model pattern. This library should work out of the box with custom fixtures.

You can see an example here.

The incredible power of Deno Deploy

Anish Karandikar — Wed, 23 Mar 2022 16:40:51 +0000

As you folks might know Deno is a modern evolution of Node.js, from the original creator Ryan Dahl, which aims to improve on the design of and avoid many of the pitfalls of the original. It is a fast Rust based runtime for JavaScript/TypeScript.

Deno Deploy is an accompanying service that instantly deploys Deno code to dozens of regions around the world with a single publicly available URL in front of it. It automatically routes requests to the nearest deployed region based on the caller's location! And all this with one click of a button!

It... is... incredible!!!

In comparison, achieving something similar in AWS is almost comically complicated (sorry, I know AWS teams work hard but still... ¯\_(ツ)_/¯).

Having dabbled in serverless development and ops in the past, I wanted to see if Deno Deploy indeed lived up to its promise. So I wrote and deployed a simple script that displays the environment variable DENO_REGION in the response.

import { serve } from "https://deno.land/std@0.120.0/http/server.ts";

function handler(req: Request): Response {
  return new Response(
    `<pre style="font-size:10em;">
       DENO_REGION=<br>
       ${Deno.env.get("DENO_REGION")}
     </pre>`,
    { headers: { "content-type": "text/html" } }
  );
}

console.log("Listening on http://localhost:8000");
await serve(handler);

As soon as I clicked the Save & Deploy button, my URL was live: https://show-region.deno.dev

But how do we know if requests from around the world will be routed to the nearest region as claimed?

Enter GeoPeeker - an excellent service that loads any webpage from different locations around the world and grabs screenshots from there.

Passing our URL to GeoPeeker shows:

As we can clearly see, the request from Singapore is routed to asia-southeast1, Brazil to southamerica-east1 and so on. Very cool indeed!

You can get started with Deno Deploy here - have fun!

𝚒𝚗𝚝𝚎𝚐𝚛𝚒𝚏𝚢 - Firestore referential integrity via triggers

Anish Karandikar — Wed, 23 Jan 2019 16:01:36 +0000

Firestore is amazing, but...

Google Cloud Firestore is a serverless NoSQL document database that scales horizontally - which means it adds/removes nodes to serve your database based on demand automagically. It also does some fancy indexing that allows query times to be proportional to result size instead of total data size. So basically if your query returns 10 records, it will take the same time to run if the total data size is 10, 100, 1000 or squillions of records.

It offers an expressive query language, but does have some limitations which guarantee O(ResultSet) performance. Also, while designing NoSQL database schemas, we have to often "unlearn" data normalization principles we learnt building relational databases.

For example, say you had a database that records comments made by users who have usernames and profile photos. Traditionally you would have stored a foreign key called userId in the comments table, and performed a "join" to get comments together with usernames and profile photos.

But in a NoSQL schema, data is often denormalized - in this case for example, username and photo are repeated in each comment record for ease of retrieval.

The key question then of course is how are updates to username/photo reflected across all comments made by a user? In the case of Firestore, one could write a Cloud Function triggered by updates to any user record which replicates the update to all comment records.

𝚒𝚗𝚝𝚎𝚐𝚛𝚒𝚏𝚢 can help!

𝚒𝚗𝚝𝚎𝚐𝚛𝚒𝚏𝚢 is an npm library that offers pre-canned Firestore triggers that help maintain referential and data integrity in some commonly occurring scenarios.

Attribute Replication

Scenario - Continuing the users/comments example above, you could have a schema like this:

  /users/
    userId/
      username
      photoURL

  /comments/
    commentId/
      body
      userId       <-- foreign key
      username     <-- replicated field
      photoURL     <-- replicated field

Solution - To enforce referential integrity on updates of username/photoURL, simply use:

exports.replUserAttrs = integrify({
  rule: 'REPLICATE_ATTRIBUTES',
  source: {
    collection: 'users',
  },
  targets: [
    {
      collection: 'comments',
      foreignKey: 'userId',
      attributeMapping: { 
        'username': 'username', 
        'photoURL': 'photoURL', 
      },
    },
  ],
});

Stale Reference Deletion

Scenario - Say you have an articles collection, where each article can have zero or more comments each with an articleId foreign key. And you want to delete all comments automatically if the corresponding article is deleted.

  /articles/
    articleId/
      body
      updatedAt
      isPublished
      ...

  /comments/
    commentId/
      articleId   <-- foreign key
      body
      ...

Solution - To delete all comments corresponding to a deleted article, use:

exports.delArticleRefs = integrify({
  rule: 'DELETE_REFERENCES',
  source: {
    collection: 'articles',
  },
  targets: [
    {
      collection: 'comments',
      foreignKey: 'articleId',
    },
  ],
});

Count Maintainence

Scenario - Say you want to record which users have liked any particular article and also be able to quickly determine how many total likes an article has received.

  /likes/
    likeId/
      userId
      articleId    <-- foreign key

  /articles/
    articleId/
      likesCount   <-- aggregate field

Solution - To maintain a live count of number of likes stored in the corresponding article document, use:

[
  module.exports.incrementLikesCount,
  module.exports.decrementLikesCount,
] = integrify({
  rule: 'MAINTAIN_COUNT',
  source: {
    collection: 'likes',
    foreignKey: 'articleId',
  },
  target: {
    collection: 'articles',
    attribute: 'likesCount',
  },
});

Notice that you get two triggers, one to increment and another to decrement the likesCount attributes for every addition or deletion in the likes collection.

Deploying

𝚒𝚗𝚝𝚎𝚐𝚛𝚒𝚏𝚢 is meant to be used in conjunction with firebase-functions and firebase-admin. Indeed, they are peerDependencies for 𝚒𝚗𝚝𝚎𝚐𝚛𝚒𝚏𝚢. Typically, your setup would look like:

const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp();
const db = admin.firestore();
const { integrify } = require('integrify');

integrify({ config: { functions, db } });

// Use integrify here...

Then you would deploy the functions returned by 𝚒𝚗𝚝𝚎𝚐𝚛𝚒𝚏𝚢 like any other Firebase Function:

firebase deploy --only functions

Source Code

Check out the source code, and feel free to open any issues, send out PRs or general comments!

anishkny / integrify

🤝 Enforce referential and data integrity in Cloud Firestore using triggers

𝚒𝚗𝚝𝚎𝚐𝚛𝚒𝚏𝚢

🤝 Enforce referential and data integrity in Cloud Firestore using triggers

Introductory blog post

Usage

// index.js
const { integrify } = require('integrify');

const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp();
const db = admin.firestore();

integrify({ config: { functions, db } });

// Automatically replicate attributes from source to target
module.exports.replicateMasterToDetail = integrify({
  rule: 'REPLICATE_ATTRIBUTES',
  source: {
    collection: 'master',
  },
  targets: [
    {
      collection: 'detail1',
      foreignKey: 'masterId',
      attributeMapping: {
        masterField1: 'detail1Field1',
        masterField2: 'detail1Field2',
      },
    },
    {
      collection: 'detail2',
      foreignKey: 'masterId',
      attributeMapping: {
        masterField1: 'detail2Field1',
        masterField3: 'detail2Field3',
      }

…

View on GitHub

Thanks for reading ✌️✌️✌️

Animating a commit based Sudoku game using Puppeteer

Anish Karandikar — Wed, 31 Oct 2018 15:46:00 +0000

A few months ago I came across an intriguing GitHub repo:

xiegeo / commit-sudoku

Accepting pull requests to collectively finish a Sudoku puzzle.

commit-sudoku

1	5	7	2	9	6	8	3	4
8	9	2	3	4	5	6	1	7
4	3	6	1	8	7	2	5	9
7	4	1	6	5	2	3	9	8
9	2	8	7	3	4	5	6	1
3	6	5	8	1	9	4	7	2
6	8	4	9	7	3	1	2	5
2	1	9	5	6	8	7	4	3
5	7	3	4	2	1	9	8	6

Rules

There are two types of pull requests, moves and others.

A move must only include filling in one number in one cell in the puzzle.

A player can only make at most one move a day.
A player can remove any moves made by himself or herself without the above limitation.
Moves are accepted in order of first submission.
Conflicting moves are skipped, they can be resubmitted…

View on GitHub

It said it was "accepting pull requests to collectively finish a Sudoku puzzle". I was fascinated. The idea was simple. The author had set up an empty Sudoku board and anyone could add a number to it as long as it was a valid Sudoku move (& max 1 move/day/author). Pretty soon I found myself sending out PRs - adding a 4 here, or a 7 there.

It was a bit difficult to specify your intended move because the board was represented as a simple HTML table in the repo's README.md file.

<table>
  <tr>
    <td>1
    <td>&nbsp;
    <td>7
    <td>&nbsp;
    <td>9
    <td>&nbsp;
    <td>&nbsp;
    <td>&nbsp;
    <td>4
    ...

So I sent an enhancement PR embedding row numbers in the board - and lo! - not only was the PR accepted, but the author @xiegeo even offered me commit access to the main repo! I was psyched! Soon along with sending out "move" PR's I also started checking and merging PR's sent out by other contributors and doing general housekeeping tasks.

When I went back to the repo, after a longer delay, I noticed that the board was a lot more filled than I remembered. This got me thinking - how had the board evolved over time since the first move? So I decided to try and animate the evolution of the Sudoku board state from commit history.

Step 1: Get a list of commits

First, I needed to get a list of commit SHAs that were made to the repo in reverse (oldest first) order.

const commitSHAs = execSync('git log --reverse --no-merges --format=format:%H')
  .toString().split('\n');

Step 2: Get Sudoku state for each commit

Next, for each commit, I need to extract the state of the board from the repo's README.md file. I used git show to do this.

const readmeContents = execSync(`git show ${commitSHA}:README.md`)
  .toString().split('\n');

Step 3: Render each state in Puppeteer

Now comes the interesting part. I have the state of the board as HTML for each commit that I need to render as a PNG. For this, I used Puppeteer - a high level Node.js API for controlling headless Chrome.

await page.goto(`data:text/HTML,${HTMLHeader} ${tableState}`);
await page.screenshot({ path: `tmp/T${new Date().getTime()}.png` });

Here tableState contains the HTML representing table state retrieved from the README.md file in the previous step. HTMLHeader is a little boilerplate to apply GitHub's style.

data:text/HTML,... is very interesting. It's called a Data URL. Basically you can put any HTML after it and the browser will render it. For example, click on this URL: data:text/HTML,<h1>hello, world!</h1>.

Note: The spec recommends text/html (lowercase) not text/HTML (uppercase), but I was having issues rendering it correctly in this blog. Both seem to work.

Step 4: Create animated GIF

Finally, I used the excellent gifencoder package to generate an animated GIF from the list of PNG files that were generated from the previous step.

pngFileStream(`tmp/T*.png`)
  .pipe(encoder.createWriteStream({ repeat: 0, delay: 200, quality: 50 }))
  .pipe(fs.createWriteStream(outputGIF));

Result

So here is the resulting animation showing how the game played out:

Step 5 (Bonus): Hook up to TravisCI and Surge.sh

I did not want to be running this script manually on my machine on every commit to update the animation. To begin with I had missed a few commits to the repo and that's why I started down this path in the first place. Luckily getting TravisCI to run this script on every commit (to master) was a breeze.

after_success:
  - if [ "$TRAVIS_BRANCH" = "master" -a "$TRAVIS_PULL_REQUEST" = "false" ]; then cd animation && npm install && npm run generate; fi

This tells TravisCI to only generate the animation if there's an actual commit to master and if all tests pass (i.e. Sudoku board is in a good state).

Finally, I upload the resulting GIF to Surge.sh - which offers simple static asset hosting - so that it can be showcased in the repo's README.md.

SURGE_LOGIN=xxxxxxx@xxxx.com SURGE_TOKEN=xxxxxxxxxx npx surge folder my-endpoint.surge.sh

And that's it!

There are a few details that were skipped in this post for brevity - but you can dig into the complete code here - including how you can run it locally.

Let me know what you thought of the post!

Cheers!

1	5	7	2	9	6	8	3	4
8	9	2	3	4	5	6	1	7
4	3	6	1	8	7	2	5	9
7	4	1	6	5	2	3	9	8
9	2	8	7	3	4	5	6	1
3	6	5	8	1	9	4	7	2
6	8	4	9	7	3	1	2	5
2	1	9	5	6	8	7	4	3
5	7	3	4	2	1	9	8	6

1	5	7	2	9	6	8	3	4
8	9	2	3	4	5	6	1	7
4	3	6	1	8	7	2	5	9
7	4	1	6	5	2	3	9	8
9	2	8	7	3	4	5	6	1
3	6	5	8	1	9	4	7	2
6	8	4	9	7	3	1	2	5
2	1	9	5	6	8	7	4	3
5	7	3	4	2	1	9	8	6

DEV Community: Anish Karandikar

Code coverage for a running Deno server

Step 1: Make your server a testable module

Step 2: Write a test that calls the server

Step 3: Run test in coverage mode

Conclusion

anishkny / deno-api-tests-coverage

Measure code coverage of a Deno server when running Postman (newman) API tests

Deno API Tests Coverage

Requirements

Usage

Background

Code coverage for a Next.js app using Playwright tests

anishkny / playwright-test-coverage-demo

Demonstrates usage of playwright-test-coverage - a Playwright extension that collects code coverage from running end-to-end tests

Playwright Test Coverage Demo

Usage

Sample

License

Step 1: Instrument your code

Step 2: Write tests using the playwright-test-coverage extension

Step 3: Measure code coverage

Wrap up

anishkny / playwright-test-coverage-demo

Demonstrates usage of playwright-test-coverage - a Playwright extension that collects code coverage from running end-to-end tests

Playwright Test Coverage Demo

Usage

Sample

License

anishkny / playwright-test-coverage

Extends Playwright test to measure code coverage

Playwright Test Coverage

Prerequisites

Installation

Usage

PS: Using with fixtures

The incredible power of Deno Deploy

𝚒𝚗𝚝𝚎𝚐𝚛𝚒𝚏𝚢 - Firestore referential integrity via triggers

Firestore is amazing, but...

𝚒𝚗𝚝𝚎𝚐𝚛𝚒𝚏𝚢 can help!

Attribute Replication

Stale Reference Deletion

Count Maintainence

Deploying

Source Code

anishkny / integrify

🤝 Enforce referential and data integrity in Cloud Firestore using triggers

𝚒𝚗𝚝𝚎𝚐𝚛𝚒𝚏𝚢

Usage

Animating a commit based Sudoku game using Puppeteer

xiegeo / commit-sudoku

Accepting pull requests to collectively finish a Sudoku puzzle.

commit-sudoku

Rules

Step 1: Get a list of commits

Step 2: Get Sudoku state for each commit

Step 3: Render each state in Puppeteer

Step 4: Create animated GIF

Result

Step 5 (Bonus): Hook up to TravisCI and Surge.sh

Step 2: Write tests using the `playwright-test-coverage` extension

1	5	7	2	9	6	8	3	4
8	9	2	3	4	5	6	1	7
4	3	6	1	8	7	2	5	9
7	4	1	6	5	2	3	9	8
9	2	8	7	3	4	5	6	1
3	6	5	8	1	9	4	7	2
6	8	4	9	7	3	1	2	5
2	1	9	5	6	8	7	4	3
5	7	3	4	2	1	9	8	6