DEV Community: Dylan Watson

Sociable Tests: Integration tests without the pain!

Dylan Watson — Thu, 11 Nov 2021 14:21:00 +0000

I have been known to complain about how slow and painful to debug SpringBoot integration tests can be. Recently, however, I have found an alternative! 😁 A way to reclaim confidence in integration testing, without sacrificing the speed and debug-ability of unit tests.

For the most part, unit tests offer a great alternative to integration tests but sometimes they are not enough. Often it's still important to have a small number of tests that test how your system fits together. For me, sociable unit tests fill this role.

Integration tests (Such as Spring Boot tests) give confidence that your system hangs together, that you can wire all your dependencies and that your @Controllers are configured to expose your endpoints properly. But for all that, they are just too slow when it gets to testing your logic to any considerable degree. I want tests that run in milliseconds.. not seconds. Especially when I'm talking about running 1000s of tests.

This is where sociable unit tests come in.

Sociable Unit Tests

Sociable Unit Tests are an attempt at a happy medium between unit tests and integration tests. They attempt to test a larger portion of the system than unit tests. Where possible, we want to avoid mocks and restrict mocking to things that would make our tests slow, brittle or non-deterministic (like API or database calls).

Solitary unit tests isolate themselves with mocks. Sociable unit tests try not to.

What are we after?

We just want our tests to be:

Fast
- This ensures we get fast feedback when we make changes (Think milliseconds)
Reliable
- The test needs to return the same result every time (You might also call this deterministic)
Realistic
- Your tests should be running production-like code. That's their whole point.
Structure-insensitive
- Decoupling our tests from the implementation allows us to safely refactor our code without breaking tests

Kent Beck talks more about some other important test properties here

What can I do?

Let's take a look at a few things that can help push our tests in this direction.

✅ Avoid spring-style (@Autowire-ing) dependency injection in tests (Fast)
✅ Construct objects within the test (Reliable)
✅ Prefer real dependencies for testing (Realistic)
✅ Only test against public methods (Structure-insensitive)

So what about these Sociable Unit Tests then?

This image illustrates the different types of Microservice tests. Sociable Unit Tests are Unit tests that may encompass the same amount of business logic as an integration test without the pain of also testing the network and framework!

"Sociable Unit Tests" are essentially unit tests that don't mock their dependencies.

Don't mock your dependencies. It's rude.

Show me a f$@%ing example

Imagine we have these 2 classes, ProductService and PricingEngine:



class ProductService {
  PricingEngine engine;

  public ProductService(PricingEngine engine) {
    this.engine = engine;
  }

  public double getPriceFor(Product product) {
    return engine.calculatePrice(product.getCost());
  }
}



class PricingEngine {
  double markup;

  public PricingEngine(double markup) {
    this.markup = markup;
  }

  public double calculatePrice(double cost) {
    return cost * markup;
  }
}

The ProductService has a PricingEngine which it delegates, to calculate the price.

Let's take a look at how we'd test it in both approaches (solitary and sociable).

Solitary Approach

You will probably recognise the following approach to testing as it has become one of the most popular in recent years. To isolate our system-under-test, we will mock its dependencies, like so:



// Solitary Unit Test

@Test
public void shouldGetPrice() {
  PricingEngine engine = mock(PricingEngine.class);
  Product product = new Product(10);

  when(engine.calculatePrice(10)).thenReturn(13);

  ProductService productService = new ProductService(engine);
  double price = productService.getPriceFor(product);

  assertThat(price).isCloseTo(13.0, within(0.1));
  verify(engine, times(1)).calculatePrice(10)
}

But if we think about it, the only thing this is testing is that we call a specific method on the engine.

If we refactor ProductService to call a different (but equivalent) method, our test would fail. Now not all mock-heavy tests are this terrible but it happens far too often.

Wouldn't it be great if our test continued to pass, so we could prove we haven't broken anything?!

Enter the sociable unit test.

Sociable Unit Test Approach

The core thing we want to test is that we can get a price for a given product.. so let's give it another go with that in mind!



// Sociable Unit Test

@Test
public void shouldGetPrice() {
  Product product = new Product(10);

  ProductService productService = new ProductService(new PricingEngine(1.3));

  double price = productService.getPriceFor(product);

  assertThat(price)
   .isCloseTo(13.0, within(0.1));
}

That's it! A much simpler test.. and it's less sensitive to breakages from structure changes. In fact, in this case, we mostly just removed code.

Make a change

What if we now add the requirement for our price to include a discount based on the customer?

We might end up with a change that looks like:



// ProductService

  public double getPriceFor(Product product, Customer customer) {
    return engine.calculatePrice(product.getCost(), customer.getDiscount());
  }



// PricingEngine

  public double calculatePrice(double cost, double discount) {
    return cost * markup - cost * discount;
  }

Solitary Approach

For a "solitary"-style unit test, to test this requirement, we need to create a customer, pass it to the getPriceFor method... as well as pass the discount into every reference to the PricingEngine#calculatePrice method.



@Test
public void shouldGetPrice() {
  PricingEngine engine = mock(PricingEngine.class);
  Product product = new Product(10);
  Customer customer = new Customer(0);

  when(engine.calculatePrice(10, 0)).thenReturn(13);

  ProductService productService = new ProductService(engine);

  double price = productService.getPriceFor(product, customer);

  assertThat(price)
   .isCloseTo(13.0, within(0.1));
  verify(engine, times(1)).calculatePrice(10, 0);
}

However, with a sociable unit test we only test the public methods of ProductService so don't care how PricingEngine is used. This makes the change much simpler.



// Sociable Unit Test

@Test
public void shouldGetPrice() {
  Product product = new Product(10);
  Customer customer = new Customer(0);

  ProductService productService = new ProductService(new PricingEngine(1.3));

  double price = productService.getPriceFor(product, customer);

  assertThat(price)
   .isCloseTo(13.0, within(0.1));
}

We can see from the above that the required change is just:

Create a customer
Pass it to ProductService#getPriceFor to fix the compilation error.

Even better, if ProductService already had the info it needed to pass to PricingEngine, the test wouldn't even need updating.
On a suite of hundreds or thousands of tests, the effect sociable-style tests can have on refactoring is huge!

We'd want to add other tests or assertions for this new functionality but the simple act of using real dependencies in our tests puts us in a much better place from the start.

Conclusion

Sociable unit tests are an approach that works just as well for small scoped unit tests as it does for integration testing whole pieces of a microservice. Whilst you may still find the need to mock external dependencies such as APIs or slow, non-deterministic ones like a database or filesystem, attempting to use more real dependencies in your tests can have a huge impact on both the speed and usefulness of your tests.

Give it a go and let me know what you think!

DevOps Coding Practices

Dylan Watson — Tue, 06 Apr 2021 22:46:23 +0000

Following on from the previous post in this series The Enterprise DevOps Mindset, this article explores some of the more practical things you can do within your organisation (or startup) to enable faster, less risky releases.

Tiny changes are the best

Small changesets can have massive benefits.

Smaller reviews can:

Allow developers to give better focus and feedback to reviews
Reduce the overhead of any one review
Enable faster reviews
Lessen the cost (and risk) of merge conflicts
Get changes into the hands of users (or QAs) earlier for earlier feedback
Can have lower impact, allowing other devs in the same code base to deploy and launch their features without being impacted by your changes
Contribute to a better, more collaborative design

// Detect dark theme var iframe = document.getElementById('tweet-397664295875805184-772'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=397664295875805184&theme=dark" }

Whilst funny, there is a huge amount of truth in the tweet above! We've all seen it happen, where in order to review, developers are forced to context-switch away from their own work only to find that the pull request is enormous. They take a cursory glance at it and say.. LGTM!

How do we combat it? Well, mostly it comes down to:

Having a plan
- Taking the time to think and plan a series of changes or PRs that are individually releasable
Ensuring a pull request is focussed on only 1 thing
Writing tests early, so you have the confidence to merge your change

Take a look at this for an interesting rule-of-thumb with regards to review sizing: 150-150 rule
He suggests that 150 lines of production code and 150 lines of test code serves as a good suggested maximum someone should be expected to review and review well. Obviously this one is subjective, but it's a good number to at least get a feel for what is "too big".

Backwards Compatibility

When developing APIs, as with any piece of software, you want to ensure that you can deploy your change at any time. You don't want to force all consumers of your API to be deployed at the same time as the API (even if there's only one consumer). This would create a lock-step deployment.

Coupling your backend and frontend deployments can cause massive headaches. If you discover an issue with either of them you'll need to rollback both. Creating extra work and delaying all the other pieces of work in both systems.

To avoid this we must only make non-breaking API changes.

This means:

Only add endpoints, don't remove them
Never add new mandatory parameter to your requests (they must always start out optional)
Never remove a field from a response
Never rename a field or endpoint (This will actually have the same effect as removing).

Ok ok, I know that sounds a bit dramatic. When I say never, I mean before making these removals you must consider the impact it will have on production systems before making this change.

In order to do that, first update all your consumers to handle your change (not using the field, or sending the new field).. then and only then can you make your API change.

One tool I've found useful for this is Swagger Brake, as well as it's plugins for Maven and Gradle.

Keystone Interfaces

One good way to allow your code to be merged and tested without affecting the user is with a Keystone Interface.

Basically, this entails building all the functionality (with all required tests) for a particular feature (over multiple PRs) then, only once everything is ready, we build the UI (or API endpoint) to expose the functionality. This is obviously simplest when the feature can be exposed via a single button but can work well for all kinds of features. In this case, the button becomes the Keystone that is placed at the end and allows us access to the functionality.

This method is very powerful. As Martin Fowler states, it can even be used to hide whole pages of functionality, we can just use a link to that page as the Keystone. It can work just as well in an API where all the functionality is slowly fleshed out, including DB schema and migrations, and then once it's ready and tested, raise a PR to include the change in an existing endpoint. For internal APIs, where you control all consumers you probably don't even need to go to this much effort... Just add the new endpoint.

I've previously felt odd about this particular technique as it feels odd to have code in your codebase that doesn't get used yet (See Dead Code, however as long as the feature is still being worked on, this approach can have some serious benefits and avoid some of the complexities and effort with more involved techniques like feature toggles.

Dark Launches

Martin Fowler also mentions Dark Launching, which is the idea of calling the code for a new feature without the user being able to tell. If done correctly, this could allow you to test out new functionality and it's performance impacts in production without overly affecting the user.

You might wish to expand some existing functionality to display more information to the user. You could flesh this extra info out, collecting it and gathering production stats on performance or errors (obviously you need to ensure the errors don't affect the user). Once you're happy, add the code to expose the new functionality to the user.. a bit like a Keystone Interface again.

Feature Toggles

Once you have considered all other avenues (such as Dark Launching and Keystone Interfaces), in order to avoid launching your feature until you're ready you might need to use a Feature Toggle.

It's often still useful to think about your functionality in terms of Keystones. If your feature allows it, create a Keystone and put your feature toggle around that. Once you have to finished building out and testing your feature (with the feature toggled on for you), you can switch on the toggle.. exposing your feature toggled Keystone to the world.

Not all changes allow you to toggle your code at the UI level but the deeper down in the code you place your toggle, the more intertwined with the test of the code it will be and the harder it will be to test and gain confidence in it.

So place your toggles at the highest level of abstraction you can in your code and try to avoid littering several of the same toggles throughout your code.

If you can't see it, it didn't happen

Lastly, but possibly the most important point is that you must have the ability to see what is going on in production. That means alerting, logging and monitoring.

It should go without saying that if we want to reduce the time taken to become aware of a production issue then we need to be monitoring production for issues!

In the modern software space, you will be inundated with tools that can get you visibility of your production system.
Let's take a look at some of the types of tools to watch out for.

Frontend App

Alerting for errors
HTTP call logging
User tracking
Event Analytics
Demographic analysis

There are so many things that you COULD track and as you build out your system you will likely need to know more and more about how your system is really used.
The key thing is that your chosen solution allows you to continually expand out your knowledge of the system as required.

Some tools that I've seen be used with success:

Sentry (I can't recommend this enough!)
Rollbar
Google Analytics
Firebase
LogRocket
AppSignal

Backend Services

Again there are plenty of things you could track, hopefully, some (like performance or resource usage) are already handled by your platform.

As with frontend monitoring, it doesn't matter so much what you choose, just have something in place that can expand as your visibility requirements increase.. and they will.

Here are a few backend monitoring tools that I've seen work well:

Splunk (Really great at alerting, querying and data transformation)
ELK (Elasticsearch & Kibana)
Prometheus
AWS CloudWatch
xMatters
PagerDuty
ServiceNow
DataDog

Alerts should raise incidents in your incident management system (severity determined by log).

You can even easily calculate your system uptime as:

upTime=100−numOfErrors∗recoveryTime∗100time\%\space upTime = 100 - \frac{numOfErrors * recoveryTime * 100} {time}

For a really good breakdown of what we are actually looking for when monitoring production, take a look at:
Monitoring Production Methodologically

Continuous Delivery

Continuous Delivery is all about automating everything in your process. Everything, right up to the point where you make a manual decision to deploy to production.

The more that is automated, the easier your release process becomes, the more releases that you will be willing to do.

Whilst it might be difficult for highly controlled industries (e.g Government and Finance), at some point we could even automate the decision to deploy to production. We call this continuous deployment. This is just a natural progression of continuous delivery but isn't required to make serious strides towards faster, easier releases.

The medium ground is having a tool with all the codified criteria you have to approve a release to go to production, then just click the button yourself. But given everything we've discussed increasing risk by delaying, if you've got a tool to do the work, once you have confidence in that, why not just let the tool release when it's ready?

For some tips from Atlassian on how to automate visibility of what you are sending into production, see: 6 steps to better release management in Jira Software

Automate, Automate, Automate

Despite everything, you may find that the organisation mandates a whole raft of other checks, post-merge. Don't be disheartened! Once you and your team are following these methodologies the next steps are to begin slowly automating (or even better, negotiating) away the additional post-merge steps.

Conclusion

There are many techniques out there to help us release faster.
The key things to keep in mind are:

Prevent your work from blocking other developers
Commit and merge in small bite-sized chunks
Trust your tests (and if you can't, fix them!)
If you can't see what's happening in production, it's probably broken

And most importantly, using these techniques and others, find a way to release to production on a regular basis. Even if you are in a heavily regulated industry, even if you are required to make no customer-facing changes, you can still gain massive benefits from deploying your (dark launched, toggled off, inactive) code to production on a regular basis.

The Enterprise DevOps Mindset

Dylan Watson — Tue, 06 Apr 2021 22:43:29 +0000

Firstly, let's start with 2 commonly heard statements:

Enterprise doesn't do DevOps
or
Releasing is hard

- signed, every enterprise developer ever

Whilst Enterprise companies tend to have a lot of bureaucracy (especially around releasing), we should still look to improve the pace at which we deliver working software to our customers.

Does this apply to my Enterprise org?

Yep! Any organisation whose customers, regulators, and developers appreciate faster, more reliable software releases with less risk can employ these thought processes.

What is DevOps?

Let's start with a simple shared vision of what DevOps actually is.

DevOps is
a set of practices
that works to
automate processes
so software teams can
build, test, and release
software faster and more reliably.

From Atlassian DevOps

The target

What are we looking to achieve here?
We are hoping to:

Reduce the time it takes for our work (including bug fixes) to land in our user's hands
Reduce the time it takes to become aware of a problem in Production
Reduce the risk and longevity of downtime due to a release
Reduce the impact of any particular bug introduced

Ok, but how?

Git master is sacred

You must ensure that the latest version of your code is always deployable. If you need to "cut a release" and branch off master, you will be less inclined to release on a regular basis.

If your code is not "ready to release" at any time, your ability to safely respond when shit hits the fan is heavily affected.

Think about it, if you first need to rollback your code to a "known" safe place, then this is yet another step in the process before you can even begin fixing your production incident.

In order to keep master "production-ready", all merges to master must also be ready for production.

This is achieved by 2 key things:

Automation
- Anything that gives you the confidence to release, should be automated and it should prevent branches from merging to master if they fail
- Your automation should make it VERY hard to accidentally break production
Coding Practices
- Write your code in a way that allows you to deploy to production safely without causing an outage due to a mistake.
- Feature Toggles, Keystones, and backward compatibility are just some of the techniques that can help maintain prod readiness.

Part 2 of this series "Coding Practices" will give some examples of ways to ensure your code is production-ready.

You build it, you run it

The person best placed to fix issues with a feature is the person that just wrote code for it.
If that person is in charge of dealing with any customer issues with that piece of work they will:

Better understand the problems when they occur
Be more inclined to take care when writing the code
Have better context and incentive to improve the supportability of that code in the future

By getting your Devs at least involved in this process, your software quality will be given a good chance to improve, if only through empathy for the user and themselves.

See Atlassian's take on You build it, you run it and another on some of their ideas around site reliability engineering

Small releases are best

If you spend a year developing software before giving it to a user, you are spending a year building up risk. Any piece of that years (or even months) development could (and likely does) have an undiscovered bug.

If anything goes wrong (and it will), you will have to sift through a whole year of code and features to discover the issue.

On the other hand, if you make a single change and deploy it to production you will likely have much more confidence about deploying to production. Why? Well, if it breaks, it is much easier to have confidence around which change caused the issue... And you can simply rollback to the previous version (or even write a test, fix the bug, and redeploy).

Due to our new confidence in finding issues and rolling back changes quickly, the risk of any 1 change is drastically reduced. We also massively increase our ability to avoid downtime due to an introduced bug.

With additional confidence, we can release faster which gives us much better context when things do go wrong since, we have only just finished working on that section of the code.
This only helps give us the confidence we need to release at an even faster cadence and so on.

Simply stated:

Small change == less risk == faster to release

For further reading here is an interesting Kent Beck article about taking small changes to the extreme: Testing the boundaries of collaboration

Here is also an interesting read on the usefulness of pull-requests, I think the key takeaway here should be that small, very short-lived branches are best.

Automated testing is the key

The other lynchpin in our DevOps plan is testing. If we don't have sufficient automated testing to give us the confidence to release then the plan falls apart. If we rely on manual testing for this confidence, and this manual testing is performed AFTER merging to master, then your manual testing becomes a blocker to production. However, because you have merged to master, you don't just block your code from being released but also any other code that is merged after yours. This puts pressure on the QAs, causing unnecessary delays or forcing them to drop their testing standards.

This is not to say that manual exploratory testing doesn't have its place, it can.
Just that with sufficient automated testing, manual testing shouldn't block a release. If used correctly however, it can be used to validate functionality (often best done by someone with first-hand experience in the business). Manual testing should never be used for anything that you ever want to run more than once to give you confidence you haven't broken something.

For 2 very interesting points of view on appropriate automated testing levels, check out these articles by:

Kent Beck

”Unit” tests

Kent C. Dodds.

Write tests. Not too many. Mostly integration.

(Apparently, all Kents have strong feelings about tests!)

However, if your QAs are finding issues that if deployed to production would impact users, then you are in luck! The next post in this series "DevOps Coding Practices" will take you through several techniques to avoid blocking production deployments, whilst still maintaining safe and reliable production environments.

Let's put it into practice

Next up are some simple DevOps coding practices to follow to ensure master is ready to release at any time.

DevOps Coding Practices

Dylan Watson ・ Apr 6 '21

#devops #enterprise #discuss #monitoring

Java/Angular Build Speed Improvements

Dylan Watson — Mon, 07 Sep 2020 02:34:45 +0000

I've been inspired to fix our build times after watching:

http://dan.bodar.com/2012/02/28/crazy-fast-build-times-or-when-10-seconds-starts-to-make-you-nervous/

I'll be updating this as I investigate different techniques this week.

Here are a list of improvements and the effect they had on build times.

Java & Spring Boot

Spring lazy init

spring.main.lazy-initialization=true
This made the build twice as slow.. and broke the swagger tests as some of the endpoints didn't get instantiated.

Logging

Reduce all logging to only log on WARN
logging.level.root=WARN

Have a single "test" profile

The more profiles you have in tests, the more the spring DI ApplicationContext cache has to be reset. This can have a massive speed impact on your tests.

Test Slices

https://www.baeldung.com/spring-tests#5-using-test-slices

Turn integration tests into unit tests

This one requires considering that integration tests or "SpringBootTests" are really just testing that spring is configured correctly.
Most of the logic should be in fast unit tests.
Have maybe a single integration test

Typescript/Angular/Jest

Switch to Jest

Use @swc-node/jest

https://github.com/Brooooooklyn/swc-node

Use babel for jest

Make your Angular tests 1000% better by switching from Karma to Jest

Dylan Watson — Thu, 05 Dec 2019 02:37:34 +0000

Note: A more recent article might provide better results. Check out this one first

It sounds sensationalist but it's true.

One of the projects I'm working on has an Angular 8 frontend with over 1000 unit/component tests. These used to all run in Karma and take around 15mins but now they take about 1 min.

But why?

What fast tests not good enough for you?
Some other things I've been loving:

Nice error messages
Easy debugging in VS Code (finally!)
Really nice auto run and error message plugins for VS code
Ability to write to disk (Maybe not that useful but I found it handy for some tests)

But how?

Well, let me tell ye a story.

Actually scrap that, you're reading this because you want to convert to jest, maybe you've tried it before and failed, maybe you just want to give it a go - either way let's dig into it.

The approach

If you have a decent sized project, (as with anything in Software) the best way to do it is incrementally.

As we have over 1000 tests, we knew it would take a while to convert them and couldn't do the "big bang" approach as we have about 5 different teams working on the app at any one time - we knew we'd need to run karma and jest side-by-side for a period of time. For us, this ended up being nearly a week but it could have taken way longer!

We naturally are following best software dev practices, so at the end of each step we should be able to create a pull request, run our build, tests and merge to master safely.

Just remember, this is a marathon not a sprint (pardon the pun). As soon as you get a test suite/file passing, commit it. Don't commit broken tests (sounds obvious but you can forget this in the heat of a conversion like this). And don't forget to enlist the help of your fellow developers. This will affect them too so they will want to help out - let them!

With this in mind, our basic approach was this:

Install jest
Get the first test running with Jest (perhaps a brand new test)
Migrate an old test suite/file, using what we've learnt
Write a script to migrate an old suite (based on the manual process we just went though)
Migrate the next test suite using the script, adding anything to the script that is missing
Rinse & Repeat until all the tests are migrated.

Remember, as soon as a test is green -> commit it!
(Jests --onlyChanged flag is very handy here)

Getting started

We start by setting up the jest basics.

Install it:

npm install --save-dev jest @types/jest jest-preset-angular glob @angular-builders/jest

Create a jest.config.js (for Angular) in the project folder:

var preset = require("jest-preset-angular/jest-preset");
module.exports = {
  ...preset,
  preset: "jest-preset-angular",
  setupFilesAfterEnv: ["./setupJest.js"],
  testMatch: ["**/*.test.ts"],
  globals: {
    ...preset.globals,
    "ts-jest": {
      ...preset.globals["ts-jest"],
      tsConfig: "src/tsconfig.test.json",
      isolatedModules: true,
    },
  },
};

Create a setupJest.js file with a single import (you may add others later):

import "jest-preset-angular/setup-jest";

Create a src/tsconfig.test.json for jest:
This should be very similar to your main tsconfig, but with jest types added.

{
  "extends": "../tsconfig.json",
  "compilerOptions": {
    "outDir": "../out-tsc/spec",
    "baseUrl": "./",
    "module": "commonjs",
    "types": ["jest", "node", "jest-extended"]
  },
  "files": ["polyfills.ts"],
  "include": ["**/*.test.ts", "**/*.d.ts", "../setupJest.ts"]
}

If you use jasmine.createSpy or jasmine.createSpyObj, to aid in the migration, you may need a create-spy.ts:

export function createSpyObj<T>(
  baseName: string | (keyof T)[],
  methodNames?: (keyof T)[]
): jest.Mocked<T> {
  if (!methodNames) {
    methodNames = Array.isArray(baseName) ? baseName : [];
  }

  const obj: any = {};

  for (let i = 0; i < methodNames.length; i++) {
    obj[methodNames[i]] = jest.fn();
  }

  return obj;
}

export const createSpy = (
  baseName?
) => {
  return jest.fn();
}

Import this where ever you have broken tests (after running the migration script) relating to creatSpy or createSpyObj.

In order to get jest to actually run, you'll need to create a new test config for karma in your angular.json and replace the existing one with jest:

        "test": {
          "builder": "@angular-builders/jest:run",
          "options": {
            "tsConfig": "<rootDir>/src/tsconfig.test.json"
          }
        },

If you simply replace karma with jest, you will not be able to run karma and jest tests side-by-side!

Instead, rename the existing "test" config in angular.json to "karma:

Then add another script to your package.json
"test-karma": "ng run <you project>:karma"

From now on, jest will run your jest tests and npm run test-karma will run the leftover karma tests.

Your npm test script should now look like this:

"test": "ng test && npm run test-karma"

Visualising Progress

Since this is a big job, we want to see some progress and get others involved, so having a script that outputs the percentage of tests that have been converted is also a really good morale boost.

Here is the script we used. We simply ran it at the end of our builds.

Create a file and name it something painfully obvious, like check-progress.js:

var glob = require("glob")

Reset = "\x1b[0m"
FgRed = "\x1b[31m"
FgGreen = "\x1b[32m"
FgYellow = "\x1b[33m"
FgWhite = "\x1b[37m"

let specs = glob.sync("src/**/*.spec.ts");
let tests = glob.sync("src/**/*.test.ts");

console.log(FgYellow, `${specs.join('\n')}`, Reset)

if (specs.length) {
  console.log(FgRed, specs.length + " slow karma tests")
} else {
  console.log(FgGreen, 'Wooooooooooooooooooooo! Jest conversion complete!')
}
console.log(FgWhite, tests.length + " fast jest tests")
console.log(FgGreen, (tests.length * 100 / (tests.length + specs.length)).toFixed(2) + "% complete in switching tests to jest", Reset)

Then just run node check-progress.js

Finally, your npm test script should now look like this:

"test": "ng test && npm run test-karma && node check-progress.js"

Plugins

If you are using VS Code, you may find the plugins Jest and Jest Runner very handy for running and also debugging your tests (Finally!).

The actual migration

With all our setup out of the way, we should be able to start incrementally converting tests.
There are tools out there like jest-codemods that are meant to do the conversion for you but we didn't have any luck with this, so we built our own. Below is the simple script we used. When we found a case or type of test it couldn't handle, we simply added to the script. You will likely need to continue that pattern for your tests, but this might be a good start.

Note that since we want to run karma specs alongside jest tests (until we've finished converting all the tests), we have chosen the convention of spec.ts for karma tests and test.ts for jest tests. The script below will, after conversion, rename the spec to *.test.ts so your git diff will likely show a bunch of deleted files (the spec.ts files). For this reason it's probably best to just run this on a single test file to start with.

Create a file called convert-to-jest.js:

var fs = require('fs')
var filename = process.argv[2]

if (!filename) {
  let specs = require('glob').sync("src/**/*.spec.ts");
  for (spec of specs) {
    if (!spec.includes('pact')) {
      convertToJest(spec);
    }
  }
} else {
  convertToJest(filename);
}

function convertToJest(filename) {
  if (!filename.startsWith('C:')) {
    filename = './' + filename
  }

  fs.readFile(filename, 'utf8', function (err, data) {
    if (err) {
      return console.log(err);
    }
    var result = data;
    result = result.replace(' } from \'@ngneat/spectator\';', ', SpyObject } from \'@ngneat/spectator/jest\';');
    result = result.replace('} from \'@ngneat/spectator\';', ', SpyObject } from \'@ngneat/spectator/jest\';');
    result = result.replace(/SpyObj</g, 'SpyObject<');
    result = result.replace(/\.and\.returnValue/g, '.mockReturnValue');
    result = result.replace(/\.spec\'/g, '.test');
    result = result.replace(/jasmine\.SpyObj/g, 'SpyObj');
    result = result.replace(/jasmine\.createSpy/g, "createSpy");
    result = result.replace(/spyOn/g, 'jest.spyOn');
    result = result.replace(/spyOnProperty/g, 'spyOn');
    result = result.replace(/expect\((.*)\.calls\.first\(\)\.args\)\.toEqual\(\[(.*)\]\);/g, 'expect($1).toHaveBeenCalledWith($2);')
    result = result.replace(/expect\((.*)\.calls\.any\(\)\)\.toBe\((.*)\);/g, 'expect($1).toHaveBeenCalledWith($2);');
    result = result.replace(/expect\((.*)\.calls\.mostRecent\(\)(\.args\[.*\])?\)\.toEqual\((.*)\);/g, 'expect($1).toHaveBeenCalledWith($2);');
    result = result.replace(/expect\((.*)\.calls\.count\(\)\)\.toBe\((.*)\);/g, 'expect($1).toHaveBeenCalledTimes($2);');
    result = result.replace(/expect\((.*)\.calls\.count\(\)\)\.toEqual\((.*)\);/g, 'expect($1).toHaveBeenCalledTimes($2);');
    result = result.replace(/\.calls\.first\(\).args/g, '.mock.calls[0].args');
    result = result.replace(/and.callFake/g, 'mockImplementation');
    // result = result.replace(/createService\(/g, 'createServiceFactory(');
    // result = result.replace(/createService,/g, 'createServiceFactory,');

    if (result.includes('createSpyObj')) {
      result = result.replace(/jasmine\.createSpyObj/g, 'createSpyObj');
      result = result.replace(/createSpyObject/g, 'createSpyObj');

      var numberOfSlashesinFilename = (filename.replace('./src/app/', '').match(/\//g) || []).length;
      var prefix = "./"
      for (var i = 0; i < numberOfSlashesinFilename; i++) {
        prefix += "../"
      }

      result = 'import { createSpyObj } from \'' + prefix + 'shared/testing/SpyObj\';\r\n' + result;
    }

    result = result.replace('import SpyObj = SpyObj;', '');
    result = result.replace('import Spy = jasmine.Spy;', '');
    result = result.replace('import createSpyObj = createSpyObj;', '');
    result = result.replace(/ Spy;/g, ' jest.SpyInstance;');
    result = result.replace(/jasmine\.Spy;/g, 'jest.SpyInstance;');

    if (!result.includes('@ngneat/spectator') && result.includes('SpyObject')) {
      result = 'import { SpyObject } from \'@ngneat/spectator/jest\';\r\n' + result;
    }
    if (result.includes('MatDialog') && !result.includes('@angular/material/dialog')) {
      result = result.replace(/import \{(.*)MatDialog, (.*)\}/g, 'import {$1$2}');
      result = result.replace(/import \{(.*)MatDialogModule, (.*)\}/g, 'import {$1$2}');
      result = result.replace(/import \{(.*)MatDialogModule(.*)\}/g, 'import {$1$2}');
      result = result.replace(/import \{(.*)MAT_DIALOG_DATA, (.*)\}/g, 'import {$1$2}');
      result = result.replace(/import \{(.*)MatDialogRef, (.*)\}/g, 'import {$1$2}');
      result = 'import { MatDialog, MatDialogModule, MAT_DIALOG_DATA, MatDialogRef } from \'@angular/material/dialog\';\r\n' + result;
    }

    if (result.includes('withArgs')) {
      result = result.replace(/(.*)\.withArgs\((.*)\)\.mockReturnValue\((.*)\)/g, `$1.mockImplementation(flag => {
        switch (flag) {
          case $2:
            return $3;
        }
      })`);
    }

    result = result.replace(/jest\.jest/g, 'jest');

    let newFile = filename.replace('.spec.ts', '.test.ts');
    fs.writeFile(newFile, result, 'utf8', function (err) {
      if (err)
        return console.log(err);
      console.log('Successfully wrote ' + newFile);
      if (newFile != filename) {
        fs.unlinkSync(filename);
      }
    });
  });
}

You'll just need to run:
node convert-to-jest.js <optional path to specific test>

The interesting bit

Now we get to the interesting bit - running the test.
Assuming you've setup your angular.json for jest correctly, you should be able to just run ng test.

I call this "the interesting bit" because I can't really give you much more guidance if it doesn't work. You'll need to figure out why your tests aren't working, for yourself. Of course, if you get lucky and they just work, it's time to convert the next test!

You may also find that if you bulk convert all the tests, there may be some that "just work". If this is the case, you can simply commit these and move on with the rest. You'll also find one command very handy:
ng test --onlyChanged
This is git aware and only runs tests that have changes sitting uncommitted in your git repo. You will find this very handy if you try to bulk convert your tests.

Also since jest outputs a lot of error info, when there are failures, you may want to additionally add:
ng test --onlyChanged --bail
This means that jest will stop on the first test failure, allowing you to focus on that.

Armed with these simple techniques alone, you should be able to convert a bulk of your tests quite quickly.

Results (Check my maths)

Our builds used to take about 15mins to run 1200 tests. After converting to jest our tests now take about 1.5mins. Thats a change from 80 test/min up to 800 test/min - 1000% faster! Ok technically I could just say 10x faster but bigger numbers are better right?