DEV Community

Discussion on: How long does it take for your test suite to run?

Collapse
 
vaclavhodek profile image
vaclavhodek

Around 60 minutes for the backend part of Localazy. However, it includes also integration tests that run tens of thousands of simulated requests to verify everything from the basic behavior to concurrent request measuring, etc. It tests around 700 scenarios. Many of them are complex situations involving, e.g., uploading strings for translation, translating them, reviewing, exporting, verifying output, notifications, sent emails, status changes, etc.

These tests are in place to verify not that "units" are working well but that everything together works as expected when all the architectural pieces (DB, cache, message broker, storage, etc.) are involved.

Also, for each of the integration tests, the environment is "restarted" which means to recreate the database structure, etc. It adds a lot of overhead.

However, this is not an issue as the complete tests suite is not used during the development. It's mandatory to pass before releasing on internal testing servers and to production.

Collapse
 
drews256 profile image
Andrew Stuntz

How often do you run the entire codebase? Are you guys just running the unit tests locally during development?

Collapse
 
vaclavhodek profile image
vaclavhodek

The entire test suite is used once per several days only.

Unit tests for the backend are run locally and used extensively during development. It's API development, so basically TDD. Sometimes, instead of a unit test, integration test is used to verify the complex workflow during development.

For the frontend (not my part, so I have only limited knowledge), e2e tests are run automatically using Github Actions.

Thread Thread
 
drews256 profile image
Andrew Stuntz

Would you run the entire test suite more often if it was faster? What events trigger a full test suite run? When you’re going to deploy? I’m a big proponent of continuous delivery and run my test suite for a particular code base pretty much every commit.

Thread Thread
 
vaclavhodek profile image
vaclavhodek

We plan to move it to the cloud, but since the integration tests need the whole environment to be running, it needs more time than I currently have.

However, it's run before the deployment to testing or production as a mandatory check.

For other things like our CLI tool, we have the whole release chain for the RC version as a Github Action, so except for the backend, we use CD/CI extensively.

Thread Thread
 
drews256 profile image
Andrew Stuntz

Ahh. I see. Get it building and running in the cloud! IMO it's always worth it in the end.

Thread Thread
 
vaclavhodek profile image
vaclavhodek

Check this article what we already do with Github Actions: localazy.com/blog/localazy-cli-sof...

And from yesterday, we also build the NodeJS version and publish it to NPM automatically. I'm going to write more about it soon :-).

Thread Thread
 
drews256 profile image
Andrew Stuntz

Yesssss. This is great. I haven't done too much with Kotlin or related tools. But, I have used fastlane quite a bit as a CLI for working with native apps.

fastlane.tools/

It's seriously a god send. Reduces repetitive deploys from 30-45 minute dances of uploads and clicking through UI's to about 1 or 2 commands. It's great. We even used it to start doing real time deployments of one of our native applications. I love stuff like this and how much more productive it makes you!

Thread Thread
 
vaclavhodek profile image
vaclavhodek

In fact, the Github Action is not only doing everything for us automatically, but it also documents what is necessary to do.

It's a simple CLI tool but there are so many different operations to do with every release - many more than described in the article such as signing the binaries for Windows and macOS, building RC version, packing 3 different docker containers, uploading to NPM, Docker, APT repo, RPM repo, Brew, our servers, etc.

E.g., for Brew, it needs to be a tar archive accompanied by a YAML descriptor containing SHA checksum. It's not possible for a mortal person to remember all of these steps and to reproduce them for each release.

Thread Thread
 
vaclavhodek profile image
vaclavhodek

Btw, if you like how we do it, you can upvote us on ProductHunt. We've launched today :-).

producthunt.com/posts/localazy