Around 60 minutes for the backend part of Localazy. However, it includes also integration tests that run tens of thousands of simulated requests to verify everything from the basic behavior to concurrent request measuring, etc. It tests around 700 scenarios. Many of them are complex situations involving, e.g., uploading strings for translation, translating them, reviewing, exporting, verifying output, notifications, sent emails, status changes, etc.
These tests are in place to verify not that "units" are working well but that everything together works as expected when all the architectural pieces (DB, cache, message broker, storage, etc.) are involved.
Also, for each of the integration tests, the environment is "restarted" which means to recreate the database structure, etc. It adds a lot of overhead.
However, this is not an issue as the complete tests suite is not used during the development. It's mandatory to pass before releasing on internal testing servers and to production.
I'm a full stack developer working on building web applications for all sorts of sized companies using a wide variety of web technologies. Mostly working in Ruby and JavaScript.
The entire test suite is used once per several days only.
Unit tests for the backend are run locally and used extensively during development. It's API development, so basically TDD. Sometimes, instead of a unit test, integration test is used to verify the complex workflow during development.
For the frontend (not my part, so I have only limited knowledge), e2e tests are run automatically using Github Actions.
I'm a full stack developer working on building web applications for all sorts of sized companies using a wide variety of web technologies. Mostly working in Ruby and JavaScript.
Would you run the entire test suite more often if it was faster? What events trigger a full test suite run? When you’re going to deploy? I’m a big proponent of continuous delivery and run my test suite for a particular code base pretty much every commit.
We plan to move it to the cloud, but since the integration tests need the whole environment to be running, it needs more time than I currently have.
However, it's run before the deployment to testing or production as a mandatory check.
For other things like our CLI tool, we have the whole release chain for the RC version as a Github Action, so except for the backend, we use CD/CI extensively.
I'm a full stack developer working on building web applications for all sorts of sized companies using a wide variety of web technologies. Mostly working in Ruby and JavaScript.
I'm a full stack developer working on building web applications for all sorts of sized companies using a wide variety of web technologies. Mostly working in Ruby and JavaScript.
Yesssss. This is great. I haven't done too much with Kotlin or related tools. But, I have used fastlane quite a bit as a CLI for working with native apps.
It's seriously a god send. Reduces repetitive deploys from 30-45 minute dances of uploads and clicking through UI's to about 1 or 2 commands. It's great. We even used it to start doing real time deployments of one of our native applications. I love stuff like this and how much more productive it makes you!
In fact, the Github Action is not only doing everything for us automatically, but it also documents what is necessary to do.
It's a simple CLI tool but there are so many different operations to do with every release - many more than described in the article such as signing the binaries for Windows and macOS, building RC version, packing 3 different docker containers, uploading to NPM, Docker, APT repo, RPM repo, Brew, our servers, etc.
E.g., for Brew, it needs to be a tar archive accompanied by a YAML descriptor containing SHA checksum. It's not possible for a mortal person to remember all of these steps and to reproduce them for each release.
Around 60 minutes for the backend part of Localazy. However, it includes also integration tests that run tens of thousands of simulated requests to verify everything from the basic behavior to concurrent request measuring, etc. It tests around 700 scenarios. Many of them are complex situations involving, e.g., uploading strings for translation, translating them, reviewing, exporting, verifying output, notifications, sent emails, status changes, etc.
These tests are in place to verify not that "units" are working well but that everything together works as expected when all the architectural pieces (DB, cache, message broker, storage, etc.) are involved.
Also, for each of the integration tests, the environment is "restarted" which means to recreate the database structure, etc. It adds a lot of overhead.
However, this is not an issue as the complete tests suite is not used during the development. It's mandatory to pass before releasing on internal testing servers and to production.
How often do you run the entire codebase? Are you guys just running the unit tests locally during development?
The entire test suite is used once per several days only.
Unit tests for the backend are run locally and used extensively during development. It's API development, so basically TDD. Sometimes, instead of a unit test, integration test is used to verify the complex workflow during development.
For the frontend (not my part, so I have only limited knowledge), e2e tests are run automatically using Github Actions.
Would you run the entire test suite more often if it was faster? What events trigger a full test suite run? When you’re going to deploy? I’m a big proponent of continuous delivery and run my test suite for a particular code base pretty much every commit.
We plan to move it to the cloud, but since the integration tests need the whole environment to be running, it needs more time than I currently have.
However, it's run before the deployment to testing or production as a mandatory check.
For other things like our CLI tool, we have the whole release chain for the RC version as a Github Action, so except for the backend, we use CD/CI extensively.
Ahh. I see. Get it building and running in the cloud! IMO it's always worth it in the end.
Check this article what we already do with Github Actions: localazy.com/blog/localazy-cli-sof...
And from yesterday, we also build the NodeJS version and publish it to NPM automatically. I'm going to write more about it soon :-).
Yesssss. This is great. I haven't done too much with Kotlin or related tools. But, I have used fastlane quite a bit as a CLI for working with native apps.
fastlane.tools/
It's seriously a god send. Reduces repetitive deploys from 30-45 minute dances of uploads and clicking through UI's to about 1 or 2 commands. It's great. We even used it to start doing real time deployments of one of our native applications. I love stuff like this and how much more productive it makes you!
In fact, the Github Action is not only doing everything for us automatically, but it also documents what is necessary to do.
It's a simple CLI tool but there are so many different operations to do with every release - many more than described in the article such as signing the binaries for Windows and macOS, building RC version, packing 3 different docker containers, uploading to NPM, Docker, APT repo, RPM repo, Brew, our servers, etc.
E.g., for Brew, it needs to be a tar archive accompanied by a YAML descriptor containing SHA checksum. It's not possible for a mortal person to remember all of these steps and to reproduce them for each release.
Btw, if you like how we do it, you can upvote us on ProductHunt. We've launched today :-).
producthunt.com/posts/localazy