That's the thing. In big projects, you should have multiple levels of tests. The unit tests should be the fastest ones and should be executed after each push. The integration tests are the ones that should be executed before merging feature branch into master branch (if it triggers deployment). Any other long running tests (like separate test automation that's clicking around the application and takes tens of minutes to finish) should be run nightly.
Exactly Pavol! When I read this article I get scared, my integration tests take at least 3 hours... And yes, I am doing like you describe.
3+ hours means the entire team cannot deploy more than twice during work hours. If your engineering and business leadership is OK with that then what you can do...
Is that the best possible outcome? I think not, see my comment above on how fast we can go with a 1.5h (sequential) build.
As you said whole unit test must run in less than 10 minutes (average)... and our unit tests aren't different. But, our integration tests take more than 3 hours (depends of course of characteristics of these machine).
When you run seeds, and migrations in each test with a hundreds of endpoints I think that time is acceptable, I don't think you are thinking correctly in this particularly kind of tests (integration and acceptance), this must take several hours (it depends how big is your project and your team), and we only run this when we think we have all stuff done to deploy for production. But, in daily work sprints we run unit test in less than 10 minutes of course...
There are many telltale signs that easily differentiate a unit test from an integration test: Encapsulation, Complexity and Test failure. So, I think we're talking about different things...
whether you have a different approach... I am all ears :)
What I refer to as our 1h30min sequential build being cut down to 7mins does include all Cucumber scenarios too (integration & acceptance tests). They all run seeding and db migration prior to launching Firefox for UI testing. So it really is possible. :) We use a Semaphore feature called Boosters, which is Ruby only atm.
We’re a place where coders share, stay up-to-date and grow their careers.
We strive for transparency and don't collect excess data.