My case against running containers in tests

I've been working with Testcontainers for over 3 years in my day job.
Initially I bought into the idea - being able to run service tests locally, with realistic mocks for your external dependencies. But I've changed my opinion.

The main problem is that they require a fair amount of setup and maintenance. Plenty of code will be written, so you need to figure out the right pattern for scaling it. You'll encounter issues with image downloads (there are some beefy images out there), or container instantiation. Especially if you have machines with multiple OSes, or if you run these tests in your CI/CD. And troubleshooting is a pain - often your only recourse is going through the logs of those containers, which is not straightforward.

Because of the effort involved, we ended up with service tests that only covered the same happy path that our end-to-end test did. WTF testing pyramid, I know, but the reality is that we needed to deliver features - it's hard to justify spending a whole day on the service tests (which happened anyway).

Speaking of reality, I didn't feel like these tests gave me that much reassurance on what would happen in a deployed environment. I doubt the containers for our dependencies could replicate how cloud-native services
actually perform - a DynamoDB image wouldn't have the eventual consistency of the actual service, for example. Then there's the configuration that we added to those cloud resources via Terraform; you can't reflect that in
the service tests either.

Okay, rant over. But what is the alternative? I propose a combination of two things.

Numero uno - in your unit test suite, make sure you have tests that run through your entire application, as your service test would've done, with only the external dependency touchpoints being mocked.

Numero dos - create one/multiple playground environment where developers can quickly deploy their branches.

I probably missed out some edge cases where this wouldn't work. But I hope my ramblings managed to convince you, my dear reader, to think carefully before deciding to add container-based service tests.