Current Situation
Now I am building the CRUD API endpoints for the system, there is no WebSockets and no event-driven architecture and there is not any advanced concepts.
I wanted to start setting up the testing environment and workflow from the beginning. Also I need to try Test-Driven Development, so I am going to apply it at some parts in the application. Now the testing environment is completely ready and the first integration test is ready.
Unit and Integration Tests
In this project I found that I need more integration tests, actually in many situations I need to check what is the behavior of the endpoint at specific situations. So most of the tests will be like "simulate a client sending a request, then check the response and/or the database state.". Hence, there will be more integration tests than unit tests.
I know this deviates from the classic testing pyramid. However, given the nature of this codebase as its mainly a CRUD orchestration without huge complex business logic computations, leaning on integration tests makes more sense here than chasing the pyramid's default ratio.
Setting up the Testing Environment
1. Tools
First of all, I used pytest as the testing framework that helps me write and run tests. I also need a way to send requests to my FastAPI app and get real responses back, so I used httpx.AsyncClient, which calls the app's code directly in-process, simulating real HTTP requests without needing an actual running server or network layer that maps the HTTP requests to the actual code.
2. Testing Database Setup
This is very important point, if the tests use the real production/development database, they will pollute data. So there must be a testing database that are made specifically for the tests to use, it should be a completely separate, throwaway database.
Test Containers
On every run for the tests, we can start an actual Postgres Docker container that contains the testing database server. Then we run the existing migrations against the new server, so we get a new database server with a new database with the required schema (same as the production/development database) just for testing.
To achieve this I used testcontainers-postgres to spin up a real, temporary Postgres Docker container once per test session (not per test, since that'd be slow). That is the separate database server that is thrown after every test session.
3. Overriding the Injected Database Dependency
For now when FastAPI notices that an endpoint needs a database session, it injects one that is connected to the real database.
What we need is to tell FastAPI to inject a session connecting to another database server which for testing.
4. Isolate the Tests
We have to isolate the tests so each test sets its own data inside the database and then rolled back. We need every test to start clean, without leftover data from a previous test, so test A data changes never affect test B.
This can be achieved by utilizing the Isolation principle in ACID principles that PostgreSQL DBMS follows, stating that any uncommitted changes in a session are isolated and can not be visible by other sessions. So we just rollback any changes a test made so there is not any persisted durable data on the database, every test make its changes via an separate isolated session and never really commits inside the database (and any commits made during the test are absorbed by an outer transaction that is always rolled back after each test, commits are considered savepoints).
Interesting Issue Faced Me
One subtle issue I hit: my connection pool (that managed by test_engine object) was session-scoped for performance, but pytest-asyncio creates a new event loop per test method by default. Async database connections are tied to the event loop that created them, so reusing a reserved connection across tests (across event loops) caused failures. The fix was configuring pytest-asyncio to share a single event loop for the whole session, matching the connection pool's lifetime, so there is only one event loop that will use connections for the single test engine.
Top comments (0)