When you build an API testing framework, checking status codes is the easy part.
assert response.status_code == 200
The hard part is validating the JSON payload. You need to check that the fields are correct, that nested data exists, and eventually, that the schema matches what the API documentation promised.
I've been building Praman, an open-source QA engine in Python. I just finished the API automation layer, and the test suite is now at 663 passing tests. Here is the architectural decision I had to make when adding JSON Schema validation.
If you want to validate a JSON payload against a schema in Python, the standard tool is the jsonschema library. It is excellent, it is powerful, and it supports Draft 2020-12.
But here is the problem: I don't want to force every user of Praman to install it.
Some teams just want to make a GET request and check a few fields. They don't need a full schema validator. If I make jsonschema a required dependency in my pyproject.toml, I am bloating their environment for a feature they might never use.
So I used an optional dependency architecture.
I put the schema validation logic in a dedicated module, but I didn't add jsonschema to the required dependencies. Instead, it lives in an optional extra. If a user tries to call the schema validation method without having the library installed, Praman raises a clear error message:
"To use JSON Schema validation, please install the optional dependencies: pip install praman[schema]"
This is how professional libraries are built. The core package stays lean, and the advanced features are opt-in. It respects the client's environment.
I separated the API assertions into three levels of complexity.
First, top-level field validation. If I just want to check that the response contains a user_id and a status field, I can do that without any external libraries. It's just a dictionary lookup.
Second, nested path validation. If I need to check response["data"]["attributes"]["created_at"], I wrote a simple path resolver. Again, zero dependencies.
Third, full JSON Schema validation. This is where the optional dependency kicks in. If the client actually needs to enforce a Draft 2020-12 schema, they install the extra.
This layered approach means the framework is useful immediately for simple tests, but it scales up when the project gets serious.
When you test APIs, you need to know if the request actually succeeded.
The second half of the API layer was about handling authentication and headers cleanly. I added a Praman-owned ApiClient. It handles Bearer authentication, configurable headers, and the standard HTTP methods: GET, POST, PUT, PATCH, and DELETE.
More importantly, it validates HTTP status codes automatically. If an API returns a 500 when you expected a 200, the framework fails the test with a clear message, rather than forcing the test author to write assert response.status_code == 200 on every single test.
The test suite is now at 663 passing tests. The full build passed on a clean wheel installation, meaning a client can install Praman, run an external API test, and everything works.
If you want to look at the code, the repository is public on GitHub: https://github.com/pranromumu/praman
Next up is the remaining Phase 2 work: UI automation and Page Objects. The API foundation is done. The framework is growing.
Top comments (0)