Have you ever integrated with an internal/external service? Ever wondered how the service is returning different kinds of errors & issues so quickly? Or about the process that is happening behind the scenes to facilitate this?
Yes, you are right. We are talking about the validations!
Just imagine what a chaotic world it would be if there are no validations or prechecks in a service.
There would be just a successful response if you are very lucky, or there would be an arbitrary error, mostly internal server error or request timeout, which leaves you with no clue of what went wrong.
Thankfully someone has thought through this issue and found a way out of it.
What is API Request Validation?
API request validation is the process of checking the alignment of client requests with the laid-out API specifications.
It’s a very crucial process in the request lifecycle. An API request needs to be validated for correctness before processing further to avoid unexpected/wrong errors and slow/invalid responses.
Why do We Need API Request Validation?
In the context of REST APIs, a request body is deliberately designed based on the required parameters for the processes happening behind the scenes. If there is a miss in any mandatory parameter(s), the process will fail.
Now there is nothing wrong with the process failing. However, if we talk about scale, there is a cost involved both for the client and the server; just think of the cost as request latency and server resources. All that is because of a miss in the request body, which can be handled in a much better way at the beginning of the request.
In the coming sections of this blog, we will be exploring different ways to handle cases like these and how we at Decentro are dealing with them, then we will have a quick implementation of the same.
Our core application is on Flask; hence, the illustrations will be similar. Flask is a web framework for python. It is a small and easy to extend microframework without the native support of ORM.
API INIT – Initial part of the flow, where we check the credits, headers, auth, etc.
API Process INIT – The core flow is based on the business logic.
What are We Trying to Do Here?
Consider the following request body, we will be using the same throughout the blog post.
We want to validate the request body and transform in some cases, based on whether all the required parameters are received and all the values obtained for the parameters are of acceptable types and ranges. Consider the following validation rules for this request body.
name
- It should be a json object
- first and last are required
- middle is optional
- All nested values are string
date_of_birth
- Should be a string, matching YYYY-MM-DD format.
gender
- Should be a string.
- Should match any of the following [M, F, O]
- Should transform the value to upper case before checking the match.
address
- It should be a json object
- flat_number, locality, and pincode are required
- landmark is optional
- pincode should be of six-digit string
- need to extract area, city, and state from pincode and add them to this object
social_presence
- this is a list of object
- the list can be empty
- if an object is present then it should be mandatory to have site_name and site_url.
Now let’s have a look at some of the ways we can handle the verification and validation of the above request packet given the domain rules.
Different Ways to Implement API Request Validation
The Noob Way
You can write a method for this API to validate the request body and raise an exception if something goes wrong. For instance,
This type of implementation will work, but there are a few issues with it
- This will not be easy to manage and maintain, and there will be more chances of missing a parameter validation.
- The complexity of handling this will increase when we add more parameters to the request body, adding 2nd order nesting.
- Think of a scenario where we need to add/update a parameter in one of the nested blocks in further API iterations. Then?
Is there any better way to do this job?
Yes, of course!
The Champ Way!
We can use serializers.
Serializers are built to convert data from one form to another; For instance, it can convert a queryset into a python dict or create a data class from a JSON object. We can leverage this functionality and add validations between the flow.
In serializers, we define a schema with validation rules and transformation methods. Then, we need to load the data in the serializer, and if there is an issue with the data, the serializer will raise an exception or return the transformed data.
We can even add a pre-serialization method and post-serialization to pre-process and post-process the data, which can be executed in the respective sequence.
These methods are also beneficial. Consider the case of gender parameter; it is a categorical variable and case insensitive, so we need to upper-case this before further validation check.
We will be using this approach to validate the request body in this blog post. And we at Decentro are following a very similar approach to deal with validations and transformation.
Here Comes Marshmallow!
Marshmallow is the object serialization library that we will be using. We will be resolving the nested blocks first.
Resolving Name Block
Resolving the Address
Resolving the Social Presence, Each Object
Now, since we have built subschemas for all the nested blocks, let’s build the schema for the parent block.
Finally, we just need to load the request data and handle the exception raised from the schema, if any, and handle it appropriately.
And, that’s all we have to do in order to validate the request body.
Just to complete things, this is how you will be seeing the errors, if there is a miss in the request body.
Just to complete things, this is how you will be seeing the errors, if there is a miss in the request body.
What’s Next?
This was a brief introduction to the serializer-based validation using marshmallow. There is a lot more to explore in this, many more cases to be handled, listing a few of them which are in my thoughts are following.
- Conditional validation in the nested fields, think of the scenario where one parameter in a block has a direct relationship with a parameter in another block or consider parent block parameter relationship with the child block parameter.
- Advanced pre-processing and transformation of the request body, optimizing the request body, consider it as taking a few parameters from the client and transforming them at the server end, like what we have done for the address field.
- Reusing existing serializers in other APIs, think of name and address as nested fields in other APIs. We can even extend this without changing the base serializer; these are classes only. So let’s suppose you want to have a salutation field in the name block in one of the APIs. You can extend the name serializer and add salutation and related validations; the rest are already there. How amazing! Right?
We can use these API serializers to generate dynamic documentation of your API. These serializers hold everything you need for the API documentation, parameters, accepted values, accepted types, accepted ranges, error messages.
What would you like us to take up in this series? Let us know in the comments; we’re all ears! 😇
Final Words
So till now, I guess I was able to illustrate my point of using serializers for the request validation and how effective the whole process is in order to maintain and extend the API validations.
You can find the fully functional code of the example mentioned in this blog on Decentro’s GitHub page.
References
Originally published at: https://decentro.tech/blog/pocket-guide-to-api-request-validation/
Top comments (0)