Note: This post was originally published in Shaadi tech blog and can also be accessed at: https://tech.shaadi.com/2020/08/10/managing-unexpected-data-at-runtime-in-typescript/
How it all began..
In order to ensure an awesome matchmaking journey at Shaadi, we are always releasing new features. To engineer this process for speed and reliability, we use various technologies and software development practices. Typescript is one such technology that has immensely helped us in this mission.
Typescript provides us with static type-checking which can identify many programming errors at compile-time resulting in fewer bugs and reduced overall time spent on debugging. It does so by statically analyzing the code and finally compiles it to Javascript.
But why should we care about runtime type safety?
Typescript will not make any assertions on the data during runtime since all the type information is stripped off during the compilation phase itself. That means, if the data received from external sources like an API or a Cache is in an unexpected shape or differs from the type that we initially declare, it will most likely cause bugs that may be hard to track down since they will occur far from the root cause of these bugs.
In this article, I will be exploring how we can take a step forward from what Typescript provides us and embrace type-checking at runtime for handling external data.
Where Can This Help Us?
Recently, I was working on adding a feature that allows members to Login on the website using an OTP instead of the traditional way that uses a password. It’s a two-step process wherein members will first enter their registered email address or mobile number to receive an OTP. Then they will enter this OTP on the next screen which after validation, will log them into their account.
On each step, an API (as described below) will handle the user input
/otp/send ->
This will validate the username and send an OTP to the user
/users/login ->
This will validate if the user has entered the correct OTP and handle success and failure cases accordingly.
This was a good use case where I could introduce type safety for the API layer.
What options do we have?
Languages like Java and Golang can access type information during runtime using reflection. Rust has a feature called macros which can automatically build decoders for a particular struct at runtime. But this is not possible in Typescript.
Manually writing decoders will be a lot of work and also get difficult as the complexity of the data increases. Instead, we will be using a library that will help us create decoders with ease so that we can validate the data against the validators at runtime to check if it matches our expectation.
We have two major libraries for this in Typescript io-ts and runtypes. Check out the npm trends for these libraries below.
Both the libraries are suitable for the task of validating data at runtime. Let us see an example of how we used io-ts in our application at Shaadi.com to achieve this.
Defining validators for our example
The following code defines the shape of the response that is expected from the API:
import { type, boolean, string, intersection, TypeOf, partial } from "io-ts";
//Dynamic Types
const sendApiResponse = type({
is_sent: boolean,
text_message: string
});
const validateApiResponse = intersection([
type({
uid: string,
next_action: string,
}),
partial({
session_id: string,
domain: string,
access_token: string,
email_token: string
})
]);
//Static Types
export type sendApiResponse = TypeOf<typeof sendApiResponse>;
export type validateApiResponse = TypeOf<typeof validateApiResponse>;
The above-defined dynamic types can be used as validators for our API response. Since we would also need static types for compile type, io-ts provides a Typeof operator which extracts a static type from the validator.
In the validateApiResponse type, some of the properties are optional considering cases like deactivated user where a session_id or access_token won’t be supplied by the API. Hence, we have to create a mixed type by passing an object with the required properties to type and optional ones to partial. Also, next_action property should ideally be a union of valid next states that are possible for the user. If anything apart from that is received from the API, it should instantly fail with an error message.
Also, one more detail to note here is that we should exclude those properties from the validator which we are not going to use in our application. This also answers the question – What if the API signature changes in the future? As long as it doesn’t mess with the data that we are using, it won’t affect us.
Decoder for enforcing the validators
Now that we are done defining the validators, it’s time to write code that enforces these type definitions at runtime. Check out the below code which I have written for this specific example.
import { Type } from "io-ts";
import { fold, pipe } from '~/helpers/fp-utils.ts';
//Decode API Response
export function decodeResponse(
apiType:string,
response:unknown
):Promise<unknown> {
switch(apiType){
case 'send':
return decodeToPromise(sendApiResponse,response);
case 'validate':
return decodeToPromise(validateApiResponse,response);
default:
return Promise.reject({ validatorErrors: "Incorrect Validator" })
}
}
//Validate Dynamic Types
function decodeToPromise<T, O, I>(
validator: Type<T, O, I>,
input: I
): Promise<T> {
return pipe(
validator.decode(input),
fold(
// failure handler
(errors) =>
Promise.reject({ validatorErrors: JSON.stringify(errors) }),
// success handler
(result) =>
Promise.resolve(result)
)
)
}
The function decodeResponse does nothing but helps us keep the decoder logic generic by passing it appropriate parameters for each API. Before we move on to the logic for decoder lets understand a few things that we should understand (the bare minimum, feel free to further explore on each concept).
A value of type Type<A, O, I> (called “codec”) is the runtime representation of the static type A
The pipe sends the output of one function to another for further processing
The fold is used in the above program to handle success and failure case from an Either type
The Either<E, A> type represents a computation that might fail with an error of type E or succeed with a value of type A
The function decodeToPromise returns a promise based on whether the API response conforms to the shape defined in the validator.
validator.decode(input)
The above line in the code decodes the input based on the validator and returns an Either<E, A> type which is then handled by fold.
Also, one more detail to note here is that the Typescript implementation for pipe and fold is present in a library called fp-ts which is a dependency for io-ts. But since the cost of adding the package fp-ts is considerably high, we are maintaining the implementation of only those functions that we use in fp-utils.ts.
Finally, check out the code to use the decoder during an API call.
import { sendApiResponse, decodeResponse } from './types'
try{
const response:sendApiResponse = await generateOtp({ username, type: 'login' }).then((res:unknown) => decodeResponse('send',res));
... success logic ...
}
catch(error){
error.hasOwnProperty("validatorErrors") &&
throw new Error(error.validatorErrors);
... failure logic ...
}
The above code just calls the function decodeResponse (which we defined earlier) with the response supplied by the API. If an error occurs, the catch block will check if it is from the validator we defined and throw a meaningful error message instead of just carrying on the execution with improper data that will create unwanted side effects later.
In summary
Even though Typescript helps in reducing certain bugs by providing compile type safety, the lack of runtime type checking can still introduce some bugs which could be very hard to debug
To make this job easier, we used a library io-ts that can quickly identify bugs that arise from the unknown nature of the external data
It helped us to create validators for our API requests which will throw meaningful error messages if the response is not in expected shape
This was a quick introduction on using io-ts to manage data from an API at runtime. It’s not just limited to dynamic data from API requests but can also be used to create validators for all types of external data that our app has to handle. This will help us in identifying issues that arise due to incompatible dynamic data, thus saving hours of debugging time and frustration.
This article has just scratched the surface. You can achieve a lot more using io-ts and runtypes. I will suggest you to experiment with it and decide if it’s helpful for you. Also, if you would like to try functional programming in Typescript – do check out the library fp-ts. It provides us with popular patterns and abstractions that are available in most functional programming languages.
Thanks for reading. Do comment if you have any questions or feedback.
Top comments (0)