In this article we will discuss schema validation testing using Prism, the differences it has with contract testing and provide an introduction on how we can run it locally.
Current state and issues
Our team is part of an organisation whose aim is to create a cloud-native SaaS bank operating system that will empower banks to evolve to a new era by leaving behind the (old) monolithic approach.
As part of the development lifecycle, when we release our code or product, we need to be as much accurate and confident as possible. One of the common undesired occasions is when we receive customer’s feedback during that lifecycle, which causes a lot of back and forth that actually costs.
The preferred way would be to have a frequent check that will most probably catch errors easier and earlier before reaching the end user. This is where Prism comes into play.
In our case, we had an e2e suite that was running user journeys against microservices. Obviously that is not a one of a kind situation, but due to the architectural structure the microservices were being hit either directly or via proxies (Apigee or Ambassador) so we needed a way to validate the requests and the responses.
One feasible way where we can set up a structure and actually validate the requests and the responses would be with PACT contract testing. This approach will provide the confidence that we seek for but the disadvantage occurs on the amount of time and effort that needs to be spent both from the Consumer and the Provider teams. However, despite having PACT tests, there are cases where we need to be as autonomous as possible and do a validation within the team’s scope. Here is where Prism comes into play.
What is Prism?
Prism is an open-source HTTP Mock & Proxy Server that acts as a transformation and validation layer between the actual and the expected result. It can be used as a contract testing tool as well as a logging one. Prism works in collaboration with OpenAPI v2 and OpenAPI v3, that generates request and response objects based on the provided yaml or json files.
So, in our case, during the execution, we had it validating the called endpoint’s request and response bodies against those generated objects.
In case that any of the examined objects did not match with the expected ones, Prim provides information regarding the error that can be easily used to generate a report.
Schema testing vs Contract testing
Schema testing uses a generalised notation that defines the structure that a request and response are supposed to have at a point in execution time. It validates that a system is compatible to a given schema.
Contract testing, on the other hand, defines how two systems are able to communicate by agreeing on what should be sent between them and providing concrete examples (contracts) to test the agreed behaviour.
The difference between them arises with contract testing going one step further on just defining a schema, requiring both parties to come to a consensus on the allowed set of interactions allowing evolution over time.
In simple words, contract testing is more concrete as it defines strictly how the two systems are supposed to communicate, while schema testing is more abstract as there is a general validation on the expected structure of the request and response payloads.
How to use Prism
Our E2E suite is developed using Spring Boot, so we decided to use the Spring’s Thymeleaf library to construct and customise the reports and then inform the related microservice teams. Using Thymeleaf we created a boilerplated prism-report.html file that was updated on demand when errors occurred during the execution.
As already stated, Prism can help checking for discrepancies between an API implementation and the OpenAPI document that describes. To start of we need to install Prism either locally
npm install @stoplight/prism-cli
or via a Docker image:
docker pull stoplight/prism
The generic run command is:
npx prism proxy <OPEN_API_SPEC_YAML_FILE> <UPSTREAM_URL> — port <PROXY_PORT>
where,
a) OPEN_API_SPEC_YAM_FILE is the yaml file that OpenAPI uses to generate the DTOs
b) UPSTREAM_URL is the host/proxy that the request goes by
c) PROXY_PORT is the assigned port for requests (we can provide any port we want)
How we decided to use it
Finally, in order to make the whole process as autonomous as possible, we created a Jenkinsfile that runs in a daily manner and does the aforementioned validation and then posts the results on a specific Slack channel.
As shown below there is a stage in our Jenkinsfile that runs a bash script to kick off the validation. The whole suite is Dockerised so we have the ability to provide the service(s) as environment variable(s). We can define, with PRISM_PROXY_CHOICE, if we want to run a specific service or all of them.
stage("Run Prism Schema validation") {
steps {
catchError(buildResult: 'SUCCESS', stageResult: 'UNSTABLE') {
script {
sh '''
if [ "${PRISM_PROXY_CHOICE}" != "All" ] ; then
export PRISM_PROXY_SERVICES=${PRISM_PROXY_CHOICE};
fi
./ci/execution/dockerRunExecutor.sh
'''
}
}
}
}
In our bash script we retrieve the OpenAPI specs that Prism will run against. In the code block below, you will observe that we have assigned a specific port for each Apigee/Ambassador service, so we can have the validations isolated and the produced result will be categorised per service. So, for example, if a request is made to any endpoint of firstgateway it will automatically be redirected to localhost:5000 where the Prism Server will perform the schema validation.
OPENAPI_SPECS_PATH=./src/main/resources/openapi-specs
GATEWAYS_PORT_MAPPING=("firstgateway:5000"
"secondgateway:5001"
"thirdgateway:5002"
"fourthgateway:5003"
"fifthgateway:5004"
"sixthgateway:5005"
"seventhgateway:5006"
"eighthgateway:5007"
"ninethgateway:5008")
if [ -z "${PRISM_PROXY_SERVICES}" ] || [ "${PRISM_PROXY_SERVICES}" = "All" ] ; then
#If prism prismProxyService is empty then start proxy service for all available gateways
echo "Starting prism proxy for all gateways"
for gatewaymap in "${GATEWAYS_PORT_MAPPING[@]}"; do
GATEWAY=${gatewaymap%%:*}
PRISM_PORT=${gatewaymap#*:}
prism proxy -h 0.0.0.0 ${OPENAPI_SPECS_PATH}/${GATEWAY}.yaml ${PRISM_UPSTREAM_URL} -p ${PRISM_PORT} &
done
else
for prismProxyService in ${PRISM_PROXY_SERVICES//,/ }; do
for gatewaymap in "${GATEWAYS_PORT_MAPPING[@]}"; do
GATEWAY=${gatewaymap%%:*}
PRISM_PORT=${gatewaymap#*:}
if [ ${prismProxyService} == ${GATEWAY} ]; then
echo "Starting Prism Proxy Server for ${GATEWAY} on port ${PRISM_PORT} listening to ${PRISM_UPSTREAM_URL}"
prism proxy -h 0.0.0.0 ${OPENAPI_SPECS_PATH}/${GATEWAY}.yaml ${PRISM_UPSTREAM_URL} -p ${PRISM_PORT} &
fi
done
done
fi
During execution, we have created a store where we keep information regarding each request (service name, path, method, response). Prism adds an extra sl-violations header, which we use after each test to update a PrismStore that saves the above information plus some extra ones like severity, error location and response code. That store will be used with Cucumber’s AfterAll annotation to complement our custom Thymeleaf report file (as shown in the last image).
After the execution stage is completed, as part of the clean up stage, we generate the report and call sendMessage() method to send a slack notification to ensure that all related teams would be informed soon enough to proceed with any needed updates before the broken functionality is deployed to any client environments.
post {
always {
script {
sh '''
./ci/utils/cleanUp.sh
'''
publishHTML(target: [
allowMissing: false,
alwaysLinkToLastBuild: false,
keepAll: true,
reportDir: 'target/prismproxy/',
reportFiles: 'prism_report.html',
reportName: 'PrismProxyReport',
reportTitles: 'Prism Test Report'])
sendMessage()
}
}
}
Conclusion
To sum up, in this article we tried to have our first approach with Prism. We explained what Prism is and what can it bring to the table. We mentioned the key differences between Prism as a schema validation tool and Pact as contract testing tool. Last but not least we explained how we decided to use Prism as part of our E2E suite and generate a report that will provide handy results to any team during the development lifecycle.
In case you are looking for a dynamic and knowledge-sharing workplace that respects and encourages your personal growth as part of it’s own development, we invite you to explore our current job opportunities and be part of Agile Actors!
Top comments (0)