Better error messages for non-existing API resources with Amazon API Gateway

One of the most popular serverless services in the AWS offering is the Amazon API Gateway. With three flavors of APIs to choose from – REST, HTTP, and WebSocket, the service is an essential piece of the AWS serverless ecosystem.

This blog post will look at the REST and HTTP versions of the Amazon API Gateway APIs and how to ensure the path or resource not found responses contain messages which are helpful to the end-user.

You can find the code from this article in this GitHub repository.

The problem

We all make mistakes – after all, we are only human. When interacting with APIs, one of the mistakes one can make is to try to request a resource that does not exist (either the path or the method is incorrect).

In an ideal world, the response generated from the API would help the user quickly resolve their issue – in our case, amend the request to use the correct resource/method combination.

In the case of the Amazon API Gateway, the default behavior for handling such situations is not ideal, especially when it comes to the REST APIs. Let us look at that flavor of the Amazon API Gateway next.

Unknown resource response in the context of Amazon API Gateway REST API

By default, if the request resource/method combination is incorrect, the API Gateway REST API will respond with the following payload with a statusCode of 403.

{ "message": "Missing Authentication Token" }

Ouch, there is room for improvement here. In fact, even the official documentation states that this behavior is confusing.

If the official documentation states that the default response is confusing, that begs the question, why have such a response be the default in the first place?

To my best knowledge, we can significantly improve the API ergonomics in this area by either modifying Amazon API Gateway gateway responses or by having a "greedy" route with ANY method that acts as a fallback for such requests. Since I'm biased toward solutions that involve configuration, let us first start with gateway responses.

Modifying gateway responses

The gateway responses define what happens if the API Gateway fails to process an incoming request.

In our particular scenario, we are interested in amending the MISSING_AUTHENTICATION_TOKEN response type – that is the response type returned to the end-user when the method/resource combination of the request to the REST API is incorrect.

The following is an AWS CDK TypeScript snippet that re-defines the gateway response in question.

const restAPI = new aws_apigateway.RestApi(this, "RestAPI", {
  /**
   * Configuration...
   */
});

restAPI.addGatewayResponse("PathOrMethodMaybeFound", {
  type: aws_apigateway.ResponseType.MISSING_AUTHENTICATION_TOKEN,
  templates: {
    "application/json": `{"message": "$context.error.message", "hint": "Is the method/path combination correct?"}`
  }
});

1. Since the MISSING_AUTHENTICATION_TOKEN response type could also be triggered when the authentication token is missing (the method/resource combination is correct), I've opted to leave the original error message intact.

2. I did not employ any "smartness" to the template since one cannot use any VTL programming constructs in the gateway responses templates – API Gateway will ignore them. I could have tailored the message even more, using conditional statements, but in the gateway responses templates, I'm constrained to only using variables.

Now, If I were to make the request to the API and mistype the path or use a wrong method (for example, POST instead of GET), I would get the following response.

{
  "message": "Missing Authentication Token",
  "hint": "Is the method/path combination correct?"
}

An improvement, but we can do better. Let us look at the second way I mentioned earlier – using a "greedy" path with the ANY method.

Deploying a "greedy" route with ANY method

As an alternative to modifying the gateway responses, one can deploy a "greedy" route with the ANY method. Suppose the API Gateway invokes the integration backing that route. In that case, we can be confident that none of our other routes match the request instead of lacking the authentication token (if our API requires authorization).

Note
In some cases, developers might use the ANY method with a "greedy" route for monolithic AWS Lambda deployments (think running express on AWS Lambda. While I'm not a fan of such practice, I also understand that it might work well for some. If that is the case for you, you should consider tackling the issue I'm describing using your framework of choice.

Greedy route backed by AWS Lambda function

This is, arguably, a more accessible, from the IaC configuration standpoint, variant to pull off. The "greedy" ANY route is backed by an AWS Lambda function. Since we have the power of a programing language at our disposal, one has the opportunity to tailor the "method/path does not exist on this API" message to a given API user.

Note
AWS Lambda function is not the only service the API Gateway can integrate. In this scenario, AWS Step Function Express workflows would also do the trick.

The following is an AWS CDK definition of the "greedy" ANY route backed by the AWS Lambda function.

const restAPI = new aws_apigateway.RestApi(this, "RestAPI", {
  /**
   * Configuration...
   */
});

const anyAPIHandler = new aws_lambda_nodejs.NodejsFunction(
  this,
  "AnyAPIHandler",
  {
    entry: join(__dirname, "./handler.ts")
  }
);

restAPI.root
  .addResource("{proxy+}")
  .addMethod("ANY", new aws_apigateway.LambdaIntegration(anyAPIHandler));

And here is the handler.

import { APIGatewayProxyHandler } from "aws-lambda";

export const handler: APIGatewayProxyHandler = async () => {
  return {
    statusCode: 404,
    body: "Method/Path combination not found. Please check the documentation."
  };
};

Now, If I were to call the API and use either an incorrect method or path (or both) instead of the generic Missing Authentication Token message, the response would be the one I've encoded in the AWS Lambda.

Method/Path combination not found. Please check the documentation.

One important thing to remember when going this route is to ensure that we treat the "greedy" route AWS Lambda function just like any other AWS Lambda functions in our infrastructure. This means adding throttling limits and so on (if applicable). The last thing you want is a denial of wallet attack just because you forgot to add authorization to the "greedy" endpoint.

Greedy route backend by Mock Integration

The second, and the option I'm more in favor of, is to ditch the AWS Lambda function and use the Amazon API Gateway Mock Integration as the "brains" that creates the response for the "greedy" ANY route.

The following is an AWS CDK definition of the "greedy" ANY route backed by Amazon API Gateway Mock Integration.

const restAPI = new aws_apigateway.RestApi(this, "RestAPI", {
  /**
   * Configuration...
   */
});

restAPI.root.addResource("{proxy+}").addMethod(
  "ANY",
  new aws_apigateway.MockIntegration({
    passthroughBehavior: aws_apigateway.PassthroughBehavior.NEVER,
    requestTemplates: {
      "application/json": `{"statusCode": 404}`
    },
    integrationResponses: [
      {
        statusCode: "404",
        responseTemplates: {
          "application/json": `"Method/Path combination not found. Please check the documentation."`
        }
      }
    ]
  }),
  {
    methodResponses: [{ statusCode: "404" }]
  }
);

The main benefit of using the mock integration as opposed to the AWS Lambda function is that you do not have to pay for AWS Lambda function invocations. The main drawback is the fact that now you have to deal with mapping templates, VTL, and relatively complex IaC configuration.

Remember that mock integration templates allow you to perform conditional logic – something the gateway responses templates lacked (worth noting that you are still unable to harness the full power of the VTL, like in the case of AWS AppSync resolver templates).

Unknown resource response in the context of Amazon API Gateway HTTP API

By default, if the request resource/method combination is incorrect, the API Gateway HTTP API will respond with the following payload with a statusCode of 404.

{
  "message": "Not Found"
}

Not bad! An improvement over the REST API counterpart. You might wish to improve the message even further – in that case, let us look at how you could do that.

Greedy route backed by AWS Lambda function

Since the HTTP flavor of the API Gateway API does not expose any way to manipulate request/response templates, to my best knowledge, the only way to change to configure this behavior is to have some compute service behind a "greedy" ANY route. Like Amazon API Gateway REST API, one might use the AWS Step Function Express Workflows or the AWS Lambda function to achieve the desired result.

For completeness, here is the AWS CDK definition of a "greedy" ANY route for API Gateway HTTP API backed by AWS Lambda function.

const httpAPI = new aws_apigatewayv2_alpha.HttpApi(this, "HttpAPI", {
  /**
   * Configuration...
   */
});

const greedyPathHandler = new aws_lambda_nodejs.NodejsFunction(
  this,
  "GreedyPathHandler",
  {
    entry: join(__dirname, "./greedy/handler.ts")
  }
);

httpAPI.addRoutes({
  integration: new aws_apigatewayv2_integrations_alpha.HttpLambdaIntegration(
    "GreedyPathIntegration",
    greedyPathHandler
  ),
  path: "/{proxy+}",
  methods: [aws_apigatewayv2_alpha.HttpMethod.ANY]
});

And here is the handler code.

import { APIGatewayProxyHandlerV2 } from "aws-lambda";

export const handler: APIGatewayProxyHandlerV2 = async () => {
  return {
    statusCode: 404,
    body: "Method/Path combination not found. Please check the documentation."
  };
};

Like in the case of the REST counterpart, there is not that much to it. I have intentionally did not add any logic to my handlers as the logic might be highly dependent on your specific case.

Closing words

We can, and I would argue we should change the default response the API Gateway returns when the user makes a request to our API with an incorrect method/path combination, especially for the REST flavor of the API Gateway APIs. The confusion one single error message could cause cannot be underestimated.

Consider following me on Twitter – @wm_matuszewski if you are interested in similar content like this blog post.

Thank you for your precious time.

Comments

[jledesma84]

Hi there. This post is truly remarkable. I'm trying to implement Greedy route backend by Mock Integration using SAM, but with no success. I'm getting 500. I wonder if you can give me a hand with it?

`
MyApiGateway:
Type: AWS::Serverless::Api
Properties:
Name: My API
TracingEnabled: true
StageName: !Ref Environment
GatewayResponses:
BAD_REQUEST_BODY:
ResponseTemplates:
"application/json": '{"description": "$context.error.validationErrorString"}'

ProxyResource:
Type: 'AWS::ApiGateway::Resource'
Properties:
ParentId: !GetAtt MyApiGateway.RootResourceId
PathPart: "{proxy+}"
RestApiId: !Ref MyApiGateway

ProxyMethod:
Type: 'AWS::ApiGateway::Method'
Properties:
HttpMethod: ANY
ResourceId: !Ref ProxyResource
RestApiId: !Ref MyApiGateway
AuthorizationType: NONE
Integration:
Type: MOCK

IntegrationResponses:
- StatusCode: '404'
ResponseTemplates:
application/json: "{\"message\": \"Not Found\"}"
MethodResponses:
- StatusCode: '404'
`

[Yarnest]

Thank you very much very useful post. I find the documentation of api gateway specially confusing.