Building Serverless APIs on AWS

Application Program Interfaces (APIs) are essential to modern application architectures. A lot of the biggest SaaS companies are designed with an API-first approach, where APIs are at the core of the software design. (AWS is API-first.)

“API is the acronym for Application Programming Interface, which is a software intermediary that allows two applications to talk to each other.” — Mulesoft

Some of the benefits of APIs are:

• Better and easier integration
• Improved reliability, transparency, observability, performance, and scalability
• Enable modular code and features
• Building future-proof applications
• Improved feature isolation

A REST-API is an API that adheres to the REST conventions and constraints. HTTP methods are used to interact with REST APIs: GET, POST,PUT, PATCH, DELETE that correspond to CRUD operations (CREATE, READ, UPDATE, DELETE). Just like web servers serve HTML and CSS in response to HTTP request to the HTTP endpoints, an API will serve JSON content in response to the requests.

To demonstrate how to interact with APIs, let’s explore Yahoo’s public finance API. You will need to create an account to generate your API key. API Keys are used for authentication to the APIs, and are included in the headers of the HTTP requests. You should never hardcode your API keys in your code. Use environment variable that are not committed to your code repo, or rely on a secrets manager.

We will rely on the Yahoo finance API to get a real-time market price from the quote of the SPDR S&P 500 ETF (ticker: SPY). In the swagger definition, the API endpoint of interest is /v6/finance/quote. We can see that it corresponds to a HTTP GET endpoint.

This endpoint requires two optional and one required URI parameters as arguments:

• region (str, optional): defaults to US
• language (str, optional): defaults to en
• symbols (str, required): the ticker of the stock, ETF, mutual fund, … for the requested quote

Every API HTTP Request has a HTTP Status Code that is part of the response. It is important to understand HTTP Status Codes for debugging your code:

• 1xx Informational
• 2xx Success
• 3xx Redirection
• 4xx Client Error
• 5xx Server Error

For the quote endpoint, we can see that it should return a 200 HTTP Status Code for success.

Building an API Wrapper for the Yahoo Finance API using requests, my python library of choice:

import requests
from datetime import datetime, timedelta
from requests import Response
from requests import RequestException
from os import getenv

class YahooFinanceApi:
  def __init__(self) -> None:
    self.yahoo_finance_api_key = getenv("YAHOO_FINANCE_API_KEY")
    self.api_base_url = "https://yfapi.net"
  def get_headers(self) -> dict:
    """Creates the headers for the API request. Includes the API Key for authN"""
    headers = {
      'accept': 'application/json',
      'X-API-KEY': self.yahoo_finance_api_key
    }
    return headers
  def get_price_from_quote(self, ticker, region="US", language="en") -> dict:
    """
    Retrieves the price from a quotation.
    Args:
      ticker(str): the ticker of the stock, ETF, mutual fund, ...
      region (str, optional): Defaults to US.
      language(str, optional): Defaults to en.
    """
    endpoint = f"{self.api_base_url}/v6/finance/quote?region={region}&lang={language}&symbols={ticker}"
    headers = self.get_headers()
    try:
      quote = requests.get(url=endpoint, headers=headers)
      if quote.status_code == 200:
        quote = quote.json()
        return {
          "ticker": ticker,
          "price": quote["quoteResponse"]["result"][0]["regularMarketPrice"]
        }
      else:
        return {
          "HTTP status code" : quote.status_code,
          "Content": quote.content
        }
    except RequestException as exception:
        return {"error": str(exception)}

Now, executing:

quote = YahooFinanceApi().get_price_from_quote("SPY")

Returns:

quote = {
  'ticker': 'SPY',
  'price': 451.54
}

I added error handling to the API Wrapper. If the HTTP response Status Code is not 200, it means that the request failed. Let’s try to execute the request with an invalid API Key.

After purposely invalidating my API Key in my .env file, executing:

quote = YahooFinanceApi().get_price_from_quote("SPY")

Returns:

{
  'HTTP status code': 403,
  'Content': b'{"message":"Forbidden","hint":"Sign up for API key https://financeapi.com/tutorial"}'
}

The serverless approach to building APIs is gaining traction with the rise of Cloud Computing and the improvement and diversification of serverless offerings. Serverless APIs make deployments easier and let you leverage other cloud-managed services, like Amazon Cognito for Authentication.

Now that you have a solid understanding of APIs, let’s build a serverless API by leveraging the following AWS Services:

• API Gateway
• Lambda
• DynamoDB

Our API will be simple. We will create a database management tool that allows us to add items to a DynamoDB database.

1. Building the HTTP API with API Gateway

• Navigate to API Gateway
• Under the “HTTP API” type, click the orange button Build
• Enter a name for your API, then click on Next
• Leave the routes empty, we will be configuring them later on. Click on Next
• For the sake of this Demo, we will only be using the $default stage which is set to auto-deploy. API Gateway stages are useful in multi-environments where we want to rely on a single API to test before deploying to production for example. - Click on Next
• Review the details of your API, and click on Create
• The $default stage has an assigned Invoke URL that will correspond to the Base URL of our API:

https://{restapi_id}.execute-api.{region}.amazonaws.com/

1. Create the DynamoDB table

• Go to the DynamoDB console
• Click on create table
• Enter a table name, my table will have “demo-db-api” as a name.
• For the partition key, enter account_id (str) that will be the fake representation of an AWS Account ID, that is 12 digits long.

1. Create the Lambda function that will be integrated with our API’s routes

In the backend, we are going to create a Lambda function that will be invoked by the API endpoints that we will create and configure later on via routes in API Gateway. This Lambda function will be responsible for the logic to put (POST method) items to our DynamoDB database. In addition to that, we should implement API fields validation at this level, but this will not be done to keep the demo simple.

• Go to the Lambda console
• Click on Create function
• Enter a function name, my function will be called demo-api-function
• Select Python 3.9 as the runtime
• For the function role, it should have the necessary permissions to interact with our DynamoDB table
• Lambda function’s code:

import json
import base64
import boto3
def lambda_handler(event, context):
    payload = event["body"]
    payload = base64.b64decode(payload)
    payload = json.loads(payload)
    operation = payload["operation"]
        if "table_name" in payload:
            dynamo_resource = boto3.resource('dynamodb')
            dynamo_table = dynamo_resource.Table(payload['table_name'])
        else:
            raise ValueError("Invalid JSON payload. Please include 'table_name' in your payload")
    operations = {
        "GET": lambda x: dynamo_table.get_item(**x),
        "POST": lambda x: dynamo_table.put_item(**x)
    }
    operation = payload["operation"]
    if operation in operations:
        return operations[operation](payload.get("payload"))
    else:
        raise ValueError(f"The operation specified {operation} is not supported. Only GET and POST operations are supported")

• By default, the body sent in POST requests will be encoded in base64 and included in the event[“body”] of the Lambda function.

The input payload schema to be included in the body of the POST requests:

{
    "table_name": "demo-db-api",
    "operation": "POST",
    "payload": {
        "Item": {
            "account_id": 111111111111,
            "key": "value"
        }
    }
}

1. Create API Gateway routes

• Go back to the API Gateway console.
• Select the API that we created in step 1. Under the Develop section in the menu on the left, select Routes.
• Click on Create.
• Create a POST route. For the path, enter /api/db-demo.
• For this route, under the route details click on select integration, then click on create and attach an integration.
• Enter Lambda function for the integration type.
• For the lambda function, select the function that we created in step 3: demo-db-api-function.
• In advanced settings, make sure that the payload format version is 2.0. Our Lambda function does not match the standard 1.0 return format.
• Click on Create

Testing our API

import requests
import json
def add_account_to_db(account_id, **kwargs):
  """Function to post to our serverless demo API. Add the key-value pair as keyword arguments"""
  base_url = "https://qnifduojpk.execute-api.us-east-1.amazonaws.com/"
  endpoint = "api/db-demo"
  headers = {
    "accept": "application/json"
  }
  payload = {
    "table_name": "demo-db-api",
    "operation": "POST",
    "payload": {
      "Item": {
        "account_id": str(account_id),
        **kwargs
      }
    }
  }
  response = requests.post(f"{base_url}{endpoint}", data=json.dumps(payload), headers=headers)
  return response

Executing

add_account_to_db(111111111111, status="live", environment="prod", account_owner="marc", cloud_provider="AWS")

An item gets created in our DB

Authorization

Our API is up and running, but did you notice that we did not include an API Key in the headers of the request? Token-based or request parameter-based (for WebSocket APIs) Lambda function Authorizers are popular authorization options. AWS provided blueprints for Lambda function authorizers that you can find here.

Amazon Cognito user pools can also be configured to control authorization to your API.

References

https://www.restapitutorial.com/httpstatuscodes.html
https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-use-lambda-authorizer.html