DEV Community

harsh patel
harsh patel

Posted on

Beyond the Proxy: Building a Secure, Observable Serverless API with Amazon API Gateway

#aws #serverless #apigateway #cloud

Introduction: Rethinking API Gateway

If you ask many developers what Amazon API Gateway is, they’ll often say,

“It’s a way to trigger Lambda functions with HTTP requests.”

While that’s a common use case, this view undersells the service dramatically.

In reality, API Gateway is a complete API management platform. It’s your first and most critical line of defense, capable of handling validation, transformation, authentication, throttling, and observability before your Lambda function even runs.

It shifts the burden of common API concerns from your code to a managed, scalable layer.

To prove this, we’ll build TrailLog — a serverless API for logging hiking trips.

We won’t just connect endpoints; we’ll use API Gateway’s full potential to create a robust, secure, and observable API.

By the end of this guide, you will learn how to:

  • Validate & Transform: Enforce data contracts at the gateway, keeping Lambda code clean.
  • Secure Endpoints: Implement a custom Lambda authorizer for token-based access control.
  • Achieve Observability: Gain deep insights into API performance with structured CloudWatch logs.
  • Orchestrate a Governed API: Deploy a production-ready API with full visibility and control.

The Blueprint: TrailLog System Architecture

Our fully serverless architecture is designed for scalability and simplicity.

Each AWS service plays a clear role in this ecosystem.

Component Responsibility
Amazon API Gateway (REST API) The unified entry point — handles routing, validation, auth, and logging
AWS Lambda Executes backend logic for creating and retrieving trips
Amazon DynamoDB Stores all hiking data in a serverless NoSQL table
Amazon CloudWatch Centralizes logs and metrics for monitoring and debugging

This stack scales automatically with traffic, without managing or patching servers.

Architecture Diagram


Client application sends incoming HTTPS REST requests to Amazon API Gateway, which triggers AWS Lambda functions for GET, POST, and DELETE operations. Lambda interacts with Amazon DynamoDB for data storage and sends logs to Amazon CloudWatch.

Foundation: Setting Up Permissions with IAM

All Lambda functions in TrailLog share a single IAM role named lambda-traillog-role,

configured via an inline policy that grants access to DynamoDB and CloudWatch logs.

IAM Role: lambda-traillog-role

All Lambda functions in TrailLog share a single IAM role named lambda-traillog-role.

This role is granted access to the DynamoDB table TrailLog through the following inline policy.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DDBWriteTrailLog",
      "Effect": "Allow",
      "Action": [
        "dynamodb:PutItem"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:579273601939:table/TrailLog"
    },
    {
      "Sid": "DDBReadTrailLog",
      "Effect": "Allow",
      "Action": [
        "dynamodb:GetItem",
        "dynamodb:Scan",
        "dynamodb:Query"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:579273601939:table/TrailLog"
    },
    {
      "Sid": "DDBDescribeForChecks",
      "Effect": "Allow",
      "Action": [
        "dynamodb:DescribeTable"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:579273601939:table/TrailLog"
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

This role is attached to all Lambda functions in the project.

Deep Dive 1: The POST /trips Endpoint — Validation & Transformation

The first endpoint demonstrates API Gateway’s ability to offload validation and data transformation

before invoking the backend logic.

DynamoDB Table: TrailLog

Property Value
Partition Key tripId (String)
Billing Mode Pay-per-request

Lambda Function: create-trip (Python 3.12)

This Lambda function remains simple because API Gateway enforces all validation upfront.

import json, os, uuid, boto3
from datetime import datetime
from decimal import Decimal, InvalidOperation

dynamodb = boto3.resource("dynamodb")
table = dynamodb.Table(os.environ["TABLE_NAME"])

def lambda_handler(event, context):
    # API Gateway has already validated input; focus on core logic.
    trip_id = str(uuid.uuid4())
    item = {
        "tripId": trip_id,
        "userId": event["userId"],
        "trailName": event["trailName"],
        "date": event["date"],
        "distanceKm": event["distanceKm"],
        "durationMin": event["durationMin"],
        "elevationGainM": event["elevationGainM"],
        "notes": event.get("notes", ""),
        "createdAt": datetime.utcnow().isoformat() + "Z"
    }
    table.put_item(Item=item)
    return {
        "statusCode": 201,
        "headers": {"Content-Type": "application/json"},
        "body": json.dumps({"ok": True, "tripId": trip_id})
    }
Enter fullscreen mode Exit fullscreen mode

The API Gateway Configuration: Non-Proxy Integration

Disabling Lambda Proxy Integration gives us control over request handling and validation.

Setting Value
Integration Type Lambda Function
Proxy Integration Disabled
Validator Body & Params
Model TripCreate

Request Model: TripCreate 

{
  "type": "object",
  "required": ["userId", "trailName", "date", "distanceKm", "durationMin", "elevationGainM"],
  "properties": {
    "userId": {"type": "string"},
    "trailName": {"type": "string"},
    "date": {"type": "string", "pattern": "^[0-9]{4}-[0-9]{2}-[0-9]{2}$"},
    "distanceKm": {"type": "number"},
    "durationMin": {"type": "integer"},
    "elevationGainM": {"type": "integer"},
    "notes": {"type": "string"}
  }
}
Enter fullscreen mode Exit fullscreen mode

Now, any malformed input (for example, an invalid date format or incorrect type)

is rejected at the gateway, saving Lambda invocations and improving reliability.


Successfully created a new hiking trip via the POST /trips endpoint, returning a 201 status and generated tripId.

Deep Dive 2: The GET /trips/{tripId} Endpoint — Simple Proxy Integration

For simple reads, we use Lambda Proxy Integration to pass the entire request context directly to Lambda.

Lambda Function: get-trip

This function extracts tripId from the path and fetches the corresponding item from DynamoDB.

import json, os, boto3, decimal

dynamodb = boto3.resource("dynamodb")
table = dynamodb.Table(os.environ["TABLE_NAME"])

def _to_native(x):
    if isinstance(x, decimal.Decimal):
        return int(x) if x % 1 == 0 else float(x)
    return x

def lambda_handler(event, context):
    trip_id = (event.get("pathParameters") or {}).get("tripId")
    if not trip_id:
        return {"statusCode": 400, "body": "Missing tripId"}

    res = table.get_item(Key={"tripId": trip_id})
    item = res.get("Item")
    if not item:
        return {"statusCode": 404, "body": "Not found"}

    return {
        "statusCode": 200,
        "headers": {"Content-Type": "application/json"},
        "body": json.dumps(_to_native(item))
    }
Enter fullscreen mode Exit fullscreen mode

Test Request 

curl https://<api-id>.execute-api.us-east-1.amazonaws.com/dev/trips/<tripId>
Enter fullscreen mode Exit fullscreen mode


Fetched detailed hiking trip data successfully using the GET /trips/{tripId} endpoint.

Deep Dive 3: Security and Observability — /secure/admin/stats

This administrative endpoint combines two advanced API Gateway features:

custom token authorization and centralized CloudWatch observability.

1. Securing the Route with a Lambda Authorizer

We use a custom Lambda authorizer to protect the /secure/admin/stats route.

Authorizer Function: lambda-authorizer 

import json

def lambda_handler(event, context):
    DEMO_TOKEN = "Bearer trail123"
    token = event.get("authorizationToken", "").strip()

    if token == DEMO_TOKEN:
        effect, role = "Allow", "admin"
    else:
        effect, role = "Deny", "guest"

    return {
        "principalId": "trail-admin",
        "policyDocument": {
            "Version": "2012-10-17",
            "Statement": [{
                "Action": "execute-api:Invoke",
                "Effect": effect,
                "Resource": event["methodArn"]
            }]
        },
        "context": {"role": role}
    }
Enter fullscreen mode Exit fullscreen mode

Any request without X-Auth-Token: Bearer trail123 will receive a 403 Forbidden response.

2. Stats Aggregation Function: admin-stats

This function aggregates trip statistics across all records.

It’s invoked only if the authorizer grants access.

import json, os, boto3, decimal

dynamodb = boto3.resource("dynamodb")
table = dynamodb.Table(os.environ["TABLE_NAME"])

def _n(x):
    if isinstance(x, decimal.Decimal):
        return int(x) if x % 1 == 0 else float(x)
    return x

def lambda_handler(event, context):
    scan = table.scan(ProjectionExpression="tripId, distanceKm, durationMin, elevationGainM")
    items = scan.get("Items", [])
    total = len(items)
    km = sum(_n(i.get("distanceKm", 0)) for i in items)
    mins = sum(_n(i.get("durationMin", 0)) for i in items)
    gain = sum(_n(i.get("elevationGainM", 0)) for i in items)

    stats = {
        "totalTrips": total,
        "totalKm": round(float(km), 2),
        "totalHours": round(float(mins) / 60.0, 1),
        "totalElevationM": int(gain)
    }

    return {
        "statusCode": 200,
        "headers": {"Content-Type": "application/json"},
        "body": json.dumps(stats)
    }
Enter fullscreen mode Exit fullscreen mode

Test Request 

curl -H "Authorization: Bearer trail123" "$BASE/secure/admin/stats"
Enter fullscreen mode Exit fullscreen mode

Access denied without a valid token — API Gateway blocking unauthorized requests using the custom Lambda authorizer.


Authorized admin access to /secure/admin/stats returning aggregated trail statistics.

The Crown Jewel: Deep Observability with CloudWatch

To gain complete visibility into API behavior, enable both Execution Logs and Access Logs in Amazon API Gateway.

These logs provide the foundation for debugging, performance monitoring, and request analytics.

Create a Log Group for Access Logs

Before enabling access logging, create a dedicated CloudWatch Log Group where API Gateway will push structured log entries.

This ensures all API requests are tracked consistently and stored centrally.

Steps:

  1. Open CloudWatch Console → Log groups → Create log group.
  2. Enter the name: 
   /apigw/traillog/dev
Enter fullscreen mode Exit fullscreen mode
  1. (Optional) Set the retention period to 7, 30, or 90 days depending on your monitoring needs.
  2. Note down the log group ARN — you’ll use it in your API Gateway stage configuration: 
   arn:aws:logs:us-east-1:579273601939:log-group:/apigw/traillog/dev
Enter fullscreen mode Exit fullscreen mode
  1. Ensure API Gateway has permission to write logs by attaching this managed policy to its execution role: 
   arn:aws:iam::aws:policy/service-role/AmazonAPIGatewayPushToCloudWatchLogs
Enter fullscreen mode Exit fullscreen mode



CloudWatch log group /apigw/traillog/dev created to store structured access logs from API Gateway.

1. Execution Logs

Execution logs trace the internal request flow through API Gateway —

from input validation and mapping templates to Lambda invocation and response transformation.

They’re your primary tool for debugging and tracing how API Gateway processes a request.

Property Description
Log Group API-Gateway-Execution-Logs_{restApiId}/dev
Purpose Traces internal request flow and mapping templates
Best For Debugging authorization, validation, or mapping issues

Ensure your API stage has CloudWatch execution logging enabled and that API Gateway can push logs.

Attach the following managed policy to grant permissions:

arn:aws:iam::aws:policy/service-role/AmazonAPIGatewayPushToCloudWatchLogs
Enter fullscreen mode Exit fullscreen mode

2. Access Logs

Access logs summarize each API request and response in a structured JSON format.

These are invaluable for analytics, performance tracking, and auditing user behavior.

Property Description
Log Group ARN arn:aws:logs:us-east-1:579273601939:log-group:/apigw/traillog/dev
Purpose Capture per-request analytics and response metadata
Best For Monitoring latency, traffic, and client behavior

Access Log Format Template

To enable structured access logs, open your API Gateway stage and set the log format using the following JSON template.

{
  "requestId": "$context.requestId",
  "ip": "$context.identity.sourceIp",
  "time": "$context.requestTime",
  "method": "$context.httpMethod",
  "path": "$context.resourcePath",
  "status": $context.status,
  "integrationStatus": $context.integration.status,
  "userAgent": "$context.identity.userAgent",
  "authorizer": "$context.authorizer.role",
  "error": "$context.error.message"
}
Enter fullscreen mode Exit fullscreen mode

Once redeployed, every request will produce:

  • Execution Logs: Detailed traces for debugging.
  • Access Logs: High-level JSON entries for analytics.


API Gateway Execution Logs showing Lambda invocation flow, request mapping, and response transformation details.


Structured JSON access logs capturing requestId, method, path, status, and authorizer context for analytics.

Conclusion: You’ve Built More Than an API

TrailLog isn’t just a CRUD backend. It’s a governed, secure, and observable API platform built with managed AWS services.

Key Takeaways

  • Pre-Lambda Validation: Offload schema checks to API Gateway for cleaner, cheaper code.
  • Custom Authorization: Implement fine-grained security without burdening business logic.
  • Structured Logging: Use CloudWatch for comprehensive visibility and debugging.

By embracing these patterns, you can use Amazon API Gateway not just as a trigger mechanism, but as a complete API management engine — the foundation of scalable, production-ready serverless systems.

Top comments (0)