Secure Serverless with HashiCorp Vault and Lambda: Dynamic Database Credentials

In the era of cloud-native applications, managing secrets and database credentials remains one of the most critical security challenges. Traditional approaches of hardcoding credentials or storing them in environment variables create significant security risks. This article explores a revolutionary approach: dynamic database credentials using HashiCorp Vault in serverless architectures.

The Problem with Static Credentials

Most serverless applications today rely on static database credentials that suffer from several critical issues:

• Long-lived secrets that increase exposure risk
• Shared credentials across multiple services
• Manual rotation processes prone to human error
• Limited audit trails for credential usage
• Credential sprawl across configuration files and environment variables

Enter Dynamic Credentials

Dynamic credentials represent a paradigm shift in secrets management. Instead of storing permanent passwords, credentials are generated on-demand with automatic expiration and cleanup. This approach provides:

• Zero hardcoded credentials in application code
• Short-lived credentials (1-hour lifespan by default)
• Automatic rotation and cleanup
• Unique credentials per request
• Complete audit trail of all credential operations
• IAM-based authentication eliminating API key management

Architecture Overview

The solution consists of four main components working together:

┌─────────────┐    ┌──────────────┐    ┌─────────────┐
│   Lambda    │───▶│    Vault     │───▶│ RDS MySQL   │
│  Function   │    │   (EC2)      │    │  Database   │
└─────────────┘    └──────────────┘    └─────────────┘
      │                     │                  │
   IAM Auth          Dynamic Creds      Temp User

Component Breakdown

HashiCorp Vault Server: Deployed on EC2, manages the database secrets engine and handles credential generation.

AWS Lambda Function: Authenticates with Vault using IAM roles and retrieves dynamic credentials for database access.

RDS MySQL Database: Target database where temporary users are created and automatically cleaned up.

AWS Systems Manager: Securely stores the Vault root token for infrastructure management.

1. Vault Database Secrets Engine Configuration

The heart of the solution is Vault's database secrets engine, configured to manage MySQL credentials:

resource "vault_database_secret_backend_connection" "mysql" {
  backend       = vault_mount.database.path
  name          = "mysql"
  allowed_roles = ["lambda-role"]

  mysql {
    connection_url = "{{username}}:{{password}}@tcp(${aws_db_instance.main.address}:3306)/"
    username       = var.database_master_username
    password       = random_password.rds_master_password.result
    max_open_connections = 10
    max_idle_connections = 5
  }
}

2. Dynamic Role Definition

The database role defines how temporary users are created and what permissions they receive:

resource "vault_database_secret_backend_role" "lambda" {
  backend             = vault_mount.database.path
  name                = "lambda-role"
  db_name             = vault_database_secret_backend_connection.mysql.name
  creation_statements = [
    "CREATE USER '{{name}}'@'%' IDENTIFIED BY '{{password}}';",
    "GRANT SELECT, INSERT, UPDATE ON ${var.database_name}.* TO '{{name}}'@'%';"
  ]
  revocation_statements = [
    "DROP USER '{{name}}'@'%';"
  ]
  default_ttl = 3600  # 1 hour
  max_ttl     = 86400 # 24 hours
}

3. IAM Authentication Setup

Lambda functions authenticate with Vault using AWS IAM, eliminating the need for API keys:

resource "vault_aws_auth_backend_role" "lambda" {
  backend                   = vault_auth_backend.aws.path
  role                      = "lambda-role"
  auth_type                 = "iam"
  bound_iam_principal_arns  = [aws_iam_role.lambda_exec.arn]
  token_policies            = [vault_policy.lambda.name]
  token_ttl                 = 3600
  token_max_ttl             = 86400
}

4. Lambda Implementation

The Lambda function demonstrates the complete workflow:

def get_vault_token():
    """Authenticate with Vault using AWS IAM method"""
    session = boto3.Session()
    credentials = session.get_credentials()
    frozen_creds = credentials.get_frozen_credentials()

    # Create signed STS request for Vault authentication
    request = AWSRequest(
        method="POST",
        url="https://sts.amazonaws.com/",
        headers={"Content-Type": "application/x-www-form-urlencoded"},
        data="Action=GetCallerIdentity&Version=2011-06-15".encode('utf-8')
    )

    sigv4 = SigV4Auth(frozen_creds, "sts", "us-east-1")
    sigv4.add_auth(request)

    # Authenticate with Vault
    iam_request = {
        "role": "lambda-role",
        "iam_http_request_method": request.method,
        "iam_request_url": base64.b64encode(request.url.encode('utf-8')).decode('utf-8'),
        "iam_request_body": base64.b64encode(request.body).decode('utf-8'),
        "iam_request_headers": base64.b64encode(
            json.dumps(dict(request.headers)).encode('utf-8')
        ).decode('utf-8')
    }

    auth_response = requests.post(
        f"{vault_addr}/v1/auth/aws/login",
        json=iam_request,
        timeout=10
    )

    return auth_response.json()['auth']['client_token']

def get_database_credentials(vault_token):
    """Retrieve dynamic database credentials from Vault"""
    headers = {'X-Vault-Token': vault_token}

    creds_response = requests.get(
        f"{vault_addr}/v1/database/creds/lambda-role",
        headers=headers,
        timeout=10
    )

    return creds_response.json()['data']

Security Benefits

1. Credential Lifecycle Management

Every database credential has a defined lifecycle:

• Generation: Created on-demand when requested
• Usage: Valid for exactly 1 hour by default
• Expiration: Automatically expires without manual intervention
• Cleanup: Database user is automatically deleted upon expiration

2. Principle of Least Privilege

Each credential is granted only the minimum permissions required:

• Scoped permissions: Only SELECT, INSERT, UPDATE on specific database
• Time-bounded access: Maximum 24-hour lifetime
• Role-based access: Tied to specific Lambda execution role

3. Comprehensive Audit Trail

Vault provides complete visibility into credential operations:

• Authentication events: Who requested access and when
• Credential generation: Which credentials were created
• Usage patterns: How credentials are being utilized
• Expiration tracking: When credentials expire and are cleaned up

Operational Advantages

1. Zero-Touch Credential Management

Once deployed, the system requires no manual intervention:

• Automatic rotation: New credentials for every request
• Self-healing: Failed credentials don't affect subsequent requests
• Scalable: Handles thousands of concurrent credential requests

2. Developer Experience

Developers work with a simple, consistent API:

# Get credentials
vault_token = get_vault_token()
db_creds = get_database_credentials(vault_token)

# Use credentials
connection = pymysql.connect(
    host=rds_endpoint,
    user=db_creds['username'],
    password=db_creds['password'],
    database=database_name
)

3. Infrastructure as Code

The entire solution is defined in Terraform, enabling:

• Reproducible deployments across environments
• Version-controlled infrastructure changes
• Automated testing and validation
• Disaster recovery capabilities

Find the terraform template at: Hashicorp Vault Dynamic Credentials

then run:

Terraform init

Terraform plan

Terraform apply

Production Deployment Considerations

1. High Availability

For production environments, implement Vault clustering:

resource "aws_instance" "vault" {
  count = 3  # Multi-node cluster
  # ... configuration
}

resource "aws_lb" "vault" {
  # Load balancer for Vault cluster
}

2. Network Security

Deploy Vault in private subnets with proper network controls:

resource "aws_security_group" "vault" {
  ingress {
    from_port       = 8200
    to_port         = 8200
    protocol        = "tcp"
    security_groups = [aws_security_group.lambda.id]  # Only Lambda access
  }
}

Monitoring and Alerting

Key Metrics to Monitor

1. Credential Generation Rate: Track requests per second
2. Authentication Failures: Monitor failed Vault logins
3. Database Connection Errors: Alert on connection failures
4. Credential Expiration: Track credential lifecycle

CloudWatch Integration

import boto3

cloudwatch = boto3.client('cloudwatch')

def publish_metric(metric_name, value, unit='Count'):
    cloudwatch.put_metric_data(
        Namespace='VaultDynamicCredentials',
        MetricData=[
            {
                'MetricName': metric_name,
                'Value': value,
                'Unit': unit,
                'Timestamp': datetime.utcnow()
            }
        ]
    )

Cost Analysis

Infrastructure Costs

• Vault EC2 Instance: $8-15/month (t3.micro)
• RDS Database: $15-25/month (db.t3.micro)
• Lambda Execution: Pay-per-use, typically <$1/month
• Total: $25-40/month for development environment

Cost vs. Security Trade-off

While dynamic credentials add infrastructure costs, they provide:

• Reduced security incidents: Potential savings of thousands in breach costs
• Compliance benefits: Easier audit and regulatory compliance
• Operational efficiency: Reduced manual credential management overhead

Comparison with Alternatives

Approach	Security	Complexity	Cost	Scalability
Static Credentials	Low	Low	Low	High
AWS Secrets Manager	Medium	Medium	Medium	High
Dynamic Credentials	High	High	Medium	High
IAM Database Auth	Medium	Low	Low	Medium

Conclusion

Dynamic database credentials with HashiCorp Vault represent a significant advancement in serverless security. By eliminating static credentials and implementing just-in-time access, organizations can dramatically reduce their security risk while maintaining operational efficiency.

The implementation demonstrated here provides a production-ready foundation that can be extended and customized for specific organizational needs. As serverless architectures continue to evolve, dynamic credential management will become increasingly critical for maintaining security at scale.

Run the command below to confirm your set up works.

curl -s -H "X-Vault-Token: <vault-token>" \
  "http://<vault-server-ip>:8200/v1/database/creds/lambda-role" | jq .

Key Takeaways

1. Dynamic credentials eliminate the risks associated with static database passwords
2. IAM-based authentication removes the need for API key management
3. Automatic credential lifecycle management reduces operational overhead
4. Complete audit trails provide visibility into all credential operations
5. Infrastructure as Code enables reproducible, version-controlled deployments

The future of secrets management is dynamic, ephemeral, and automated. Organizations that adopt these practices today will be better positioned to handle the security challenges of tomorrow's cloud-native landscape.

Follow me for more demos and networking. Kevin Kiruri LinkedIn
Find the source code here: Hashicorp Vault Dynamic Credentials