DEV Community

Vivesh
Vivesh

Posted on

Serverless Architecture

#serverless #cloudcomputing #devops #lambda

What is Serverless Architecture?

Serverless architecture is a cloud computing execution model where the cloud provider dynamically manages the allocation and scaling of resources. Developers focus on building and deploying application code without worrying about managing the underlying infrastructure.

Key Characteristics:

  1. Event-Driven: Functions are triggered by events, such as HTTP requests, database updates, or file uploads.
  2. Pay-as-You-Go: Costs are based on actual usage (e.g., execution time and requests).
  3. Managed Infrastructure: The cloud provider handles server provisioning, scaling, and maintenance.

Components of Serverless Architecture

  1. Serverless Functions:

    • Example: AWS Lambda, Google Cloud Functions, Azure Functions.
    • Stateless and short-lived, performing a single task.

  2. Backend-as-a-Service (BaaS):

    • Managed services for databases, authentication, and messaging.
    • Examples:
      • Databases: AWS DynamoDB, Firebase Firestore.
      • Authentication: AWS Cognito, Firebase Auth.
      • Storage: AWS S3, Google Cloud Storage.

  3. API Gateway:

    • Used to expose serverless functions as RESTful APIs.
    • Example: AWS API Gateway, Azure API Management.

  4. Event Sources:

    • Triggers that invoke serverless functions.
    • Examples: S3 file uploads, DynamoDB streams, HTTP requests.

Advantages of Serverless Architecture

  1. Reduced Operational Overhead:

    • No need to manage servers, scaling, or patching.

  2. Cost Efficiency:

    • Pay only for the resources consumed (no idle costs).

  3. Scalability:

    • Automatically scales with demand.

  4. Faster Time to Market:

    • Focus on writing code rather than managing infrastructure.

  5. High Availability:

    • Built-in fault tolerance and redundancy.

Challenges of Serverless Architecture

  1. Cold Start Latency:

    • Initial invocation may experience delays due to container initialization.

  2. Limited Execution Time:

    • Functions have a maximum execution duration (e.g., 15 minutes for AWS Lambda).

  3. Monitoring and Debugging:

    • Requires specialized tools for tracing and debugging distributed systems.

  4. Vendor Lock-In:

    • Dependence on specific cloud provider services.

  5. State Management:

    • Stateless nature requires external services (e.g., databases) for maintaining state.

Serverless Use Cases

  1. Web and Mobile Backends:

    • Handle user authentication, API processing, and storage.

  2. Data Processing:

    • Process real-time streaming data or scheduled batch jobs.

  3. IoT Applications:

    • Trigger functions based on IoT device events.

  4. Chatbots:

    • Use serverless functions to process user messages.

  5. Machine Learning:

    • Perform inference using pre-trained models in serverless functions.

Popular Serverless Platforms

  1. AWS:

    • AWS Lambda
    • AWS DynamoDB
    • AWS API Gateway
    • AWS Step Functions (orchestration)

  2. Google Cloud:

    • Google Cloud Functions
    • Firebase
    • Cloud Run

  3. Microsoft Azure:

    • Azure Functions
    • Azure Logic Apps
    • Event Grid

  4. Other Platforms:

    • OpenFaaS (open-source)
    • Netlify (serverless for static sites)

Best Practices for Serverless Architecture

  1. Optimize Cold Starts:

    • Use lightweight runtime environments (e.g., Node.js).
    • Keep function packages small.

  2. Use Idempotent Functions:

    • Ensure repeated executions produce the same result.

  3. Leverage Event-Driven Design:

    • Trigger functions using cloud events (e.g., database updates, file uploads).

  4. Monitor and Log:

    • Use tools like AWS CloudWatch, Azure Monitor, or open-source tools.

  5. Secure Applications:

    • Restrict function permissions (Principle of Least Privilege).
    • Validate and sanitize inputs.

  6. Orchestrate with Step Functions:

    • Use orchestration tools for complex workflows.

Task: Create a Complex AWS Lambda Function with Multiple Triggers

In this task, we'll create an AWS Lambda function that handles multiple triggers. For this example, the Lambda function will:

  1. Process HTTP requests via API Gateway.
  2. Respond to an S3 bucket upload event.
  3. Listen to DynamoDB stream changes and log updates.

Architecture Diagram 

                   +-------------------+
HTTP Request --->  | API Gateway       |
                   +-------------------+
                          |
                          v
                   +-------------------+
                   | AWS Lambda        |
                   | (Single Function) |
                   +-------------------+
                          |
      +-------------------+-------------------+
      |                   |                   |
      v                   v                   v
+-------------+      +-----------+      +-----------------+
| DynamoDB    | ---> | Lambda    | ---> | S3 Upload Event |
| Stream      |      | Processing|      | Processing       |
+-------------+      +-----------+      +-----------------+
Enter fullscreen mode Exit fullscreen mode

Steps to Implement

Step 1: Set Up the AWS Lambda Function

  1. Create the Lambda Function:

    • Go to the AWS Lambda Console.
    • Click Create Function > Author from Scratch.
    • Name the function ComplexTriggerHandler.
    • Choose a runtime, such as Node.js 18.x or Python 3.9.

  2. Add Permissions:

    • Attach the following IAM policies:
      • AWSLambdaBasicExecutionRole
      • AmazonDynamoDBReadOnlyAccess
      • AmazonS3ReadOnlyAccess

Step 2: Configure Multiple Triggers

  1. Trigger 1: API Gateway

    • Navigate to the API Gateway Console.
    • Create a REST API.
    • Define a resource (e.g., /process).
    • Set up a POST method and link it to your Lambda function.

  2. Trigger 2: S3 Bucket Event

    • Go to the S3 Console.
    • Choose a bucket or create a new one.
    • Configure an event notification:
      • Event Type: PUT (Object Created).
      • Destination: Lambda Function > ComplexTriggerHandler.

  3. Trigger 3: DynamoDB Stream

    • Go to the DynamoDB Console.
    • Create or select a table.
    • Enable Streams:
      • Stream View Type: New Image or New and Old Images.
    • Add the Lambda function as a stream trigger.

Step 3: Write the Lambda Function Code

Node.js Example:

exports.handler = async (event) => {
    console.log("Event Received:", JSON.stringify(event, null, 2));

    if (event.requestContext) {
        // Handle API Gateway trigger
        console.log("API Gateway Triggered");
        return {
            statusCode: 200,
            body: JSON.stringify({ message: "Hello from API Gateway!" }),
        };
    } else if (event.Records && event.Records[0].eventSource === "aws:s3") {
        // Handle S3 trigger
        const bucketName = event.Records[0].s3.bucket.name;
        const objectKey = event.Records[0].s3.object.key;
        console.log(`S3 Event: Bucket - ${bucketName}, Key - ${objectKey}`);
    } else if (event.Records && event.Records[0].eventSource === "aws:dynamodb") {
        // Handle DynamoDB stream trigger
        const dynamoRecord = event.Records[0].dynamodb.NewImage;
        console.log("DynamoDB Record:", JSON.stringify(dynamoRecord));
    } else {
        console.log("Unknown Trigger");
    }

    return { statusCode: 200, body: "Processed Successfully" };
};
Enter fullscreen mode Exit fullscreen mode

Step 4: Test the Function

  1. API Gateway Test:

    • Use a tool like Postman or curl to send a POST request: 
     curl -X POST https://<API_ENDPOINT>/process

  2. S3 Upload Test:

    • Upload a file to the S3 bucket.
    • Verify that the Lambda function logs the file name and bucket details.

  3. DynamoDB Stream Test:

    • Add a new record to the DynamoDB table.
    • Check the Lambda logs for processed record details.

Step 5: Monitor and Debug

  • Use AWS CloudWatch Logs to monitor the Lambda function's execution and troubleshoot issues.
  • Set up alarms to notify you in case of errors or high latency.

Enhancements

  1. Error Handling:

    • Add retries for transient errors.
    • Use Dead Letter Queues (DLQ) for failed invocations.

  2. Optimized Logging:

    • Use structured logging (e.g., JSON) for better analysis.

  3. Integration:

    • Integrate with AWS Step Functions for advanced workflows.
    • Add SNS or SES for email notifications on specific events.

Here's a Terraform script to automate the setup of an AWS Lambda function with multiple triggers: API Gateway, S3 bucket event, and DynamoDB stream.

Terraform Script 

# Define the AWS provider
provider "aws" {
  region = "us-east-1"  # Change this to your desired region
}

# IAM Role for Lambda Execution
resource "aws_iam_role" "lambda_role" {
  name = "lambda_execution_role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action    = "sts:AssumeRole"
        Effect    = "Allow"
        Principal = { Service = "lambda.amazonaws.com" }
      },
    ]
  })
}

# Attach policies to the IAM Role
resource "aws_iam_role_policy_attachment" "lambda_basic_policy" {
  role       = aws_iam_role.lambda_role.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}

resource "aws_iam_role_policy_attachment" "lambda_dynamodb_policy" {
  role       = aws_iam_role.lambda_role.name
  policy_arn = "arn:aws:iam::aws:policy/AmazonDynamoDBReadOnlyAccess"
}

resource "aws_iam_role_policy_attachment" "lambda_s3_policy" {
  role       = aws_iam_role.lambda_role.name
  policy_arn = "arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess"
}

# Lambda Function
resource "aws_lambda_function" "lambda_function" {
  function_name    = "ComplexTriggerHandler"
  runtime          = "nodejs18.x"
  role             = aws_iam_role.lambda_role.arn
  handler          = "index.handler"
  filename         = "${path.module}/lambda_function.zip"

  source_code_hash = filebase64sha256("${path.module}/lambda_function.zip")

  # Environment variables (optional)
  environment {
    variables = {
      LOG_LEVEL = "INFO"
    }
  }
}

# API Gateway
resource "aws_api_gateway_rest_api" "api" {
  name        = "LambdaAPI"
  description = "API Gateway for Lambda function"
}

resource "aws_api_gateway_resource" "resource" {
  rest_api_id = aws_api_gateway_rest_api.api.id
  parent_id   = aws_api_gateway_rest_api.api.root_resource_id
  path_part   = "process"
}

resource "aws_api_gateway_method" "method" {
  rest_api_id   = aws_api_gateway_rest_api.api.id
  resource_id   = aws_api_gateway_resource.resource.id
  http_method   = "POST"
  authorization = "NONE"
}

resource "aws_api_gateway_integration" "integration" {
  rest_api_id             = aws_api_gateway_rest_api.api.id
  resource_id             = aws_api_gateway_resource.resource.id
  http_method             = aws_api_gateway_method.method.http_method
  integration_http_method = "POST"
  type                    = "AWS_PROXY"
  uri                     = aws_lambda_function.lambda_function.invoke_arn
}

resource "aws_lambda_permission" "api_gateway" {
  statement_id  = "AllowAPIGatewayInvoke"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.lambda_function.function_name
  principal     = "apigateway.amazonaws.com"
  source_arn    = "${aws_api_gateway_rest_api.api.execution_arn}/*/*"
}

# S3 Bucket and Event Trigger
resource "aws_s3_bucket" "s3_bucket" {
  bucket = "lambda-trigger-bucket-${random_id.bucket_suffix.hex}"
}

resource "random_id" "bucket_suffix" {
  byte_length = 8
}

resource "aws_s3_bucket_notification" "s3_notification" {
  bucket = aws_s3_bucket.s3_bucket.id

  lambda_function {
    lambda_function_arn = aws_lambda_function.lambda_function.arn
    events              = ["s3:ObjectCreated:*"]
  }
}

resource "aws_lambda_permission" "s3_permission" {
  statement_id  = "AllowS3BucketInvoke"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.lambda_function.function_name
  principal     = "s3.amazonaws.com"
  source_arn    = aws_s3_bucket.s3_bucket.arn
}

# DynamoDB Table and Stream
resource "aws_dynamodb_table" "dynamodb_table" {
  name           = "lambda-trigger-table"
  billing_mode   = "PAY_PER_REQUEST"
  hash_key       = "id"
  stream_enabled = true
  stream_view_type = "NEW_AND_OLD_IMAGES"

  attribute {
    name = "id"
    type = "S"
  }
}

resource "aws_lambda_event_source_mapping" "dynamodb_trigger" {
  event_source_arn = aws_dynamodb_table.dynamodb_table.stream_arn
  function_name    = aws_lambda_function.lambda_function.arn
  starting_position = "LATEST"
}

# Output API Gateway URL
output "api_gateway_url" {
  value = aws_api_gateway_rest_api.api.execution_arn
}
Enter fullscreen mode Exit fullscreen mode

Steps to Use the Script

  1. Prepare the Lambda Function Code:

    • Write the Lambda code (index.js) similar to the example in the previous section.
    • Zip the code (lambda_function.zip) and place it in the same directory as the Terraform script.

  2. Initialize Terraform:

   terraform init
Enter fullscreen mode Exit fullscreen mode
  1. Plan the Deployment: 
   terraform plan
Enter fullscreen mode Exit fullscreen mode
  1. Apply the Configuration: 
   terraform apply
Enter fullscreen mode Exit fullscreen mode
  1. Test the Triggers:
    • Use the output API Gateway URL to send an HTTP request.
    • Upload a file to the S3 bucket.
    • Add or modify records in the DynamoDB table.

Clean Up

After testing, destroy the resources to avoid incurring unnecessary charges:

terraform destroy
Enter fullscreen mode Exit fullscreen mode

Happy Learning !!!

Top comments (6)

Collapse
 
xinitd profile image
Shakhzhakhan Maxudbek •

Thanks for article and happy new year!

Collapse
 
tn_decor profile image
TN_Decor •

Thanks for sharing!

Collapse
 
martygo profile image
Martins Gouveia •

Congrats. The post is a gem.

Collapse
 
extinctsion profile image
Aditya •

Nice article!

Collapse
 
574n13y profile image
Vivesh •

Thanks, & Happy Learning !!!

Collapse
 
kevin_neeham_2be4dc345439 profile image
Kevin Neeham •

I'll never forget the day I fell victim to crypto fraudsters. I had invested a significant amount of money, $267,000, in what I thought was a legitimate cryptocurrency platform, but it turned out to be a sophisticated and well planned cryptocurrency scam. I lost everything - my entire investment, gone in an instant with no knowledge of what to do or whom to turn to for help in my distress.

I was devastated, feeling like I'd been punched in the gut. I didn't know where to turn or who to trust. I felt like I'd never see my money again.

Seeking information that might help in the retrieval of my lost funds was when I discovered Coreassetinc. A trusted friend recommended them to me, saying they specialized in recovering lost funds from crypto scams. I was skeptical at first, but desperate for a solution I had to reach out to them through their contact details " Coreassetinc @ Gmail dot com or Telegram handle: @ Coreassetinc".

From the moment I contacted Coreassetinc, their promptness, reassurance, determination to help with my situation and attention to details, I knew I was in good hands. Their team was professional, empathetic, and knowledgeable. They listened to my story, asked the right questions, and quickly got to work on my case with the information's I provided.

The recovery process was complex, but Coreassetinc's experts navigated it with ease. They worked tirelessly to track down my funds, communicating with me every step of the way.

And then, the moment of truth arrived. The news that Coreassetinc had successfully recovered my lost funds, I was overjoyed, feeling like I'd been given a second chance. What really stood out during my experience with Coreassetinc was their dedication to the recovery process, the team went above and beyond, using sophisticated tracking tools and cyber forensics to gather critical information and process every available data.

I can't thank CoreAssetIn enough for their help. Their expertise, dedication, and support made all the difference. If you've lost money to crypto fraudsters, don't give up hope. Contact Coreassetinc - they might just change your life like they changed mine. They truly gave me back my financial future.

Some comments may only be visible to logged-in visitors. Sign in to view all comments.

Read next

abhay_yt_52a8e72b213be229 profile image

Introduction to Docker: A Comprehensive Guide to Containers, Images, and Modern Software Development

Abhay Singh Kathayat -

abhay_yt_52a8e72b213be229 profile image

Docker in Development and Testing Environments: Streamlining DevOps Workflows

Abhay Singh Kathayat -

kasuken profile image

Using Git Maintenance in GitHub Actions: Optimize Your Repositories Automatically

Emanuele Bartolesi -

matteoguadrini profile image

How to create own Python project in 5 minutes like a pro

GU aka Matteo Guadrini -