DEV Community: codeisgood

Invoking an AWS Lambda Function with Terraform

codeisgood — Wed, 24 Apr 2024 04:56:35 +0000

Invoking an AWS Lambda Function with Terraform

Introduction

AWS Lambda is a serverless compute service that allows you to run code without provisioning or managing servers. Terraform, on the other hand, is an infrastructure-as-code (IaC) tool that enables you to define and provision cloud resources in a declarative manner. In this article, we'll explore how to create an AWS Lambda function using Terraform and set up automatic invocations whenever a terraform apply command is executed.

Prerequisites

Before we begin, ensure that you have the following prerequisites:

An AWS account with appropriate permissions to create Lambda functions and IAM roles.
Terraform installed on your local machine.

Terraform Code

Below is an example of Terraform code that achieves our goal. We'll create an AWS Lambda function, an IAM role for the function, and configure a CloudWatch Events rule to trigger the Lambda function on every terraform apply.

provider "aws" {
  region = "us-west-2"  # Replace with your desired AWS region
}

# Create an AWS Lambda function
resource "aws_lambda_function" "my_lambda_function" {
  function_name = "my-lambda-function"
  role          = aws_iam_role.my_lambda_role.arn
  handler       = "index.handler"
  runtime       = "python3.8"
  filename      = "lambda_function.zip"  # Replace with the actual path to your Lambda code ZIP file
}

# Create an IAM role for the Lambda function
resource "aws_iam_role" "my_lambda_role" {
  name = "my-lambda-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17",
    Statement = [
      {
        Action = "sts:AssumeRole",
        Effect = "Allow",
        Principal = {
          Service = "lambda.amazonaws.com"
        }
      }
    ]
  })
}

# Attach a policy to the Lambda role (e.g., permissions to log to CloudWatch)
resource "aws_iam_policy_attachment" "my_lambda_policy_attachment" {
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"  # Replace with your desired policy ARN
  roles      = [aws_iam_role.my_lambda_role.name]
}

# Create a CloudWatch Events rule to trigger the Lambda function on every `terraform apply`
resource "aws_cloudwatch_event_rule" "my_lambda_trigger" {
  name        = "my-lambda-trigger"
  description = "Trigger Lambda on Terraform apply"
  event_pattern = jsonencode({
    source      = ["aws.terraform"],
    detail_type = ["Terraform Apply"]
  })
}

# Create a CloudWatch Events target to invoke the Lambda function
resource "aws_cloudwatch_event_target" "my_lambda_target" {
  rule      = aws_cloudwatch_event_rule.my_lambda_trigger.name
  arn       = aws_lambda_function.my_lambda_function.arn
  target_id = "my-lambda-target"
}

Conclusion

By following the steps above, you can create an AWS Lambda function using Terraform and ensure that it is automatically invoked whenever you apply changes to your infrastructure. Remember to replace the placeholders with your actual values, and happy coding!

DynamoDB Scan with sort

codeisgood — Fri, 12 Jan 2024 02:16:01 +0000

To query and retrieve 100 records from DynamoDB and order them by a sort key column in Node.js, you can use the AWS SDK for JavaScript. Make sure you have the AWS SDK installed and configured. Here's an example code snippet:

const AWS = require('aws-sdk');

AWS.config.update({
  region: 'your-dynamodb-region', // Replace with your DynamoDB region
  accessKeyId: 'your-access-key-id', // Replace with your AWS access key ID
  secretAccessKey: 'your-secret-access-key', // Replace with your AWS secret access key
});

const dynamoDB = new AWS.DynamoDB.DocumentClient();
const tableName = 'your-dynamodb-table-name'; // Replace with your DynamoDB table name
const sortKeyColumn = 'your-sort-key-column'; // Replace with your sort key column name

const params = {
  TableName: tableName,
  Limit: 100,
  ScanIndexForward: true, // Set to false for descending order
  KeyConditionExpression: '#sk > :sk', // Replace > with < for descending order
  ExpressionAttributeNames: {
    '#sk': sortKeyColumn,
  },
  ExpressionAttributeValues: {
    ':sk': '0', // Replace with your start key value
  },
};

dynamoDB.query(params, (err, data) => {
  if (err) {
    console.error('Error querying DynamoDB:', err);
  } else {
    console.log('Queried records:', data.Items);
  }
});

Leveraging AWS Athena in Lambda Functions for Serverless Data Analysis

codeisgood — Sun, 01 Oct 2023 20:49:25 +0000

Introduction:

As a software professional working with AWS services, you're likely familiar with the benefits of serverless computing. AWS Lambda allows you to run code without provisioning or managing servers, making it an ideal choice for various tasks, including data processing and analysis. When combined with AWS Athena, a serverless query service, you can unlock the power of on-demand, SQL-based analytics on your data stored in Amazon S3. In this write-up, we'll explore how to use AWS Athena within Lambda functions for efficient and cost-effective data analysis.

Why AWS Athena and Lambda?

Serverless Architecture: Lambda and Athena both follow a serverless architecture, meaning you don't need to worry about infrastructure management. This enables you to focus solely on your code and analysis.

Scalability: Lambda scales automatically to handle any volume of incoming events or data. Athena is designed for parallel processing, ensuring speedy query execution even on large datasets.

Cost-Efficiency: You pay only for the compute resources used during query execution in Athena and the actual runtime of your Lambda function. There are no upfront costs or idle resources.

Steps to Implement Athena in Lambda:

Data Preparation: Before querying data in Athena, ensure your datasets are properly structured in Amazon S3. Athena supports various file formats, including Parquet, ORC, and JSON.

IAM Roles: Create an IAM role that allows your Lambda function to access both Athena and the S3 buckets where your data resides. This role should have appropriate permissions to run queries.

Lambda Function: Write a Lambda function in your preferred programming language (e.g., Java since you work with Java technologies). This function should use the AWS SDK to interact with Athena.

Query Execution: Inside your Lambda function, you can use the Athena SDK to run SQL queries against your S3 data. You can pass parameters and retrieve query results programmatically.

Response Handling: Process and analyze the results within your Lambda function. You can transform the data, perform calculations, and generate insights as needed.

Error Handling: Implement robust error handling to ensure your Lambda function gracefully handles issues like query failures or network problems.

Use Cases:

Real-time Data Analysis: Trigger Lambda functions in response to events, such as new data arriving in S3 buckets. Use Athena to perform real-time analysis on this data.

Historical Data Exploration: Given your interest in history and fact-finding, you can use Athena to query historical data sets stored in S3 and generate insights or reports.

Automated Reporting: Schedule Lambda functions to run Athena queries at specific intervals and generate automated reports or dashboards.

Conclusion:

Combining AWS Athena and Lambda provides a serverless, scalable, and cost-effective solution for data analysis. As a software professional leading a technical team, you can leverage this integration to drive data-driven decisions, whether it's for real-time analytics, historical exploration, or automated reporting. By harnessing the power of these AWS services, you can streamline your data analysis workflows and focus on extracting valuable insights from your data.

const AWS = require('aws-sdk');

async function submitQuery(query) {
  const client = new AWS.Athena();

  const params = {
    QueryString: query,
  };

  const response = await client.startQueryExecution(params).promise();

  return response.QueryExecutionId;
}

async function checkQueryStatus(queryId) {
  const client = new AWS.Athena();

  const params = {
    QueryExecutionId: queryId,
  };

  const response = await client.getQueryExecution(params).promise();

  return response.QueryExecution.Status.State;
}

async function fetchQueryResult(queryId) {
  const client = new AWS.Athena();

  const params = {
    QueryExecutionId: queryId,
  };

  const response = await client.getQueryResults(params).promise();

  return response.ResultSet.Rows;
}

async function main(event, context) {
  const query = event.query;

  // Submit the query
  const queryId = await submitQuery(query);

  // Check the status of the query until it is completed
  let status = 'pending';
  while (status !== 'completed') {
    await new Promise((resolve) => setTimeout(resolve, 1000));
    status = await checkQueryStatus(queryId);
  }

  // Fetch the result of the query
  const result = await fetchQueryResult(queryId);

  // Return the result to the caller
  return result;
}

exports.handler = main;

Create API Gateway with Step Function Integration

codeisgood — Tue, 21 Mar 2023 01:56:31 +0000

Often we need to integrate Lambda with API Gateway which is a very popular pattern, however as we think about asynchronous processing we need to integrate Step Functions (which will have a chain of actions) with API gateway.

Below is a simple terraform example of how to create an API Gateway with Step Function Integration

provider "aws" {
  region = "us-east-1"
}

resource "aws_api_gateway_rest_api" "example_api" {
  name = "example_api"
}

resource "aws_api_gateway_resource" "example_resource" {
  rest_api_id = aws_api_gateway_rest_api.example_api.id
  parent_id   = aws_api_gateway_rest_api.example_api.root_resource_id
  path_part   = "example"
}

resource "aws_api_gateway_method" "example_method" {
  rest_api_id   = aws_api_gateway_rest_api.example_api.id
  resource_id   = aws_api_gateway_resource.example_resource.id
  http_method   = "POST"
  authorization = "NONE"
}

resource "aws_api_gateway_integration" "example_integration_step_function" {
  rest_api_id = aws_api_gateway_rest_api.example_api.id
  resource_id = aws_api_gateway_resource.example_resource.id
  http_method = aws_api_gateway_method.example_method.http_method
  integration_http_method = "POST"
  type = "AWS"

  uri = aws_sfn_state_machine.example_step_function.execution_arn
}

resource "aws_sfn_state_machine" "example_step_function" {
  name = "example_step_function"
  definition = <<EOF
{
  "Comment": "A simple AWS Step Functions state machine",
  "StartAt": "FirstState",
  "States": {
    "FirstState": {
      "Type": "Pass",
      "Result": "Hello, world!",
      "End": true
    }
  }
}
EOF
}

resource "aws_api_gateway_deployment" "example_deployment" {
  depends_on = [
    aws_api_gateway_integration.example_integration_step_function
  ]

  rest_api_id = aws_api_gateway_rest_api.example_api.id
}

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "states:StartExecution"
            ],
            "Resource": [
                "arn:aws:states:<REGION>:<ACCOUNT_ID>:stateMachine:<STATE_MACHINE_NAME>"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": [
                "arn:aws:logs:<REGION>:<ACCOUNT_ID>:log-group:/aws/lambda/<LAMBDA_FUNCTION_NAME>:*"
            ]
        }
    ]
}