DEV Community: Benson Macharia

Master Secure File Uploads to AWS S3 in Node.js with Express and Multer

Benson Macharia — Wed, 08 Jan 2025 20:31:28 +0000

Uploading files securely is a key part of protecting user data in any web application. In this blog, we'll walk you through creating a secure file upload system using Node.js, Express, Multer, and AWS S3. We'll focus on best practices to ensure your uploads are safe and your application stays secure. This guide is perfect for beginners who want to learn how to handle file uploads while keeping security a top priority.

Now, let's dive in

Prerequisites

Basic knowledge of Node.js/Javascript
Basic knowledge of AWS and CloudFormation
AWS Account. Create one here for free
Patience

Step 1: Environment Set Up

1. Install Node.js and npm:

Download and install Node.js from nodejs.org.
Verify the installation:

$ node -v
#v20.18.1
$ npm -v
#10.8.2

2. Install AWS CLI:

Download and install the AWS CLI from aws.amazon.com/cli.
Configure the AWS CLI with your credentials:

$ aws configure
#AWS Access Key ID [****************H53C]: 
#AWS Secret Access Key [****************1CcY]: 
#Default region name [us-east-1]: 
#Default output format [json]:

Step 2: Create the Node.js Application

1. Initialize a new Node.js project:

$ mkdir node-file-upload-api
$ cd node-file-upload-api
$ npm init -y

2. Install necessary packages:

$ npm install express aws-sdk multer dotenv uuid

3. Create the application file (`app.js`):

Create a app.js file
Add the following javascript code
Here we are creating a function to send a POST request to upload our file and generate a presignedURL for the uploaded file.

// app.js
// Import the required packages
const express = require('express');
const multer = require('multer');
const AWS = require('aws-sdk');
const { v4: uuidv4 } = require('uuid');

// Load environment variables from a .env file
require('dotenv').config();

// Create an Express application
const app = express();

// Create an S3 instance with the specified region
const s3 = new AWS.S3({ region: process.env.AWS_REGION });

// Configure Multer to save uploaded files to the 'uploads/' directory
const upload = multer({ dest: 'uploads/' });

app.post('/upload', upload.single('file'), (req, res) => {
    // Get the uploaded file from the request
    const file = req.file;
    const fileKey = `${uuidv4()}-${file.originalname}`;

    const s3Params = {
        Bucket: process.env.S3_BUCKET,
        Key: fileKey,
        Body: require('fs').createReadStream(file.path), // Create a readable stream from the uploaded file
        ContentType: file.mimetype
    };

    s3.upload(s3Params, (err, data) => {
        if (err) {
            return res.status(500).send(err);
        }

        // Generate presigned URL
        const presignedUrlParams = {
            Bucket: process.env.S3_BUCKET,
            Key: fileKey,
            Expires: 60 * 60 // URL expiration time in seconds (e.g., 1 hour)
        };

        s3.getSignedUrl('getObject', presignedUrlParams, (err, presignedUrl) => {
            if (err) {
                return res.status(500).send(err);
            }

            res.status(200).send({
                message: 'File uploaded successfully',
                url: data.Location,
                presignedUrl: presignedUrl
            });
        });
    });
});

// Start the server on port 3000
app.listen(3000, () => {
    console.log('Server running on port 3000');
});

4. Create a `.env` file for environment variables:

Create a .env file
Add the following information

AWS_REGION=us-east-1
S3_BUCKET=my-node-app-bucket-382c-c803-a96a-49f1

Make sure that your bucket name is unique. You can use the Online UUID Generator to generate a unique UUID to append to the name.

Step 3: Create the CloudFormation Template

1. Create a file named `s3-file-upload.yml` and add the following content

Create a s3-file-upload.yml file
Add the following code

AWSTemplateFormatVersion: "2010-09-09"

Resources:
  S3Bucket:
    Type: "AWS::S3::Bucket" # Defines an S3 bucket resource
    Properties:
      BucketName: !Sub "my-node-app-bucket-382c-c803-a96a-49f1" # Sets the bucket name
      PublicAccessBlockConfiguration: # Configures public access settings for the bucket
        BlockPublicAcls: true # Blocks public ACLs
        BlockPublicPolicy: false # Allows custom bucket policies
        IgnorePublicAcls: true # Ignores public ACLs
        RestrictPublicBuckets: true # Restricts public bucket access
      VersioningConfiguration:
        Status: Enabled # Enables versioning for the bucket

  S3BucketPolicy:
    Type: "AWS::S3::BucketPolicy" # Defines a bucket policy resource
    Properties:
      Bucket: !Ref S3Bucket # References the S3 bucket created above
      PolicyDocument:
        Version: "2012-10-17" # Specifies the version of the policy language
        Statement:
          - Effect: Allow # Allows the specified actions
            Principal: "*" # Applies to all principals (users)
            Action: "s3:GetObject" # Allows the GetObject action
            Resource: !Sub "${S3Bucket.Arn}/*" # Applies to all objects in the bucket
            Condition:
              Bool:
                aws:SecureTransport: "true" # Requires requests to use HTTPS
          - Effect: Allow # Allows the specified actions
            Principal:
              AWS: !Sub "arn:aws:iam::${AWS::AccountId}:role/MyAppRole" # Allows the IAM role to access the bucket
            Action: "s3:PutObject" # Allows the PutObject action
            Resource: !Sub "${S3Bucket.Arn}/*" # Applies to all objects in the bucket

  MyAppIAMRole:
    Type: AWS::IAM::Role # Defines an IAM Role resource
    Properties:
      RoleName: MyAppRole # Assigns a name to the IAM role
      AssumeRolePolicyDocument:
        Version: "2012-10-17" # Specifies the version of the policy language
        Statement:
          - Effect: Allow # Allows the following action
            Principal:
              Service: "ec2.amazonaws.com" # Specifies that the EC2 service can assume this role
            Action: "sts:AssumeRole" # Allows EC2 instances to assume the role
      Policies:
        - PolicyName: S3AccessPolicy # Names the inline policy for the role
          PolicyDocument:
            Version: "2012-10-17" # Specifies the version of the policy language
            Statement:
              - Effect: Allow # Allows the following actions
                Action: "s3:PutObject" # Grants permission to upload objects to S3
                Resource: !Sub "${S3Bucket.Arn}/*" # Applies to all objects in the specified S3 bucket

In the above CloudFormation template, we are creating an S3 bucket with a name that includes the AWS account ID to make it unique. It blocks all public access and enables versioning. The template also adds a policy allowing anyone to download objects from the bucket, but only over HTTPS for secure data transmission.

Step 4: Deploy AWS Resources with CloudFormation

1. Deploy the stack:

We will now deploy our AWS CloudFormation stack using the template file we created above.

$ aws cloudformation deploy --template-file s3-file-upload.yml --stack-name S3FileUploadStack --capabilities CAPABILITY_NAMED_IAM
#Waiting for changeset to be created..
#Waiting for stack create/update to complete
#Successfully created/updated stack - S3FileUploadStack

Here's a breakdown of each part:

aws cloudformation deploy: This is the AWS CLI command to deploy a CloudFormation stack.
--template-file s3-file-upload.yml: Specifies the path to the CloudFormation template file (s3-file-upload.yml) that defines the resources and configurations.
--stack-name S3FileUploadStack: Sets the name of the CloudFormation stack to S3FileUploadStack.
--capabilities CAPABILITY_NAMED_IAM: Acknowledges that the stack requires IAM resources with custom names, allowing the deployment to create or modify IAM roles and policies.

2. Confirm S3 bucket creation:

After deployment, login into your AWS console and confirm that the S3 bucket has been provisioned from the CloudFormation stack.

Step 5: Run Your Node.js Application

Confirm you have set the environment variables in the .env file with the correct S3 bucket name.
Start the application:

$ node app.js
#Server running on port 3000

Step 6: Upload a File

1. Use Postman or cURL to send a POST request to `http://localhost:3000/upload` with a file.

Using Postman

Create a new POST request.
Set the URL to http://localhost:3000/upload.
In the "Body" tab, choose "form-data", add a key file, and select a file to upload

Using Curl

$ curl -F "file=@/tmp/files/test-upload.png" http://localhost:3000/upload

2. Receive the file URL in the response, indicating successful upload. The response will be in the following format.

{
    "message": "File uploaded successfully",
    "url": "https://my-node-app-bucket-382c-c803-a96a-49f1.s3.amazonaws.com/87587526-e70b-451d-a405-ceab366016a8-test-upload.png",
    "presignedUrl": "https://my-node-app-bucket-382c-c803-a96a-49f1.s3.amazonaws.com/87587526-e70b-451d-a405-ceab366016a8-test-upload.png?AWSAccessKeyId=AKIATA6LX4GWXV6MH53C&Expires=1736370100&Signature=66dKUQRW%2F47La3MPPngITEhFE%2FA%3D"
}

Conclusion

In conclusion, mastering secure file uploads to AWS S3 in Node.js with Express and Multer involves more than just basic functionality—it requires a strong focus on security. By integrating Multer for efficient file handling, leveraging AWS S3's robust security features like bucket policies and presigned URLs, and enforcing HTTPS connections, we ensure that uploaded files are securely managed and accessed. This approach not only protects sensitive data but also helps prevent unauthorized access and potential vulnerabilities, making it a reliable and secure solution for modern web applications.

Mastering Application Security: The Power of Rate Limiting

Benson Macharia — Fri, 15 Mar 2024 09:04:54 +0000

What is Rate Limiting?
Rate limiting is a technique used in web and API development to control the rate of traffic sent or received by a server.

Why Rate Limiting?
Rate limiting is a crucial aspect of modern application security, offering a powerful defense against various types of attacks such as brute force, DDoS, and API abuse. By restricting the number of requests a user or IP address can make within a certain timeframe, rate limiting helps prevent server overload, maintains application performance, and enhances overall security posture.

Implementation
In this article, we'll delve into the concept of rate limiting in Laravel; a popular PHP framework. We will explore how to set it up, customize it to suit your application's needs, and handle common scenarios. By the end, you'll have the knowledge and confidence to implement rate limiting in your Laravel applications, enhancing their security and stability.

Design

In Laravel, requests are routed through the router service, which directs them to the appropriate middleware for processing, then to controllers for handling business logic, models for interacting with the database, and finally to views for rendering the response back to the client.

Let's Set Up
You can download the complete source code here or you can follow this Laravel guide to scaffold your application as follows.



$ composer create-project laravel/laravel:^10.0 laravel-rate-limiting
$ cd laravel-rate-limiting
$ php artisan serve

Remember to edit your .env file and then run



$ php artisan migrate

We will additionally create:-

Three controllers i.e. LoginRegisterController.php, APIController.php and ProductController.php under app/Http/Controllers
Two Models i.e. User.php and Product.php under app/Models.
Auth and Product views under resources/views
Migrations under database/migrations

Rate Limiting Implementation
To implement rate limiting, we will make use of the rate limiting capability provided with Laravel Middleware to throttle incoming HTTP requests before they reach our controllers.
To do that, we will modify the App\Providers\RouteServiceProvider.php to define the rate limiter configurations to suit our application needs.



<?php
namespace App\Providers;
class RouteServiceProvider extends ServiceProvider
{
public function boot(): void
    {
        // Modify rate limiting for APIs to 60 requests per minute
        RateLimiter::for('api', function (Request $request) {
            return Limit::perMinute(60)->by($request->user()?->id ?: $request->ip());
        });

        // Rate limit auth endpoints to 10 requests per minute by user IP
        RateLimiter::for('auth', function (Request $request) {
            return Limit::perMinute(10)->by($request->ip());
        });

        // Rate limit products endpoints to 30 requests per minute by user ID
        RateLimiter::for('product', function (Request $request) {
            return Limit::perMinute(30)->by($request->user()?->id);
        });

        $this->routes(function () {
            Route::middleware('api')
                ->prefix('api')
                ->group(base_path('routes/api.php'));

            Route::middleware('web')
                ->group(base_path('routes/web.php'));
        });
    }
}

In the above example, we have defined two rate limiters, one called auth that limits access to the auth routes to 10 requests per minute by user IP and another one called product that limits access to products endpoints to 30 requests per minute by user ID. We have also modified the rate limiter for APIs to 60 requests per minute by user IP or ID.
We can now apply the rate limiters to the routes we want to throttle using the middleware as follows:

routes/api.php



// Endpoint requires authentication and is throttled through the auth rate limiter
Route::middleware(['auth:api','throttle:auth'])->get('/profile', function (Request $request) {
    return $request->user();
});

// Endpoints do not require authentication but they are throttled through the auth rate limiter
Route::middleware(['throttle:auth'])->post('/user/register', [APIController::class, 'register']);
Route::middleware(['throttle:auth'])->post('/user/login', [APIController::class, 'login']);

// Endpoints require authentication and are throttled through the auth rate limiter
Route::middleware(['auth:api','throttle:auth'])->get('/users', [APIController::class, 'users']);
Route::middleware(['auth:api','throttle:auth'])->get('/user/{id}', [APIController::class, 'user']);

// Endpoints require authentication and are throttled through the auth product limiter
Route::middleware(['auth:api','throttle:product'])->post('/product', [APIController::class, 'newProduct']);
Route::middleware(['auth:api','throttle:product'])->get('/products', [APIController::class, 'index']);
Route::middleware(['auth:api','throttle:product'])->get('/product/{id}', [APIController::class, 'product']);

routes/web.php



// Not throttled
Route::get('/', function () {
    return view('welcome');
});

// Throttled through the auth rate limiter
Route::group(['middleware' => 'throttle:auth'], function () {
    Route::get('/register', [LoginRegisterController::class, 'register'])->name('register');
    Route::post('/store', [LoginRegisterController::class, 'store'])->name('store');
    Route::get('/login', [LoginRegisterController::class, 'login'])->name('login');
    Route::post('/authenticate', [LoginRegisterController::class, 'authenticate'])->name('authenticate');
    Route::get('/dashboard', [LoginRegisterController::class, 'dashboard'])->name('dashboard');
    Route::post('/logout', [LoginRegisterController::class, 'logout'])->name('logout');
});

// Throttled through the product rate limiter
Route::middleware(['middleware' => 'throttle:product'])->group(function () {
    Route::get('/products', [ProductController::class, 'index'])->name('products');
    Route::get('/product/{product}', [ProductController::class, 'show'])->name('product.show');
    Route::post('/product', [ProductController::class, 'store'])->name('product.store');
});

Testing Rate Limiting
Moment on truth.!
Run


 php artisan serve

1. API
We have eight API endpoints as below

User register ```

$ curl -X POST --header "Content-Type: application/json" --data '{"name":"Jane Doe", "email":"janedoe@example.com", "password":"pAssW0rd@s3cr3t", "password_confirmation":"pAssW0rd@s3cr3t"}' http://localhost:8000/api/user/register

HTTP/1.1 201 Created
X-RateLimit-Limit: 10
X-RateLimit-Remaining: 9
{"message":"User registered successfully","user":{"name":"John Doe","email":"johndoe@example.com","api_token":"hSAx1XBcpFDNUNFqCQvQ9OOgGfBF6t5rzxJicEsjsh0o4HGmaBCVhA7tOEpp","updated_at":"2024-03-14T23:30:45.000000Z","created_at":"2024-03-14T23:30:45.000000Z","id":9}}

On successful user registration we can see the response above. Note the `X-RateLimit-Limit` and `X-RateLimit-Remaining` values meaning we have nine more requests remaining allowed before we get blocked. We can simulate this using a tool such as [Burp suite](https://portswigger.net/burp/communitydownload) repeater or intruder as illustrated in the image below.

![Error Too May Requests API](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/c3nq36qvh7b11oragahb.png)
After sending 10 requests, we will be blocked on the 11th one as per the configuration we set in the API auth limiter.

Try the other endpoints as below.
- User login

$ curl -X POST --header "Content-Type: application/json" --data '{"email":"janedoe@example.com", "password":"pAssW0rd@s3cr3t"}' http://localhost:8000/api/user/login

- List all users

$ curl -X GET --header "Content-Type: application/json" --header "Authorization: Bearer z7ZOw6IqUbDHLdozERIwp4w9Ti8wfvaI51b8xTqp3Vc4w4JDCYAoqTbunQ12" http://localhost:8000/api/users

- Get user by ID

$ curl -X GET --header "Content-Type: application/json" --header "Authorization: Bearer z7ZOw6IqUbDHLdozERIwp4w9Ti8wfvaI51b8xTqp3Vc4w4JDCYAoqTbunQ12" http://localhost:8000/api/user/1

- Get user profile

$ curl -X GET --header "Content-Type: application/json" --header "Authorization: Bearer z7ZOw6IqUbDHLdozERIwp4w9Ti8wfvaI51b8xTqp3Vc4w4JDCYAoqTbunQ12" http://localhost:8000/api/profile

- Add product

$ curl -X POST --header "Content-Type: application/json" --header "Authorization: Bearer z7ZOw6IqUbDHLdozERIwp4w9Ti8wfvaI51b8xTqp3Vc4w4JDCYAoqTbunQ12" --data '{"name":"Nike Air Force 1 Shadow", "quantity":300, "price":4500.00}' http://localhost:8000/api/product

- Get all products

$ curl -X GET --header "Content-Type: application/json" --header "Authorization: Bearer z7ZOw6IqUbDHLdozERIwp4w9Ti8wfvaI51b8xTqp3Vc4w4JDCYAoqTbunQ12" http://localhost:8000/api/products

- Get product by ID

$ curl -X GET --header "Content-Type: application/json" --header "Authorization: Bearer z7ZOw6IqUbDHLdozERIwp4w9Ti8wfvaI51b8xTqp3Vc4w4JDCYAoqTbunQ12" http://localhost:8000/api/product/1

HTTP/1.1 200 OK
X-RateLimit-Limit: 30
X-RateLimit-Remaining: 29
{"id":1,"name":"Product One","quantity":23,"price":400.5,"created_at":"2024-03-14T15:59:26.000000Z","updated_at":"2024-03-14T15:59:26.000000Z"}

For the products endpoints we will be throttled at 30 requests per minute as per our API product limiter.

**_2. Web_**
On the web, all our authentication related pages will be rate limited at 10 requests per minute as per our auth limiter configuration while those pages concerning products will be throttled at 30 requests per minute.

- Throttled authentication requests
![Throttled auth requests](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/92g91146zuno2kjjxv4r.png)

- Throttled product requests
![Throttled product requests](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/u0mroaipt3wys88acn2j.png)

- User credentials bruteforce attack prevented on login page

![User login page bruteforce blocked](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/lyh22gf8ekrdwdvj15gx.png)

**Finally**
As we have seen, rate limiting in Laravel is a powerful tool for controlling the flow of requests to your application, ensuring its stability and security. By following the guidelines outlined in this article, you can effectively implement rate limiting to protect your application from abuse and ensure a smooth user experience.

**Let's connect**
- Connect on [LinkedIn](https://www.linkedin.com/in/benson-macharia)
- Check out my [GitHub](https://github.com/bensonmacharia)

Implementing Key and Token Authentication for a NodeJS API on AWS SAM

Benson Macharia — Sun, 05 Nov 2023 23:03:51 +0000

In this article, we will look at how to create a securely authenticated serverless NodeJs API leveraging on AWS API Gateway key and custom JWT token. AWS SAM (Serverless Application Model) creates secure, high-performing APIs and provides developers with a simplified development environment that allows them to solely focus on writing code without worrying about server management, infrastructure scaling, or hardware failures.

Designs

High-Level Architecture Diagram

The API calls from a mobile or web client are routed through an API Gateway to a Lambda function for processing. Data is persisted in a DynamoDb and logs stored on CloudWatch.

Sequence Diagram

The user first creates an account and then login using their credentials i.e. username and a password to get an access token. Making use of the access token, the user is then able to create a new event, update details and view all the events available.

Prerequisite

Basic skill in NodeJs
AWS Account
AWS CLI

Follow this guide (AWS CLI) to install.

$ aws --version
aws-cli/2.7.32 Python/3.9.11 Darwin/23.0.0 exe/x86_64 prompt/off

SAM CLI

Follow this guide (SAM CLI) to install.

$ sam --version
SAM CLI, version 1.73.0

Source
You can get the full source code here.

$ git clone https://github.com/bensonmacharia/sam-node-api.git

Let's Build

1. Environment Setup

Create the project folder and initialise the sam project.

$ mkdir sam-projects  && cd sam-projects
$ sam init -r nodejs18.x -d npm -n sam-node-api
Which template source would you like to use?
Choice: 1
Choose an AWS Quick Start application template
Template: 6
Would you like to enable X-Ray tracing on the function(s) in your application?  [y/N]: N
Would you like to enable monitoring using CloudWatch Application Insights? [y/N]: N

We are specifying to use nodejs version 18, npm as the dependency manager for our Lambda runtime and sam-node-api as the name of our project. We have also chosen not to enable X-Ray tracing and CloudWatch Application Insights.

2. Let's do some clean up

Open the generated source code in your favourite IDE. In this case, vscode.

$ cd sam-node-api && code .

Delete the files that we will not be using.

$ rm -rf sam-node-dev events && rm -rf __tests__

3. Time now to edit the template.yaml file

In AWS SAM, template.yaml serves as the foundational configuration file for defining our serverless application. Clear the content on the template.yaml file and add the following below content.
AWSTemplateFormatVersion: Specifies the AWS CloudFormation template version. For SAM templates, this is usually set to '2010-09-09'.

AWSTemplateFormatVersion: 2010-09-09

Transform: Specifies the macro (transform) that AWS CloudFormation uses to process the template. For SAM templates, this should be set to 'AWS::Serverless-2016-10-31'.

- AWS::Serverless-2016-10-31

Description: Provides a description of the CloudFormation stack and the serverless application.

Description: >-
  sam-node-api

Resources: This section defines the AWS resources that make up your serverless application, such as AWS Lambda functions, API Gateway APIs, DynamoDB tables, and more. In this case we have an API Gateway, Secrets Manager, Lambda functions and DynamoDB as below.

API Gateway

Resources:
  # Create an API Gateway and add an API Key
  ApiGatewayEndpoint:
    Type: 'AWS::Serverless::Api'
    Properties:
      StageName: Prod
      Auth:
        # Require API Key for all endpoints
        ApiKeyRequired: true
        UsagePlan:
          CreateUsagePlan: PER_API
          UsagePlanName: GatewayAuthorization

Secret Manager

JWTUserTokenSecret:
    Type: AWS::SecretsManager::Secret
    Properties:
      Name: JWTUserTokenSecret
      Description: "This secret has a dynamically generated secret password."
      # Generate a random string 30 charatcers long for the JWT secret
      GenerateSecretString:
        GenerateStringKey: 'jwt_secret'
        PasswordLength: 30
        ExcludeCharacters: '"@/\:;+*'''
        SecretStringTemplate: '{"secret_name": "sam-node-app-jwt-secret"}'

Get All Events Lambda Function

# This is a Lambda function config associated with the source code: get-all-events.js for getting all the Events.
  getAlleventsFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: src/handlers/get-all-events.getAllEventsHandler
      Runtime: nodejs18.x
      Architectures:
        - x86_64
      MemorySize: 128
      Timeout: 100
      Description: Includes a HTTP get method to get all events from a DynamoDB table.
      Policies:
        # Give Create/Read/Update/Delete Permissions to the EventsTable
        - DynamoDBCrudPolicy:
            TableName: !Ref EventsTable
        - AWSSecretsManagerGetSecretValuePolicy:
            SecretArn: !Ref JWTUserTokenSecret
      Environment:
        Variables:
          # Make table name accessible as environment variable from function code during execution
          SAMPLE_TABLE: !Ref EventsTable
      Events:
        Api:
          Type: Api
          Properties:
            RestApiId:
              Ref: ApiGatewayEndpoint
            # Expose the API through the path /v1/api/events
            Path: /v1/api/events
            Method: GET

Get Single Event By ID Lambda Function

# This is a Lambda function config associated with the source code: get-event.js for getting a single Event by its ID
  getEventByIdFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: src/handlers/get-event.getEventByIdHandler
      Runtime: nodejs18.x
      Architectures:
        - x86_64
      MemorySize: 128
      Timeout: 100
      Description: Includes a HTTP get method to get one event by id from a DynamoDB table.
      Policies:
        # Give Create/Read/Update/Delete Permissions to the EventsTable
        - DynamoDBCrudPolicy:
            TableName: !Ref EventsTable
        - AWSSecretsManagerGetSecretValuePolicy:
            SecretArn: !Ref JWTUserTokenSecret
      Environment:
        Variables:
          # Make table name accessible as environment variable from function code during execution
          SAMPLE_TABLE: !Ref EventsTable
      Events:
        Api:
          Type: Api
          Properties:
            RestApiId:
              Ref: ApiGatewayEndpoint
            # Expose the API through the path /v1/api/event/<id>
            Path: /v1/api/event/{id}
            Method: GET

Create an Event Lambda Function

# This is a Lambda function config associated with the source code: put-item.js for posting an Event into the database
  addEventFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: src/handlers/add-event.addEventHandler
      Runtime: nodejs18.x
      Architectures:
        - x86_64
      MemorySize: 128
      Timeout: 100
      Description: Includes a HTTP post method to add one event to a DynamoDB table.
      Policies:
        # Give Create/Read/Update/Delete Permissions to the EventsTable
        - DynamoDBCrudPolicy:
            TableName: !Ref EventsTable
        - AWSSecretsManagerGetSecretValuePolicy:
            SecretArn: !Ref JWTUserTokenSecret
      Environment:
        Variables:
          # Make table name accessible as environment variable from function code during execution
          SAMPLE_TABLE: !Ref EventsTable
      Events:
        Api:
          Type: Api
          Properties:
            RestApiId:
              Ref: ApiGatewayEndpoint
            # Expose the API through the path /v1/api/event
            Path: /v1/api/event
            Method: POST

User Registration Lambda Function

# This is a Lambda function config associated with the source code: user-register.js for adding a new user record into the database
  registerUserFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: src/handlers/user-register.registerUserHandler
      Runtime: nodejs18.x
      Architectures:
        - x86_64
      MemorySize: 128
      Timeout: 100
      Description: A HTTP post method to register a user and add record to a DynamoDB table.
      Policies:
        # Give Create/Read/Update/Delete Permissions to the UsersTable
        - DynamoDBCrudPolicy:
            TableName: !Ref UsersTable
      Environment:
        Variables:
          # Make table name accessible as environment variable from function code during execution
          USER_TABLE: !Ref UsersTable
      Events:
        Api:
          Type: Api
          Properties:
            RestApiId:
              Ref: ApiGatewayEndpoint
            # Expose the API through the path /v1/auth/user/register
            Path: /v1/auth/user/register
            Method: POST
            Auth:
              ApiKeyRequired: false

User Login Lambda Function

# This is a Lambda function config associated with the source code: user-login.js for fetching a user record from the database
  loginUserFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: src/handlers/user-login.loginUserHandler
      Runtime: nodejs18.x
      Architectures:
        - x86_64
      MemorySize: 128
      Timeout: 100
      Description: A HTTP post method to login a user and fetch record from a DynamoDB table.
      Policies:
        # Give Create/Read/Update/Delete Permissions to the UsersTable
        - DynamoDBCrudPolicy:
            TableName: !Ref UsersTable
        - AWSSecretsManagerGetSecretValuePolicy:
            SecretArn: !Ref JWTUserTokenSecret
      Environment:
        Variables:
          # Make table name accessible as environment variable from function code during execution
          USER_TABLE: !Ref UsersTable
      Events:
        Api:
          Type: Api
          Properties:
            RestApiId:
              Ref: ApiGatewayEndpoint
            # Expose the API through the path /v1/auth/user/login
            Path: /v1/auth/user/login
            Method: POST
            Auth:
              ApiKeyRequired: false

Events DynamoDB Table

# DynamoDB table to store Event details
  EventsTable:
    Type: AWS::Serverless::SimpleTable
    Properties:
      PrimaryKey:
        Name: id
        Type: String
      ProvisionedThroughput:
        ReadCapacityUnits: 2
        WriteCapacityUnits: 2

Users DynamoDB Table

 # DynamoDB table to store User details
  UsersTable:
    Type: AWS::Serverless::SimpleTable
    Properties:
      PrimaryKey:
        Name: username
        Type: String
      ProvisionedThroughput:
        ReadCapacityUnits: 2
        WriteCapacityUnits: 2

Outputs: Specifies the values to be exported from the stack once it's created. Outputs can include endpoint URLs, resource IDs, or any other information you want to make accessible after the deployment.

Outputs:
  ApiGateway:
    Description: "The URL is:"
    Value: !Sub "https://${ApiGatewayEndpoint}.execute-api.${AWS::Region}.amazonaws.com/Prod/"
  ApiKey:
    Description: "You can find your API Key in the AWS console: (Put in the request HEADER as 'x-api-key')"
    Value: !Sub "https://console.aws.amazon.com/apigateway/home?region=${AWS::Region}#/api-keys/${ApiGatewayEndpointApiKey}"

4. Edit the Environment Variables

Modify the env.json file to define our variables for the database names as below.

{
    "getAllEventsFunction": {
        "SAMPLE_TABLE": "EventsTable"
    },
    "getEventByIdFunction": {
        "SAMPLE_TABLE": "EventsTable"
    },
    "addEventFunction": {
        "SAMPLE_TABLE": "EventsTable"
    },
    "registerUserFunction": {
        "USER_TABLE": "UsersTable"
    },
    "loginUserFunction": {
        "USER_TABLE": "UsersTable"
    }
  }

5. Install Dependencies

# Install API Gateway SDK
$ npm install @aws-sdk/client-api-gateway

# Install Secrets Manager SDK
$ npm install @aws-sdk/client-secrets-manager 

# Install jsonwebtoken package for generating JWT tokens
$ npm install jsonwebtoken

# Install bcryptjs package for password encryption
$ npm install bcryptjs

# Install uuid package for generating uuidv4 ids
$ npm install uuid

6. Let's Handle User Registration

Create a new file user-register.mjs under sam-node-api/src/handlers

$ touch src/handlers/user-register.mjs

Add the user registration logic on user-register.mjs

// Import bcrypt for encrypting user password
import bcrypt from 'bcryptjs';
// Import uuid for generating unique user ID
import { v4 as uuidv4 } from 'uuid';

// Create a DocumentClient that represents the query to add an item
import { DynamoDBClient } from '@aws-sdk/client-dynamodb';
import { DynamoDBDocumentClient, PutCommand } from '@aws-sdk/lib-dynamodb';
const client = new DynamoDBClient({});
const ddbDocClient = DynamoDBDocumentClient.from(client);

// Get the DynamoDB table name from environment variables
const tableName = process.env.USER_TABLE;

/**
 * A HTTP post method to add one user to a DynamoDB table.
 */
export const registerUserHandler = async (event) => {
    if (event.httpMethod !== 'POST') {
        throw new Error(`postMethod only accepts POST method, you tried: ${event.httpMethod} method.`);
    }
    // All log statements are written to CloudWatch
    console.info('received:', event);

    // Get username and password from the body of the request
    const body = JSON.parse(event.body);
    // Use a random uuidv4 string as a User ID 
    const id = uuidv4();

    const username = body.username;

    // Generate a hashed password string
    const password = bcrypt.hashSync(body.password, 8);

    // Creates a new user, or replaces an old user record with a new one
    var params = {
        TableName: tableName,
        Item: { id: id, username: username, password: password }
    };

    try {
        const data = await ddbDocClient.send(new PutCommand(params));
        console.log("Success - user registered or updated", data);
    } catch (err) {
        console.log("Error", err.stack);
    }

    const responseData = {
        message: "Registration successful. Proceed to login",
        username: body.username

    }

    const response = {
        statusCode: 201,
        body: JSON.stringify(responseData)
    };

    // All log statements are written to CloudWatch
    console.info(`response from: ${event.path} statusCode: ${response.statusCode} body: ${response.body}`);
    return response;
};

7. Handle User Login

Create a new file user-login.mjs under sam-node-api/src/handlers

$ touch src/handlers/user-login.mjs

Add the user login logic on user-login.mjs

// Import bcrypt for encrypting user password and comparing the password hash
import bcrypt from 'bcryptjs';
// Create a DocumentClient that represents the query to add an item
import { DynamoDBClient } from '@aws-sdk/client-dynamodb';
import { DynamoDBDocumentClient, GetCommand } from '@aws-sdk/lib-dynamodb';
import { GetSecretValueCommand, SecretsManagerClient } from "@aws-sdk/client-secrets-manager";
// Import jwt for generating the user token
import jwt from 'jsonwebtoken';
const client = new DynamoDBClient({});
const ddbDocClient = DynamoDBDocumentClient.from(client);
const clientsecret = new SecretsManagerClient();

// Get the DynamoDB table name from environment variables
const tableName = process.env.USER_TABLE;

/**
 * A HTTP post method for user login.
 */
export const loginUserHandler = async (event) => {
    if (event.httpMethod !== 'POST') {
        throw new Error(`postMethod only accepts POST method, you tried: ${event.httpMethod} method.`);
    }
    // All log statements are written to CloudWatch
    console.info('received:', event);

    // Get username and password from the body of the request
    const body = JSON.parse(event.body);
    const username = body.username;
    const password = body.password;

    // Fetch the JWT secret string from AWS Secrets Manager 
    const secret_value = await clientsecret.send(new GetSecretValueCommand({
        SecretId: "JWTUserTokenSecret",
    }));
    const jwt_secret = JSON.parse(secret_value.SecretString);

    var params = {
        TableName: tableName,
        Key: { username: username },
    };

    var token = "";
    var message = "";
    var status_code = 200;

    try {
        const data = await ddbDocClient.send(new GetCommand(params));
        var item = data.Item;
        // Comprate a hash of the received password from the request body with the password hash from the database, if they match login the user and generate a JWT token
        if (bcrypt.compareSync(password, item.password)) {
            // create JWT token
            const jwttoken = jwt.sign(
                {
                    userId: item.id,
                    userName: body.username,
                },
                jwt_secret.jwt_secret,
                { expiresIn: "1h" }
            );
            message = "Login successful.";
            token = jwttoken;
        } else {
            status_code = 403;
            message = "Wrong password. Try again";
            token = "NOT OKAY";
        }
    } catch (err) {
        console.log("Error", err);
    }

    const responseData = {
        message: message,
        user: {
            username: body.username,
            token: token
        }
    }

    const response = {
        statusCode: status_code,
        body: JSON.stringify(responseData)
    };

    // All log statements are written to CloudWatch
    console.info(`response from: ${event.path} statusCode: ${response.statusCode} body: ${response.body}`);
    return response;
};

8. Add Events Logic

Rename the file put-item.mjs in sam-node-api/src/handlers to add-event.mjs

$ mv src/handlers/put-item.mjs src/handlers/add-event.mjs

Edit the add-event.mjs file

// Create a DocumentClient that represents the query to add an item
import { DynamoDBClient } from '@aws-sdk/client-dynamodb';
import { DynamoDBDocumentClient, PutCommand } from '@aws-sdk/lib-dynamodb';
import { GetSecretValueCommand, SecretsManagerClient } from "@aws-sdk/client-secrets-manager";
// Import jwt for validating the user token
import jwt from 'jsonwebtoken';

const client = new DynamoDBClient({});
const ddbDocClient = DynamoDBDocumentClient.from(client);
const clientsecret = new SecretsManagerClient();

// Get the DynamoDB table name from environment variables
const tableName = process.env.SAMPLE_TABLE;

/**
 * A HTTP post method to add one Event item to a DynamoDB table.
 */
export const addEventHandler = async (event) => {
    if (event.httpMethod !== 'POST') {
        throw new Error(`postMethod only accepts POST method, you tried: ${event.httpMethod} method.`);
    }
    // All log statements are written to CloudWatch
    console.info('received:', event);

    // Get token from the authorization header
    const token = await event.headers.Authorization.split(" ")[1];
    if (!token) {
        throw new Error(`Authorization token required.`);
    }

    // Get Event id, title and description from the body of the request
    const body = JSON.parse(event.body);
    const id = body.id;
    const title = body.title;
    const description = body.description;

    // Fetch the JWT secret string from AWS Secrets Manager 
    const secret_value = await clientsecret.send(new GetSecretValueCommand({
        SecretId: "JWTUserTokenSecret",
    }));
    const jwt_secret = JSON.parse(secret_value.SecretString);

    //Check if the token is valid
    const decodedToken = jwt.verify(token, jwt_secret.jwt_secret);
    if (!decodedToken) {
        throw new Error(`Authorization token invalid or it has expired.`);
    }

    // Decode the JWT token to get user data
    const userID = decodedToken.userId;
    const userName = decodedToken.userName;

     console.info('decode-username:', userName);
     console.info('decode-userid:', userID);

    // Creates a new Event item, or replaces an old item with a new item
    var params = {
        TableName : tableName,
        Item: { id : id, title: title, description: description, added_by:  userName, added_by_id: userID}
    };

    try {
        const data = await ddbDocClient.send(new PutCommand(params));
        console.log("Success - event added successfully", data);
      } catch (err) {
        console.log("Error", err.stack);
      }

    const response = {
        statusCode: 200,
        body: JSON.stringify(body)
    };

    // All log statements are written to CloudWatch
    console.info(`response from: ${event.path} statusCode: ${response.statusCode} body: ${response.body}`);
    return response;
};

9. View All Events Login

Rename the file get-all-items.mjs in sam-node-api/src/handlers to get-all-events.mjs

$ mv src/handlers/get-all-items.mjs src/handlers/get-all-events.mjs

Edit the get-all-events.mjs file

import { DynamoDBClient } from '@aws-sdk/client-dynamodb';
import { DynamoDBDocumentClient, ScanCommand } from '@aws-sdk/lib-dynamodb';
import { GetSecretValueCommand, SecretsManagerClient } from "@aws-sdk/client-secrets-manager";
// Import jwt for validating the user token
import jwt from 'jsonwebtoken';

const client = new DynamoDBClient({});
const ddbDocClient = DynamoDBDocumentClient.from(client);
const clientsecret = new SecretsManagerClient();

// Get the DynamoDB table name from environment variables
const tableName = process.env.SAMPLE_TABLE;

/**
 * A HTTP get method to get all events from a DynamoDB table.
 */
export const getAllEventsHandler = async (event) => {
    if (event.httpMethod !== 'GET') {
        throw new Error(`getAllItems only accept GET method, you tried: ${event.httpMethod}`);
    }
    // All log statements are written to CloudWatch
    console.info('received:', event);

    // Get token from the authorization header
    const token = await event.headers.Authorization.split(" ")[1];
    if (!token) {
        throw new Error(`Authorization token required.`);
    }

    // Fetch the JWT secret string from AWS Secrets Manager 
    const secret_value = await clientsecret.send(new GetSecretValueCommand({
        SecretId: "JWTUserTokenSecret",
    }));
    const jwt_secret = JSON.parse(secret_value.SecretString);

    //Check if the token is valid
    const decodedToken = jwt.verify(token, jwt_secret.jwt_secret);
    if (!decodedToken) {
        throw new Error(`Authorization token invalid or it has expired.`);
    }

    // Get all items from the table (only first 1MB data, you can use `LastEvaluatedKey` to get the rest of data)
    var params = {
        TableName : tableName
    };

    try {
        const data = await ddbDocClient.send(new ScanCommand(params));
        var items = data.Items;
    } catch (err) {
        console.log("Error", err);
    }

    const response = {
        statusCode: 200,
        body: JSON.stringify(items)
    };

    // All log statements are written to CloudWatch
    console.info(`response from: ${event.path} statusCode: ${response.statusCode} body: ${response.body}`);
    return response;
}

10. View Event By ID Login

Rename the file get-by-id.mjs in sam-node-api/src/handlers to get-event.mjs

$ mv src/handlers/get-by-id.mjs src/handlers/get-event.mjs

Edit the get-event.mjs file

import { DynamoDBClient } from '@aws-sdk/client-dynamodb';
import { DynamoDBDocumentClient, GetCommand } from '@aws-sdk/lib-dynamodb';
import { GetSecretValueCommand, SecretsManagerClient } from "@aws-sdk/client-secrets-manager";
// Import jwt for validating the user token
import jwt from 'jsonwebtoken';

const client = new DynamoDBClient({});
const ddbDocClient = DynamoDBDocumentClient.from(client);
const clientsecret = new SecretsManagerClient();

// Get the DynamoDB table name from environment variables
const tableName = process.env.SAMPLE_TABLE;

/**
 * A HTTP get method to get one Event item by id from a DynamoDB table.
 */
export const getEventByIdHandler = async (event) => {
  if (event.httpMethod !== 'GET') {
    throw new Error(`getMethod only accept GET method, you tried: ${event.httpMethod}`);
  }
  // All log statements are written to CloudWatch
  console.info('received:', event);

  // Get the token from the authorization header
  const token = await event.headers.Authorization.split(" ")[1];
  if (!token) {
      throw new Error(`Authorization token required.`);
  }

  // Fetch the JWT secret string from AWS Secrets Manager 
  const secret_value = await clientsecret.send(new GetSecretValueCommand({
      SecretId: "JWTUserTokenSecret",
  }));
  const jwt_secret = JSON.parse(secret_value.SecretString);

  //Check if the token is valid
  const decodedToken = jwt.verify(token, jwt_secret.jwt_secret);
  if (!decodedToken) {
      throw new Error(`Authorization token invalid or it has expired.`);
  }

  // Get id from pathParameters from APIGateway because of `/{id}` at template.yaml
  const id = event.pathParameters.id;

  // Get the item from the table
  var params = {
    TableName : tableName,
    Key: { id: id },
  };

  var scode = 200;
  var bdy = "";

  try {
    const data = await ddbDocClient.send(new GetCommand(params));
    var item = data.Item;
    bdy = JSON.stringify(item);
    if (!bdy) {
      scode = 404;
      bdy = JSON.stringify("Event details not found.");
    }
  } catch (err) {
    console.log("Error", err);
    scode = 400;
    bdy = err;
  }

  const response = {
    statusCode: scode,
    body: bdy
  };

  // All log statements are written to CloudWatch
  console.info(`response from: ${event.path} statusCode: ${response.statusCode} body: ${response.body}`);
  return response;
}

11. Build and Deploy the API

Build the SAM App

$ sam build
Build Succeeded

Built Artifacts  : .aws-sam/build
Built Template   : .aws-sam/build/template.yaml

Deploy the SAM App

$ sam deploy --guided
Stack Name [sam-app]: sam-node-app
AWS Region [us-east-1]: us-east-1
#Shows you resources changes to be deployed and require a 'Y' to initiate deploy
Confirm changes before deploy [y/N]: y
#SAM needs permission to be able to create roles to connect to the resources in your template
Allow SAM CLI IAM role creation [Y/n]: Y
#Preserves the state of previously provisioned resources when an operation fails
Disable rollback [y/N]: N
Save arguments to configuration file [Y/n]: Y
SAM configuration file [samconfig.toml]: 
SAM configuration environment [default]: 

Deploy this changeset? [y/N]: y

CloudFormation outputs from deployed stack
---------------
Outputs                                                                                                                                                                                                           
---------------
Key                 ApiKey                                                                                                                                                                                        
Description         You can find your API Key in the AWS console: (Put in the request HEADER as 'x-api-key')                                                                                                      
Value               https://console.aws.amazon.com/apigateway/home?region=us-east-1#/api-keys/w22y9chd81                                                                                                          

Key                 ApiGateway                                                                                                                                                                                    
Description         The URL is:                                                                                                                                                                                   
Value               https://d2io9v58l4.execute-api.us-east-1.amazonaws.com/Prod/                                                                                                                                  
-----------------

Successfully created/updated stack - sam-node-app in us-east-1

12. Test the API

Test user registration

$ curl -X POST --header "Content-Type: application/json" --data '{"username":"bmacharia", "password":"pAssW0rd@s3cr3t"}' https://d2io9v58l4.execute-api.us-east-1.amazonaws.com/Prod/v1/auth/user/register

HTTP/1.1 200 OK
{"message":"Registration successful. Proceed to login","username":"bmacharia"}%

Test user login

$ curl -X POST --header "Content-Type: application/json" --data '{"username":"bmacharia", "password":"pAssW0rd@s3cr3t"}' https://d2io9v58l4.execute-api.us-east-1.amazonaws.com/Prod/v1/auth/user/login

HTTP/1.1 200 OK
{"message":"Login successful.","user":{"username":"bmacharia","token":"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VySWQiOiI4NDMxMTU5Ny05YTlkLTQ2NDEtYjBjMS1mYTUzMGIzNTkwNzYiLCJ1c2VyTmFtZSI6ImJtYWNoYXJpYSIsImlhdCI6MTY5ODk2MjM2OSwiZXhwIjoxNjk4OTY1OTY5fQ.keoma2P6fdXnkrRklWX1vVoyaUXB_G06p6gvQLyOHHY"}}%

Test creating event

$ curl -X POST --header "Content-Type: application/json" --header "x-api-key: Laqoh0N6kCao2yw4SeDxCa9pwRdjxwra267VGjMH" --header "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VySWQiOiI4NDMxMTU5Ny05YTlkLTQ2NDEtYjBjMS1mYTUzMGIzNTkwNzYiLCJ1c2VyTmFtZSI6ImJtYWNoYXJpYSIsImlhdCI6MTY5ODk2MjM2OSwiZXhwIjoxNjk4OTY1OTY5fQ.keoma2P6fdXnkrRklWX1vVoyaUXB_G06p6gvQLyOHHY" --data '{"id":"2", "title":"Cybersecurity Summit", "description":"Downtown Hotel, 31st December 2023"}' https://d2io9v58l4.execute-api.us-east-1.amazonaws.com/Prod/v1/api/event

HTTP/1.1 200 OK
{"id":"2","title":"Cybersecurity Summit","description":"Downtown Hotel, 31st December 2023"}%

Test getting all events

$ curl --header "Content-Type: application/json" --header "x-api-key: Laqoh0N6kCao2yw4SeDxCa9pwRdjxwra267VGjMH" --header "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VySWQiOiI4NDMxMTU5Ny05YTlkLTQ2NDEtYjBjMS1mYTUzMGIzNTkwNzYiLCJ1c2VyTmFtZSI6ImJtYWNoYXJpYSIsImlhdCI6MTY5ODk2MjM2OSwiZXhwIjoxNjk4OTY1OTY5fQ.keoma2P6fdXnkrRklWX1vVoyaUXB_G06p6gvQLyOHHY" https://d2io9v58l4.execute-api.us-east-1.amazonaws.com/Prod/v1/api/events

HTTP/1.1 200 OK
[{"description":"Downtown Hotel, 31st December 2023","id":"2","title":"Cybersecurity Summit","added_by":"bmacharia","added_by_id":"84311597-9a9d-4641-b0c1-fa530b359076"},{"description":"Whitesands Hotel, 1st January 2024","id":"1","title":"GRC Conference","added_by":"bmacharia","added_by_id":"84311597-9a9d-4641-b0c1-fa530b359076"}]%

Get Event By ID

$ curl --header "Content-Type: application/json" --header "x-api-key: Laqoh0N6kCao2yw4SeDxCa9pwRdjxwra267VGjMH" --header "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VySWQiOiI4NDMxMTU5Ny05YTlkLTQ2NDEtYjBjMS1mYTUzMGIzNTkwNzYiLCJ1c2VyTmFtZSI6ImJtYWNoYXJpYSIsImlhdCI6MTY5ODk2MjM2OSwiZXhwIjoxNjk4OTY1OTY5fQ.keoma2P6fdXnkrRklWX1vVoyaUXB_G06p6gvQLyOHHY" https://d2io9v58l4.execute-api.us-east-1.amazonaws.com/Prod/v1/api/event/1

HTTP/1.1 200 OK
{"description":"Whitesands Hotel, 1st January 2024","id":"1","title":"GRC Conference","added_by":"bmacharia","added_by_id":"84311597-9a9d-4641-b0c1-fa530b359076"}%

13. Clean Up

Delete the SAM App

$ sam delete --stack-name sam-node-app
Are you sure you want to delete the stack sam-node-app in the region us-east-1 ? [y/N]: y
Are you sure you want to delete the folder sam-node-app in S3 which contains the artifacts? [y/N]: y

Deleted successfully

Role-based Access Control in Golang with jwt-go

Benson Macharia — Tue, 20 Jun 2023 18:46:50 +0000

As a security engineer, what's the first thing you check when testing a web or mobile application? Access control misconfigurations must be at the top of your list if you like going for the low hanging fruits first.

Surprisingly, access control is one of the components that application developers think will be simple, but ends up taking much of their time with no end user value, and even worse ends being poorly done, tremendous exposing the application to security risks. It is no coincidence that Authentication and Authorization related issues take the first two positions in the OWASP Top 10 - 2023 list of common API security risks.

For this reason, in this article I am going to illustrate the application of Role-based Access Control; a simple and yet secure access control approach for enterprise APIs.

Role-based Access Control (RBAC)
Role-based Access Control is a mechanism that restricts access to an information system based on user job functions or roles. Access to data is restricted through permissions and privileges that are attached to different user roles.

Let's Build
By the end of this article we are going build a simple room booking API with Go that will allow visitors to view available rooms as well as register, login and book rooms. There will also be an administrator who will have the permissions map user to permissions as well as add and update rooms information.

Prerequisites

Basic knowledge of Go
Patience 😄

Source Code

$ git clone https://github.com/bensonmacharia/jwt-go-rbac.git

Step 1: Setting up the environment

Get your favourite IDE, mine is vscode
Download and install Go
Confirm installed go version

$ go version    
go version go1.19.4 darwin/arm64

Install MySQL and test connection

// confirm installation by checking version
$ mysql --version
mysql  Ver 8.0.32 for macos12.6 on arm64 (Homebrew)

// test connection
$ mysql -u bmacharia -p
Enter password: 
mysql>

Create a database

// create database
$ mysql> CREATE DATABASE jwt_go_rbac;
Query OK, 1 row affected (0.07 sec)

// confirm database creation
$ mysql> show databases;
+-------------------------+
| Database                |
+-------------------------+             |
| information_schema      |
| jwt_go_rbac             |
| mysql                   |
| performance_schema      |
| sys                     |
+-------------------------+
5 rows in set (0.01 sec)

Step 2: Setting up the project

Create project folder and initialise go project

$ mkdir jwt-go-rbac                                                                
$ cd jwt-go-rbac            
$ go mod init bmacharia/jwt-go-rbac            
go: creating new go.mod: module bmacharia/jwt-go-rbac

Create the main.go and .env files

// main go application file
$ touch main.go

// environment variables configuration file
$ touch .env

Open the project in your IDE and edit the .env file as below

Step 3: Configure the database connection

Install required packages

// package to load .env file
$ go get github.com/joho/godotenv
// package to allow connection to MySQL database
$ go get -u gorm.io/driver/mysql
// a simple Go HTTP web framework
$ go get github.com/gin-gonic/gin
// ORM (Object Relational Mapping) library for Go
$ go get -u gorm.io/gorm
// jwt-go package
$ go get -u github.com/golang-jwt/jwt/v5

Create a database connection file

// create a database folder and a file inside it
$ mkdir database
$ touch database/database.go

Edit the database.go file as below. We are using GORM to initiate a connection to MySQL database.

Edit the main.go file to test database connection. In this file we are loading the .env file, database connection and starting the gin web server on port 8000
Test database connection by running the go run command

$ go run main.go
2023/05/01 23:25:13 .env file loaded successfully
2023/05/01 23:25:13 `Successfully connected to the database
[GIN-debug] Listening and serving HTTP on :8000

Step 4: Create database models

Create the user and role models. The user model defines the user object details as well as user properties in the database. On the other hand, the role model details properties about a role to be assigned to each user.

// create a user.go file inside the model directory
$ mkdir model
$ touch model/user.go
$ touch model/role.go

model/user.go
model/role.go
Run database migrations

//edit main.go file to add automigration script
...
// run database migrations and add seed data
func loadDatabase() {
    database.InitDb()
    database.Db.AutoMigrate(&model.Role{})
    database.Db.AutoMigrate(&model.User{})
    seedData()
}

// load seed data into the database
func seedData() {
    var roles = []model.Role{{Name: "admin", Description: "Administrator role"}, {Name: "customer", Description: "Authenticated customer role"}, {Name: "anonymous", Description: "Unauthenticated customer role"}}
    var user = []model.User{{Username: os.Getenv("ADMIN_USERNAME"), Email: os.Getenv("ADMIN_EMAIL"), Password: os.Getenv("ADMIN_PASSWORD"), RoleID: 1}}
    database.Db.Save(&roles)
    database.Db.Save(&user)
}
// run migration
$ go run main.go
..

Confirm that users and roles tables have been created

mysql> show tables;
+-----------------------+
| Tables_in_jwt_go_rbac |
+-----------------------+
| roles                 |
| users                 |
+-----------------------+
2 rows in set (0.01 sec)
mysql> desc users;
mysql> desc roles;
mysql> select * from roles;
mysql> select * from users;

Create models for user registration, login and update

$ touch model/register.go
$ touch model/login.go
$ touch model/update.go

model/register.go
model/login.go
model/update.go

Step 5: Create controllers to interact with database content

Create the role and user controller files

$ touch controller/role.go
$ touch controller/user.go

controller/role.go
controller/user.go
Add registration and login routes

// edit main.go
func serveApplication() {
    router := gin.Default()
    authRoutes := router.Group("/auth/user")
        // registration route
    authRoutes.POST("/register", controller.Register)
        // login route
    authRoutes.POST("/login", controller.Login)

    router.Run(":8000")
    fmt.Println("Server running on port 8000")
}

Test user registration and login

// run the application
$ go run main.go
// register user
$ curl -X POST http://localhost:8000/auth/user/register \
     -H "Content-Type: application/json" \
     -H "Accept: application/json" \
     -d '{"username": "test","email":"test@bmacharia.com","password":"super^Secret!007"}'
// test user login
$ curl -X POST http://localhost:8000/auth/user/login \
     -H "Content-Type: application/json" \
     -H "Accept: application/json" \
     -d '{"username": "test","password":"super^Secret!007"}'

Step 6. Add JWT utility for token creation and validation

To create a secure Role-based authentication scheme, we need to generate a unique token when the user authenticates. This is then used to track their assigned role as they consume the availed resources. In this project we are going to use the jwt-go package to generate a JWT token that will encapsulate the user details, assigned role and permissions.
Create the JWT utility files

// create a util directory
$ mkdir util
$ touch jwt.go
$ touch jwtAuth.go

jwt.go
jwtAuth.go
Add token to user login response

// edit user controller and append 
func Login(context *gin.Context) {
     jwt, err := util.GenerateJWT(user)
     if err != nil {
        context.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
        return
    }

    context.JSON(http.StatusOK, gin.H{"token": jwt, "username": input.Username, "message": "Successfully logged in"})
}

Add admin functions routes

// edit main.go
func serveApplication() {
        adminRoutes := router.Group("/admin")
    adminRoutes.Use(util.JWTAuth())
    adminRoutes.GET("/users", controller.GetUsers)
    adminRoutes.GET("/user/:id", controller.GetUser)
    adminRoutes.PUT("/user/:id", controller.UpdateUser)
    adminRoutes.POST("/user/role", controller.CreateRole)
    adminRoutes.GET("/user/roles", controller.GetRoles)
    adminRoutes.PUT("/user/role/:id", controller.UpdateRole)
}

Step 7: Run and Test Admin Funtions

Run API

$  go run main.go

Admin login

// admin user login
$ curl -X POST http://localhost:8000/auth/user/login \
     -H "Content-Type: application/json" \
     -H "Accept: application/json" \
     -d '{"username": "test","password":"super^Secret!007"}'

Get All Users

// use admin token from login response
$ curl -X GET http://localhost:8000/admin/users \
     -H "Content-Type: application/json" \
     -H "Accept: application/json" \
     -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlYXQiOjE2ODQ0NDc1ODQsImlhdCI6MTY4NDQ0NTc4NCwiaWQiOjEsInJvbGUiOjF9.CKQf2GggCP1cnGqfTp_2R77Q7GsQBX_dxf5PSLEbTx8" \
     -d '{"username": "test","password":"super^Secret!007"}'

Get User by ID

$ curl -X GET http://localhost:8000/admin/user/1 \
     -H "Content-Type: application/json" \
     -H "Accept: application/json" \
     -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlYXQiOjE2ODQ0NDc1ODQsImlhdCI6MTY4NDQ0NTc4NCwiaWQiOjEsInJvbGUiOjF9.CKQf2GggCP1cnGqfTp_2R77Q7GsQBX_dxf5PSLEbTx8"

Update User

$ curl -X PUT http://localhost:8000/admin/user/2 \
     -H "Content-Type: application/json" \
     -H "Accept: application/json" \
     -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlYXQiOjE2ODQ0NDc1ODQsImlhdCI6MTY4NDQ0NTc4NCwiaWQiOjEsInJvbGUiOjF9.CKQf2GggCP1cnGqfTp_2R77Q7GsQBX_dxf5PSLEbTx8" \
     -d '{"username": "test","email":"test@gmail.com","role_id":"2"}'

Create Role

$ curl -X POST http://localhost:8000/admin/user/role \
     -H "Content-Type: application/json" \
     -H "Accept: application/json" \
     -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlYXQiOjE2ODQ0NDc1ODQsImlhdCI6MTY4NDQ0NTc4NCwiaWQiOjEsInJvbGUiOjF9.CKQf2GggCP1cnGqfTp_2R77Q7GsQBX_dxf5PSLEbTx8" \
     -d '{"name": "testing","description":"Test user role"}'

Get All Roles

$ curl -X GET http://localhost:8000/admin/user/roles \
     -H "Content-Type: application/json" \
     -H "Accept: application/json" \
     -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlYXQiOjE2ODQ0NDc1ODQsImlhdCI6MTY4NDQ0NTc4NCwiaWQiOjEsInJvbGUiOjF9.CKQf2GggCP1cnGqfTp_2R77Q7GsQBX_dxf5PSLEbTx8"

Update Role

$ curl -X PUT http://localhost:8000/admin/user/role/4 \
     -H "Content-Type: application/json" \
     -H "Accept: application/json" \
     -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlYXQiOjE2ODQ0NDc1ODQsImlhdCI6MTY4NDQ0NTc4NCwiaWQiOjEsInJvbGUiOjF9.CKQf2GggCP1cnGqfTp_2R77Q7GsQBX_dxf5PSLEbTx8" \
     -d '{"name":"accountant","description":"Accountant user role"}'

Add Room

$ curl -X POST http://localhost:8000/admin/room/add \
     -H "Content-Type: application/json" \
     -H "Accept: application/json" \
     -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlYXQiOjE2ODQ0NDc1ODQsImlhdCI6MTY4NDQ0NTc4NCwiaWQiOjEsInJvbGUiOjF9.CKQf2GggCP1cnGqfTp_2R77Q7GsQBX_dxf5PSLEbTx8" \
     -d '{"name": "Room 9","location":"Second Floor"}'

List all Rooms

$ curl -X GET http://localhost:8000/api/view/rooms \
     -H "Content-Type: application/json" \
     -H "Accept: application/json"

Get Room by ID

$ curl -X GET http://localhost:8000/api/view/room/3 \
     -H "Content-Type: application/json" \
     -H "Accept: application/json"

Step 8: Test the room booking service

Book a Room

$ curl -X POST http://localhost:8000/api/room/book \
     -H "Content-Type: application/json" \
     -H "Accept: application/json" \
     -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlYXQiOjE2ODUyMjk0MDEsImlhdCI6MTY4NTIyNzYwMSwiaWQiOjI0LCJyb2xlIjoyfQ.h8R51DA5N_xeCa8xR1HLeOo4JTmIGjUp3oMPJLuBv3g" \
     -d '{"room_id": 3}'

List all Bookings

$ curl -X GET http://localhost:8000/admin/room/bookings \
     -H "Content-Type: application/json" \
     -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlYXQiOjE2ODcyOTkyMTAsImlhdCI6MTY4NzI5NzQxMCwiaWQiOjEsInJvbGUiOjF9.3oztz8EgE-l3byKWzCI760FE-BmRY7B-BohnYydDElc" \
     -H "Accept: application/json"

List all User Bookings

$ curl -X GET http://localhost:8000/api/rooms/booked \
     -H "Content-Type: application/json" \
     -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlYXQiOjE2ODUyMjk0MDEsImlhdCI6MTY4NTIyNzYwMSwiaWQiOjI0LCJyb2xlIjoyfQ.h8R51DA5N_xeCa8xR1HLeOo4JTmIGjUp3oMPJLuBv3g" \
     -H "Accept: application/json"

Winding up
RBAC presents the simplest form of access control that can help prevent unauthorised access to data. This facilitates compliance to various regulatory and compliance requirements especially those related to data protection, privacy and system access. With RBAC, it's also easy to maintain an audit trail of all user activities which can significantly help speed up incident response process.

Let's connect
LinkedIn
Twitter
Blog

AWS CloudWatch Logging for Golang Microservices on EKS with Logrus and Fluent bit

Benson Macharia — Tue, 07 Feb 2023 19:57:55 +0000

With the API-first approach of building software products becoming prominent and APIs now making up the most common communication channel for internet traffic, API security becomes paramount.
API logging and monitoring is a key component in securing APIs, amongst others that include discovery, vulnerability management and compliance. APIs should have an inbuilt mechanism to collect access and transaction activities which can aid in their monitoring to identify potentially malicious activity.

Golang being one of the most popular frameworks for developing APIs due to its performance and support for concurrency, I have created a guide to aid developers collect API logs through the Go LogRus library and forward such to AWS CloudWatch from a microservice deployed in AWS EKS.

Check it here