DEV Community

Ahmet Küçükoğlu
Ahmet Küçükoğlu

Posted on • Edited on • Originally published at ahmetkucukoglu.com

3 1

Developing AWS Serverless RESTful API

This article was originally published at: https://www.ahmetkucukoglu.com/en/developing-aws-serverless-restful-api/

1. Introduction

This article series is about how to develop RestfulAPI with a serverless approach. We will use AWS as a cloud provider.

In this part, i will show the scenario and give information about what will be the result.

Our RESTful API endpoints will be like below.

[POST] api/ads

[PUT] api/ads/{id}

[DELETE] api/ads/{id}

[GET] api/ads

[GET] api/ads/{id}

We will use the following AWS services.

API Gateway : It will provide the endpoints we need for the RESTful API.

Lambda : It will provide to write function for the GET, POST, PUT and DELETE requests.

DynamoDB : It will provide NoSQL solution. In the POST, PUT and DELETE requests, we will write the data to the DynamoDB.

ElasticCache : It will provide the distributed cache solution. In the GET request, we will read the data from Redis.

S3 : It will provide our code to be versioned and stored after each deployment.

CloudFormation : It will provide us to create and manage automatically all services mentioned above.

When the project is done, the architecture on AWS will be like below.

Serverless

As a result, the record will be sent to the DynamoDB which is NoSQL solution of AWS in the POST request. Record changes in our DynamoDB table will be sended on the ElasticCache which is the solution service of the distributed cache of AWS. The data in the GET request will be read from ElasticCache instead of the DynamoDB.

2. Preparing the Development Environment

In this part, we will arrange our development environment.

Requirements

  • Visual Studio Code
  • AWS Account
  • AWS CLI
  • Serverless Application Framework
2.1. Creating a User in AWS Console

Creating an authorized user in AWS is required to make deployment with serverless.

Go to IAM from AWS Console. Choose "User" on the left menu. Click the "Add user" button.

Write "serverless" in the username fied. Choose "Programmatic Access" from the Access type section. Go to the second step(Permissions).

Step 1 — User Informations

Choose "Attach existing policies directly" from the Permissions section. Give the following policies. Go to the third step(Tags).

  • AWSLambdaFullAccess
  • IAMFullAccess
  • AmazonS3FullAccess
  • CloudWatchLogsFullAccess
  • AmazonAPIGatewayAdministrator
  • AWSCloudFormationFullAccess
  • AmazonDynamoDBFullAccess
  • AmazonElastiCacheFullAccess

Step 2 — User Permissions

Write "Application" in the Tag Key field and "Serverless RESTful API" in the Tag Value field. Go to the fourth step(Review).

Step 3 — Tagging the User

Click the "Create User" button.

Step 4 — User Review

On the screen, you will see the information about Access Key Id and Secret Access Key. Keep them in a side.

Step 5 — Access Informations of the User

2.2. The Installation of AWS CLI

Open Command Prompt and run command line below.

aws configure --profile serverlessuser
Enter fullscreen mode Exit fullscreen mode

AWS Access Key ID : The Access Key Id you will copy

AWS Secret Access Key : The Secret Access Key you will copy

Default region name : eu-central-1

Default output format : json

2.3. The Installation of the Serverless Framework

We will use the Serverless Application Framework to develop this application. You can visit the address below for the installation.

Serverless Framework - AWS Lambda Guide - Installing The Serverless Framework

3. Preparing the Endpoints

In this part, we will create the project and make deployment in a simplest way. We will not send a record to the DynamoDB yet. We will GET response after sending request to the endpoints.

Create the project folder with the command line below.

mkdir serverless-api && cd serverless-api
Enter fullscreen mode Exit fullscreen mode

Create the serverless project with the command line below.

serverless create --template aws-nodejs --name ads-api
Enter fullscreen mode Exit fullscreen mode

Open the project with the command line below via Visual Studio Code.

code .
Enter fullscreen mode Exit fullscreen mode

Update the serverless.yml file like below.

serverless.yml v1

service: ads-api
provider:
name: aws
runtime: nodejs10.x
region: eu-central-1
profile: serverlessuser
apiKeys:
- AdsAPIKey
functions:
create:
handler: create.create
events:
- http:
path: api/ads
method: post
private: true
request:
schema:
application/json: ${file(create_request.json)}
update:
handler: update.update
events:
- http:
path: api/ads/{id}
method: put
private: true
request:
schema:
application/json: ${file(update_request.json)}
delete:
handler: delete.delete
events:
- http:
path: api/ads/{id}
method: delete
private: true
getAll:
handler: getAll.getAll
events:
- http:
path: api/ads
method: get
private: true
getById:
handler: getById.getById
events:
- http:
path: api/ads/{id}
method: get
private: true
view raw serverless.yml hosted with ❤ by GitHub

Delete the handler.js file. Create 4 files to be named as "create.js", "update.js", " delete.js" and "getByld.js".

create.js v1

'use strict';
module.exports.create = async event => {
var json = JSON.parse(event.body);
return {
statusCode: 200,
body: JSON.stringify(
{
method: 'Create',
body: json,
},
null,
2
),
};
};
view raw create.js hosted with ❤ by GitHub

update.js v1

'use strict';
module.exports.update = async event => {
var json = JSON.parse(event.body);
return {
statusCode: 200,
body: JSON.stringify(
{
method: 'Update',
id: event.pathParameters.id,
body: json,
},
null,
2
),
};
};
view raw update.js hosted with ❤ by GitHub

delete.js v1

'use strict';
module.exports.delete = async event => {
return {
statusCode: 200,
body: JSON.stringify(
{
method: 'Delete',
id: event.pathParameters.id,
},
null,
2
),
};
};
view raw delete.js hosted with ❤ by GitHub

getAll.js v1

'use strict';
module.exports.getAll = async event => {
return {
statusCode: 200,
body: JSON.stringify(
{
method: 'Get All'
},
null,
2
),
};
};
view raw getAll.js hosted with ❤ by GitHub

getById.js v1

'use strict';
module.exports.getById = async event => {
return {
statusCode: 200,
body: JSON.stringify(
{
method: 'Get By Id',
id: event.pathParameters.id
},
null,
2
),
};
};
view raw getById.js hosted with ❤ by GitHub

Create a file named "create_request.json" in the project folder and paste the json in it. Defines the POST request model. Sets required fields.

create_request.json

{
"definitions": {},
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "CreateRequestModel",
"type": "object",
"properties": {
"name": {
"type": "string"
},
"description": {
"type": "string"
},
"price": {
"type": "number"
}
},
"required": [
"name",
"description",
"price"
]
}

Create a file named "update_request.json" in the project folder and paste json in it. Defines the PUT request model. Sets required fields.

update_request.json

{
"definitions": {},
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "UpdateRequestModel",
"type": "object",
"properties": {
"name": {
"type": "string"
},
"description": {
"type": "string"
},
"price": {
"type": "number"
}
},
"required": [
"name",
"description",
"price"
]
}

Make deployment to AWS by using the commad line below.

serverless deploy
Enter fullscreen mode Exit fullscreen mode

After the deployment, in a simplest way, lambda functions and the API endpoints to make request functions is created.

After the deployment process, will write 5 API endpoints and 1 API Key on the console.

Deployment output

You can see the result by sending request to the endpoints. You need to add "x-api-key" to the Headers and for the value, write API Key showed in the console.

Let's look at what changed in AWS after the process.

Stack is created in the Cloud Formation.

CloudFormation

Our code is sent to S3.

S3

API Gateway is created for the endpoints.

API Gateway

Lambda functions are created.

Lambda

4. DynamoDB Integration

In this part, we will write the data coming over the API to DynamoDB. We will read the data from DynamoDB.

Install the required packages by using the command lines below.

npm i aws-sdk
npm i uuid
Enter fullscreen mode Exit fullscreen mode

Update the serverless.yml file like below.

serverless.yml v2

service: ads-api
custom:
tableName: AdsTable
provider:
name: aws
runtime: nodejs10.x
region: eu-central-1
profile: serverlessuser
apiKeys:
- AdsAPIKey
environment:
TABLE_NAME: ${self:custom.tableName}
iamRoleStatements:
- Effect: Allow
Action:
- dynamodb:Query
- dynamodb:Scan
- dynamodb:GetItem
- dynamodb:PutItem
- dynamodb:UpdateItem
- dynamodb:DeleteItem
Resource:
- "Fn::GetAtt": AdsDynamoDBTable.Arn
functions:
create:
handler: create.create
events:
- http:
path: api/ads
method: post
private: true
request:
schema:
application/json: ${file(create_request.json)}
update:
handler: update.update
events:
- http:
path: api/ads/{id}
method: put
private: true
request:
schema:
application/json: ${file(update_request.json)}
delete:
handler: delete.delete
events:
- http:
path: api/ads/{id}
method: delete
private: true
getAll:
handler: getAll.getAll
events:
- http:
path: api/ads
method: get
private: true
getById:
handler: getById.getById
events:
- http:
path: api/ads/{id}
method: get
private: true
resources:
Resources:
AdsDynamoDBTable:
Type: AWS::DynamoDB::Table
Properties:
TableName: ${self:custom.tableName}
ProvisionedThroughput:
ReadCapacityUnits: 1
WriteCapacityUnits: 1
AttributeDefinitions:
- AttributeName: id
AttributeType: S
KeySchema:
- AttributeName: id
KeyType: HASH
view raw serverless.yml hosted with ❤ by GitHub

Update the files of create.js, update. js, delete.js, getAll.js and getByld.js like below.

create.js v2

'use strict';
const aws = require('aws-sdk');
const uuidv1 = require('uuid/v1');
const dynamoDbClient = new aws.DynamoDB.DocumentClient();
module.exports.create = async event => {
const json = JSON.parse(event.body);
const params = {
TableName: process.env.TABLE_NAME,
Item: {
id: uuidv1(),
name: json.name,
description: json.description,
price: json.price
}
};
return new Promise((resolve, reject) => {
dynamoDbClient.put(params, function (err, data) {
if (err) {
console.log(err);
reject(err);
}
else {
const response = {
statusCode: 200,
body: null
};
resolve(response);
}
});
});
};
view raw create.js hosted with ❤ by GitHub

update.js v2

'use strict';
const aws = require('aws-sdk');
const dynamoDbClient = new aws.DynamoDB.DocumentClient();
module.exports.update = async event => {
const json = JSON.parse(event.body);
const params = {
TableName: process.env.TABLE_NAME,
Key: {
'id': event.pathParameters.id
},
UpdateExpression: 'set #namek = :namev, description = :description, price = :price',
ConditionExpression: 'id = :id',
ExpressionAttributeNames: {
'#namek': 'name'
},
ExpressionAttributeValues: {
':id': event.pathParameters.id,
':namev': json.name,
':description': json.description,
':price': json.price
}
};
return new Promise((resolve, reject) => {
dynamoDbClient.update(params, function (err, data) {
if (err) {
console.log(err);
if (err.code == 'ConditionalCheckFailedException') {
const response = {
statusCode: 404
};
resolve(response);
}
else {
reject(err);
}
}
else {
const response = {
statusCode: 200,
body: null
};
resolve(response);
}
});
});
};
view raw update.js hosted with ❤ by GitHub

delete.js v2

'use strict';
const aws = require('aws-sdk');
const dynamoDbClient = new aws.DynamoDB.DocumentClient();
module.exports.delete = async event => {
const params = {
TableName: process.env.TABLE_NAME,
Key: {
'id': event.pathParameters.id
},
ConditionExpression: 'id = :id',
ExpressionAttributeValues: {
':id': event.pathParameters.id
}
};
return new Promise((resolve, reject) => {
dynamoDbClient.delete(params, function (err, data) {
if (err) {
console.log(err);
if (err.code == 'ConditionalCheckFailedException') {
const response = {
statusCode: 404
};
resolve(response);
}
else {
reject(err);
}
}
else {
const response = {
statusCode: 200,
body: null
};
resolve(response);
}
});
});
};
view raw delete.js hosted with ❤ by GitHub

getAll.js v2

'use strict';
const aws = require('aws-sdk');
const dynamoDbClient = new aws.DynamoDB.DocumentClient();
module.exports.getAll = async event => {
const params = {
TableName: process.env.TABLE_NAME
};
return new Promise((resolve, reject) => {
dynamoDbClient.scan(params, function (err, data) {
if (err) {
console.log(err);
reject(err);
}
else {
const response = {
statusCode: 200,
body: JSON.stringify(data.Items)
};
resolve(response);
}
});
});
};
view raw getAll.js hosted with ❤ by GitHub

getById.js v2

'use strict';
const aws = require('aws-sdk');
const dynamoDbClient = new aws.DynamoDB.DocumentClient();
module.exports.getById = async event => {
var params = {
TableName: process.env.TABLE_NAME,
Key: {
'id': event.pathParameters.id
}
};
return new Promise((resolve, reject) => {
dynamoDbClient.get(params, function (err, data) {
if (err) {
console.log(err);
reject(err);
}
else {
if (data.Item) {
const response = {
statusCode: 200,
body: JSON.stringify(data.Item)
};
resolve(response);
}
else {
const response = {
statusCode: 404,
body: null
};
resolve(response);
}
}
});
});
};
view raw getById.js hosted with ❤ by GitHub

Make the second deployment to AWS with the command line below.

serverless deploy
Enter fullscreen mode Exit fullscreen mode

You can see the result by sending request to the endpoints. You need to add "x-api-key" to the Headers and for the value, write API Key showed in the console.

Let's look at what changed in AWS after the process.

A table is created in the DynamoDB.

DynamoDB

5. Redis Integration

In this last chapter, we will transfer the streams in the DynamoDB(adding, udating and deleting) to ElastiCache Redis and do reading from Redis in the GetAll request.

Install the required packages with the command lines below.

npm i redis
Enter fullscreen mode Exit fullscreen mode

Before starting to make update in the files, you need to learn the subnetlds by entering AWS console.

Subnets

Update the serverless.yml file like below. You need to update the subnetlds parts with your own information.

serverless.yml v3

service: ads-api
custom:
tableName: AdsTable
provider:
name: aws
runtime: nodejs10.x
region: eu-central-1
profile: serverlessuser
apiKeys:
- AdsAPIKey
environment:
TABLE_NAME: ${self:custom.tableName}
REDIS_ENDPOINT:
"Fn::GetAtt": AdsRedisCacheCluster.RedisEndpoint.Address
iamRoleStatements:
- Effect: Allow
Action:
- dynamodb:Query
- dynamodb:Scan
- dynamodb:GetItem
- dynamodb:PutItem
- dynamodb:UpdateItem
- dynamodb:DeleteItem
Resource:
- "Fn::GetAtt": AdsDynamoDBTable.Arn
- Effect: "Allow"
Action:
- ec2:CreateNetworkInterface
- ec2:DescribeNetworkInterfaces
- ec2:DeleteNetworkInterface
Resource: "*"
functions:
create:
handler: create.create
events:
- http:
path: api/ads
method: post
private: true
request:
schema:
application/json: ${file(create_request.json)}
update:
handler: update.update
events:
- http:
path: api/ads/{id}
method: put
private: true
request:
schema:
application/json: ${file(update_request.json)}
delete:
handler: delete.delete
events:
- http:
path: api/ads/{id}
method: delete
private: true
getAll:
handler: getAll.getAll
vpc:
securityGroupIds:
- "Fn::GetAtt": AdsRedisSecurityGroup.GroupId
subnetIds:
- subnet-5daeb810
- subnet-b01aceda
- subnet-b9a5a6c4
events:
- http:
path: api/ads
method: get
private: true
getById:
handler: getById.getById
events:
- http:
path: api/ads/{id}
method: get
private: true
dynamodbTrigger:
handler: trigger.trigger
vpc:
securityGroupIds:
- "Fn::GetAtt": AdsRedisSecurityGroup.GroupId
subnetIds:
- subnet-5daeb810
- subnet-b01aceda
- subnet-b9a5a6c4
events:
- stream:
type: dynamodb
batchSize: 1
startingPosition: LATEST
arn:
"Fn::GetAtt": AdsDynamoDBTable.StreamArn
resources:
Resources:
AdsDynamoDBTable:
Type: AWS::DynamoDB::Table
Properties:
TableName: ${self:custom.tableName}
StreamSpecification:
StreamViewType: NEW_AND_OLD_IMAGES
ProvisionedThroughput:
ReadCapacityUnits: 1
WriteCapacityUnits: 1
AttributeDefinitions:
- AttributeName: id
AttributeType: S
KeySchema:
- AttributeName: id
KeyType: HASH
AdsRedisSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: For Ads Redis Cluster
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 6379
ToPort: 6379
CidrIp: 0.0.0.0/0
AdsRedisCacheCluster:
DependsOn: AdsRedisSecurityGroup
Type: AWS::ElastiCache::CacheCluster
Properties:
CacheNodeType: cache.t2.micro
Engine: redis
ClusterName: ads-redis
NumCacheNodes: 1
VpcSecurityGroupIds:
- "Fn::GetAtt": AdsRedisSecurityGroup.GroupId
view raw serverless.yml hosted with ❤ by GitHub

Update the getAll.js file like below.

getAll.js v3

'use strict';
const redis = require('redis');
const redisClient = redis.createClient({ url: '//' + process.env.REDIS_ENDPOINT + ':6379' });
module.exports.getAll = async event => {
const promise = new Promise((resolve, reject) => {
redisClient.hgetall('Ads', function (err, data) {
if (err) {
console.log(err);
reject(err);
}
else {
var ads = [];
for (const key in data) {
if (data.hasOwnProperty(key)) {
const element = data[key];
ads.push(JSON.parse(element));
}
}
const response = {
statusCode: 200,
body: JSON.stringify(ads)
};
resolve(response);
}
});
});
return promise;
};
view raw getAll.js hosted with ❤ by GitHub

Create a new file named trigger.js.

trigger.js

'use strict';
const redis = require('redis');
const redisClient = redis.createClient({ url: '//' + process.env.REDIS_ENDPOINT + ':6379' });
module.exports.trigger = async event => {
const promise = new Promise((resolve, reject) => {
event.Records.forEach(record => {
if (record.eventName == 'INSERT' || record.eventName == 'MODIFY') {
const ad = {
id: record.dynamodb.NewImage.id.S || record.dynamodb.OldImage.id.S,
name: record.dynamodb.NewImage.name.S,
description: record.dynamodb.NewImage.description.S,
price: record.dynamodb.NewImage.price.N
};
const json = JSON.stringify(ad);
redisClient.hset('Ads', ad.id, json, redisClient.print);
}
else if (record.eventName == 'REMOVE') {
redisClient.hdel('Ads', record.dynamodb.OldImage.id.S, "field", redisClient.print);
}
});
resolve(`Successfully processed ${event.Records.length} records.`);
});
return promise;
};
view raw trigger.js hosted with ❤ by GitHub

Make the third and last deployment to AWS with the command line below.

serverless deploy
Enter fullscreen mode Exit fullscreen mode

You can see the result by sending request to the endpoints. You need to add "x-api-key" to the Headers and for the value, write API Key showed in the console.

Let's look at what changed in AWS after the process.

ElasticCache is created.

ElasticCache Redis

Trigger is assigned to the DynamoDB.

DynamoDB Trigger

You can access the final version of the project from Github.

GitHub logo ahmetkucukoglu / serverless-restful

Serverless RESTful on AWS

serverless-restful

Serverless RESTful on AWS

Good luck.

Do your career a big favor. Join DEV. (The website you're on right now)

It takes one minute, it's free, and is worth it for your career.

Get started

Community matters

Top comments (0)