DEV Community

Cover image for A crash course on Serverless with AWS - Building APIs with Lambda and Aurora Serverless

A crash course on Serverless with AWS - Building APIs with Lambda and Aurora Serverless

Adnan Rahić on November 27, 2018

Ever since AWS made a serverless option of AWS RDS Aurora generally available, I've been on the edge of my seat, eagerly anticipating all the vario...
Read full post
Collapse
 
bryangerre profile image
BryanGerre •

Hey Adnan. Thank you so much for taking the time to develop this example. How would you introduce another model to this solution? Set's say I wanted to add UsersModel. I've created the users.js within the models directory. the db.js you have only passes in one model ( const Models = { Note }) could that be an array of models? const Models = [Note, User, etc]? If so how does the handler.js know which model?

Collapse
 
adnanrahic profile image
Adnan Rahić •

Add another model to the object.
In db.js:

const Models = { Note, User }

Once you want to require the models in the handler.js you can choose which one you want.

const { Note } = await connectToDatabase()
// or
const { User } = await connectToDatabase()
// or even
const { Note, User } = await connectToDatabase()

It's up to you to use what you like more. I personally think destructuring is freaking awesome. :)

Collapse
 
bryangerre profile image
BryanGerre •

Thanks so much, I'll try the above. thanks again

Collapse
 
christiams profile image
Christian Carrillo •

What do you mean with destructuring?

Thread Thread
 
kayis profile image
K •

Object destructuring

Collapse
 
bryangerre profile image
BryanGerre • • Edited

Last question, what would be the best way to associate those two tables? I've found many examples, but it would require a refactoring of the model's .js file.

Collapse
 
agstevens profile image
andrewstevens •

To install mysql locally on a mac (mojave):
brew install mysql@5.7 to install an older version of mysql. Then everything works fine locally.

To answer Weiren's question:
in the secrets.json file:
"DB_HOST": "superdupercluster.cluster-.rds.amazonaws.com"

Fixes the connection refused issue you are seeing, as 127.0.0.1 works for the local machine, but not for whatever machine that the Lambda function runs on.

However, now I am getting the same issue as sobhanthankur:

{
"errorType": "SequelizeConnectionError",
"errorMessage": "Unknown database 'test'",
"trace": [
"SequelizeConnectionError: Unknown database 'test'",
" at Promise.tap.then.catch.err (/var/task/node_modules/sequelize/lib/dialects/mysql/connection-manager.js:133:19)",
" at tryCatcher (/var/task/node_modules/bluebird/js/release/util.js:16:23)",
" at Promise._settlePromiseFromHandler (/var/task/node_modules/bluebird/js/release/promise.js:517:31)",
" at Promise._settlePromise (/var/task/node_modules/bluebird/js/release/promise.js:574:18)",
" at Promise._settlePromise0 (/var/task/node_modules/bluebird/js/release/promise.js:619:10)",
" at Promise._settlePromises (/var/task/node_modules/bluebird/js/release/promise.js:695:18)",
" at _drainQueueStep (/var/task/node_modules/bluebird/js/release/async.js:138:12)",
" at _drainQueue (/var/task/node_modules/bluebird/js/release/async.js:131:9)",
" at Async._drainQueues (/var/task/node_modules/bluebird/js/release/async.js:147:5)",
" at Immediate.Async.drainQueues as _onImmediate",
" at runCallback (timers.js:705:18)",
" at tryOnImmediate (timers.js:676:5)",
" at processImmediate (timers.js:658:5)"
]
}

Do we need to initialize the Aurora database and create the database named 'test' first, and then Sequelize will automatically create the notes table in that database for us? Or is there a missing permission for the lambda role/policy?

Collapse
 
adnanrahic profile image
Adnan Rahić •

Yes, you need to create it manually. I mentioned it in the article:

Note: If you want to access the AWS Aurora Serverless cluster follow this tutorial to learn how to do it with the CLI, or this official tutorial by AWS to connect through Cloud9. You will need to connect to the cluster and create a database manually because Sequelize can't create them for you.

I've yet to find the free time to explore the Lambda role/policy to let Sequelize create it automatically. But, I'd ideally not want my Lambda role to have access to create/delete/edit databases. It could end badly. Only letting them change the data is a bit more safe for my liking.

Collapse
 
kra007027 profile image
@kra007027 •

Hello.
Did you solve the issue?
Could you help me to solve this issue?

Collapse
 
wei0121 profile image
Weiren • • Edited

Hi Adnan, Thank you so much with your wonderful article. I did exactly same as it till doing offline test. I got an issue which it can not connect to the Aurora DB. The things I have tried is :

  • check the error from Sequelize. I got an SequelizeConnectionRefusedError with "ECONNREFUSED"
  • deploy to aws. ( same issue can not connec to db )

What I missed during the process that cause the issue?
In the secrets.json file, What should be changed according to my RDS configuration ( especially DB_HOST and DB_NAME)

Collapse
 
jmhorn profile image
John Horn •

Had the same exact issue and as someone with a mac with no mysql installed this was the solution for me. Remember a local instance of mysql needs to still be available.

Install mysql locally

brew install mysql

Start MySQL

mysql.server start OR service mysqld start

Make change to user auth

Per stack overflow need to edit password auth

mysql -u root
ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY 'password'

mysql -u root -p
"password"

Make sure to change the db.js DB_PASSWORD

db.js
"DB_PASSWORD": "password"

and voila it worked for me

Collapse
 
softnayr profile image
Ryed Ban • • Edited

Hi Adnan,

Great work buddy!

I just got one question - how would you associate another model to the Note model?
Let's say we have a User model and we would like to implement Note.belongsTo(User).

I have made something like this but it keeps me getting an error like "note is not associated to user"

const User = require('./User');
module.exports = (sequelize, type) => {
const Note = sequelize.define('note', {
id: {
type: type.INTEGER,
primaryKey: true,
autoIncrement: true
},
title: type.STRING,
description: type.STRING
})

Note.associate = (models) => {
    Note.belongsTo(models.User, {
        foreignKey: 'user_id',
        as: user
    })
}

return Note
Enter fullscreen mode Exit fullscreen mode

}

Collapse
 
lilhamad profile image
Lilhamad Adewale •

Hi @softnayr were you able to solve the "note is not associated to user" issue

Collapse
 
hzburki profile image
Haseeb Burki • • Edited

Hey Adnan. Can you please explain a little how database connections in serverless framework work? More specifically how many database connections would your code create if you call the same endpoint multiple time and if you call the same endpoint from two different source simultaneously.

I have a serverless project integrated with express and sequelize, such that a single lambda has multiple routes. The F.E calls 6 api end points for different graphs simultaneously. My code create 6 connections instead of one, and the CPU utilization on my RDS instance goes up to 60-80% :| ...

Collapse
 
adnanrahic profile image
Adnan Rahić •

This is a cool question. The keyword here is simultaneously. This has nothing to do with the Serverless Framework, and everything to do with the way AWS Lambda works under the hood.

When your front end sends 6 concurrent requests to a Lambda function, it will spawn 6 instances and every instance will handle one request. If you were to send 6 requests sequentially, one instance would be enough.

Because, in this case, you have 6 Lambda function instances, every one of those will create a database connection. Makes sense?

The way to handle this is to have one Lambda function dedicated to only database interaction, while the other act as proxies. This is very hard to configure. I'd suggest you use a serverless database that can scale as easily as Lambda. :)

Very cool question, feel free to hit me up through the chat here in dev.to if you have any more topics like these. I'd love to nerd out.

Collapse
 
sobhanthakur profile image
sobhanthakur •

Can somebody please help me out with this error

{
"errorType": "SequelizeConnectionError",
"errorMessage": "Unknown database 'test_aurora'",
"trace": [
"SequelizeConnectionError: Unknown database 'test_aurora'",
" at Promise.tap.then.catch.err (/var/task/node_modules/sequelize/lib/dialects/mysql/connection-manager.js:133:19)",
" at tryCatcher (/var/task/node_modules/bluebird/js/release/util.js:16:23)",
" at Promise._settlePromiseFromHandler (/var/task/node_modules/bluebird/js/release/promise.js:517:31)",
" at Promise._settlePromise (/var/task/node_modules/bluebird/js/release/promise.js:574:18)",
" at Promise._settlePromise0 (/var/task/node_modules/bluebird/js/release/promise.js:619:10)",
" at Promise._settlePromises (/var/task/node_modules/bluebird/js/release/promise.js:695:18)",
" at _drainQueueStep (/var/task/node_modules/bluebird/js/release/async.js:138:12)",
" at _drainQueue (/var/task/node_modules/bluebird/js/release/async.js:131:9)",
" at Async._drainQueues (/var/task/node_modules/bluebird/js/release/async.js:147:5)",
" at Immediate.Async.drainQueues as _onImmediate",
" at runCallback (timers.js:705:18)",
" at tryOnImmediate (timers.js:676:5)",
" at processImmediate (timers.js:658:5)"
]
}

Collapse
 
adnanrahic profile image
Adnan Rahić •

I answered this question here:

adnanrahic profile image Adnan Rahić twitter github
Jul 25

Yes, you need to create it manually. I mentioned it in the article:

Note: If you want to access the AWS Aurora Serverless cluster follow this tutorial to learn how to do it with the CLI, or this official tutorial by AWS to connect through Cloud9. You will need to connect to the cluster and create a database manually because Sequelize can't create them for you.

I've yet to find the free time to explore the Lambda role/policy to let Sequelize create it automatically. But, I'd ideally not want my Lambda role to have access to create/delete/edit databases. It could end badly. Only letting them change the data is a bit more safe for my liking.

You need to manually connect to the Aurora cluster and create a database.

Collapse
 
bryangerre profile image
BryanGerre •

Hey Adnan, I'm trying to query the database using

const user = await User.findAll({
where: {
user_email: event.pathParameters.user_email
}

The event.pathParameters.user_email has a %40 instead of the @ symbol.

any suggestions?

Collapse
 
imhashir profile image
Hashir Baig •

Hi Adnan!
Thanks for this amazing guide.
I wanted to ask if, performance-wise, it is fine to require "db" into every lambda function?
As far as I am aware of Sequelize (and my knowledge is limited) whenever we run an express app having models created using Sequelize, it first checks if all tables exist, and if they don't it creates them first.
Isn't it overhead in every lambda execution?
Let me know if I am mistaken.
Thank You again for this awesome post :)

Collapse
 
virendrasharma1 profile image
Virendra Sharma •

Hello Adnan, I am getting this error while connecting to database!

Serverless: Failure: connect ETIMEDOUT
SequelizeConnectionError: connect ETIMEDOUT
at Promise.tap.then.catch.err (D:\litmus-workspace\litmus-serverless\node_modules\sequelize\lib\dialects\mysql\connection-manager.js:133:19)
at tryCatcher (D:\litmus-workspace\litmus-serverless\node_modules\bluebird\js\release\util.js:16:23)
at Promise._settlePromiseFromHandler (D:\litmus-workspace\litmus-serverless\node_modules\bluebird\js\release\promise.js:517:31)
at Promise._settlePromise (D:\litmus-workspace\litmus-serverless\node_modules\bluebird\js\release\promise.js:574:18)
at Promise._settlePromise0 (D:\litmus-workspace\litmus-serverless\node_modules\bluebird\js\release\promise.js:619:10)
at Promise._settlePromises (D:\litmus-workspace\litmus-serverless\node_modules\bluebird\js\release\promise.js:695:18)
at _drainQueueStep (D:\litmus-workspace\litmus-serverless\node_modules\bluebird\js\release\async.js:138:12)
at _drainQueue (D:\litmus-workspace\litmus-serverless\node_modules\bluebird\js\release\async.js:131:9)
at Async._drainQueues (D:\litmus-workspace\litmus-serverless\node_modules\bluebird\js\release\async.js:147:5)
at Immediate.Async.drainQueues as _onImmediate
at runCallback (timers.js:705:18)
at tryOnImmediate (timers.js:676:5)
at processImmediate (timers.js:658:5)
at process.topLevelDomainCallback (domain.js:120:23)

I have exactly followed your steps but I am getting this error. My database inbound traffic is set to all but still I am getting this error. I was able to connect to my database through SSH tunneling.

Collapse
 
prajapatirohanp profile image
prajapatirohanp •

Hello Virendra,

Good morning.

Have you resolved this issue? Please let me know.

Thanks,
Rohan.

Collapse
 
realpchauhan profile image
Pradeep Chauhan •

I need a boilerplate with AWS Node.js Lambda, AWS Cognito, JWT, MongoDB, Cloudformation for user authentication and authorization
dev.to/realpchauhan/user-authentic...

Collapse
 
fwahlqvist profile image
Fredrik Wahlqvist •

Hey Adnan,
Great tutorial!!
Just wondered if you have any experience with that Lambda consistently timeout, although i get a db connection its keeps timing out and never returns anything? (input from other are of-course welcome to :) )

Collapse
 
dvddpl profile image
Davide de Paolis •

Hi Adnan. thanx for sharing. it's very useful. at work we recently ported an app from php to react + cognito + gateway + lambda + vpc to connect to a Vertica DB and just started moving to Aurora Serverless.

can i ask why you are add in msql2 in the package.json? i couldn't see it used in the code. maybe is a necessary peer dependency for Sequelize as soon as you define the dialect you want to use?

Collapse
 
appupcloud profile image
appup-cloud • • Edited

Hello Adnan,

Thank you for this article which has given me a really great start.

The steps which you wrote worked really good until I tried to connect publish SNS message to a topic

After little research I figured out that as we are putting the lambda in Aurora's serverless VPC it is not allowing me connect to any internet facing external services.

Can you suggest me a solution ?

Cheers

Collapse
 
parampal profile image
parampal •

What do you think is the best way to add parameter validation for POST/PUT calls in this example? Also is there any testing library you would recommend?

Thanks for the great tutorial!

Collapse
 
pepemiso16 profile image
pepemiso16 •

After deploying and test the endpoints, the response was 502 Bad Gateway. How can we be able to address this one?

Collapse
 
tommedema profile image
Tom Medema •

Is there a reason why you don't create the aurora database with cloudformation?