Event-driven architecture provides developers with increased availability, elasticity, scalability, and cost optimization. With the rise of microservices and the cloud, event-driven architecture is becoming increasingly important for developers to understand.
This blog explains what serverless architecture is, when it should be used, and how to use AWS serverless services like API Gateway, Lambda, SQS, SNS, and SES.
Q.What do you mean by Serverless?
Deploying an application usually requires a server infrastructure. This means either a virtual or physical server needs to be managed, along with its os and prerequisites. Using a virtual server from a cloud provider such as Amazon does eliminate physical constraints, although it still requires some level of customization for the operating system and its software processes.
The serverless architecture is a way to build and run applications and services without having to manage the infrastructure behind it. With a serverless architecture, your application still runs on servers, of course, but the server management is done by AWS. So you focus purely on the individual functions in your application code.
AWS defines its serverless services as “the native architecture of the cloud that enables you to shift more of your operational responsibilities to AWS, increasing your agility and innovation.” The diagram below presents the serverless architecture you can deploy in minutes using the solution's implementation guide.
It is also known as Function-as-a-Service (FaaS) is a serverless way to execute modular pieces of code, which can be executed in response to an event, such as a user clicking on an element in an application. This makes it easy to develop and deploy microservices with ease.
As each function can be scaled automatically which is a very cost-effective way of paying for computing resources. You only pay for the times that your functions get called, rather than paying to have your application always on and waiting for requests on so many different instances.
Q.When and why should you consider serverless ?
You should consider using a serverless provider if you have a small number of functions that you need to be deployed. It may make more sense to migrate small pieces of the application into serverless functions over time, as migrating an entire application might be very complex with system architecture.
- Cost-effective, a cost is based on the CPU Usage time.
- Simplified scalability, as the vendor handles all of the scaling on demand.
- Event driven, so no need for a complicated deploy process to roll out bug fixes and new features, developers can add and modify code for particular events.
A Few of AWS Serverless services:
A.Amazon API Gateway: The AWS API Gateway is a managed service for building, deploying, and scaling various types of APIs. An API Gateway can remove some overhead in managing multiple APIs by monitoring, logging, throttling, caching, and other infrastructural requirements.
With a few clicks in the AWS Management Console, you can create REST and Web Socket APIs that act as a front door for your applications to access data, business logic, or other AWS services from your back-end services. Some of its other benefits are,
- You can run multiple versions of the same API simultaneously with API Gateway.
- You can monitor performance of metrics and information - on API calls, data latency, error rates - from the API Gateway dashboard.
- You can decrease your costs based on the number of API requests you make, per Region, across your AWS accounts.
- You can authorize access to your APIs with AWS Identity and Access Management, or IAM, as well as Amazon Cognito.
For more information regarding the implementation of Amazon API Gateway.
B.AWS Lambda: AWS Lambda is a serverless compute service that has been designed to allow you to run your application code without having to manage and provision your own EC2 instances. This saves you from having to maintain and administer an additional layer of technical responsibility within your solution. Instead, that responsibility is passed over to AWS to manage for you.
Just upload your code, and Lambda takes care of everything required to run and scale your code with high availability and cost-effectiveness as, you pay only for the requests served and the compute time required to run your code for each of those requests.
A Lambda function on its own doesn't do much. It needs to be executed. We call that triggers. There are many AWS services that can trigger a Lambda function. When your Lambda function gets executed, Lambda needs to know what piece of your code it needs to execute. That is the entry point for your code(handler). This tells Lambda in which file or class the function to execute is, as well as its name.
Lambda will use that handler to execute your Lambda from the trigger is received, but your code probably needs to receive some more information about what just triggered it. That information is passed to the handler via what we call an "event payload."
C.Simple Queue Service (SQS) is a fully managed service offered by AWS that works with serverless systems, or any distributed architecture.
It's simply a queueing service for messages between components, It has the capability of sending, storing, and receiving these messages at scale. It is also possible to configure the service using the AWS Management Console, the AWS CLI, or using the AWS SDKs.
- The producer component of your architecture is responsible for sending messages to your queue.
- The SQS service stores the message across a number of SQS servers for resiliency within the specified queue.
- Consumers are responsible for processing the messages within your queue.
When the consumer element of your architecture is ready to process the message from the queue, the message is retrieved and is then marked as being processed by activating the visibility timeout on the message. This timeout ensures that the same message will not be read and processed by another consumer. When the message has been processed, the consumer then deletes the message from the queue. During this period, the consumer processes the message. If it fails to process a message, perhaps due to a communication error, the consumer will not send a delete message request back to SQS. As a result, if the visibility timeout expires and it doesn't receive the request to delete the message, the message will become available again in the queue for other consumers to process. There are two queues which can be used,
- Standard queues, you have unlimited throughput, at-least-once delivery, and best-effort ordering.
- First-in, first-out queues, you have high throughput, first-in, first-out delivery, and exactly-once processing.
A dead-letter queue differs from the standard and FIFA queues as this dead-letter queue is not used as a source queue to hold messages submitted by producers. Instead, the dead-letter queue is used by the source queue to send messages that fail to process for one reason or another. A dead-letter queue must be configured as the same queue type as the source is used against.
D. Simple Notification Service (SNS) is used as a publishing and subscribe messaging service. It is centered around topics (group for collecting messages).
- Users over endpoints can then subscribe to this topic and messages or events are then published to that particular topic.
- When a message is published, all subscribers to that topic receive a notification of that message.
SNS is a managed service and highly scalable, allowing you to distribute messages automatically to all subscribers across your environment, including mobile devices. It can be configured with the AWS management console, the CLI, or the AWS SDK.
SNS uses the concept of publishers and subscribers, similar to SQS.
- The producers, or publishers, send messages to a topic, which is used as a central communication control point.
- The consumers, or subscribers of the topic, are then notified of this message by HTTP, HTTPS, Email, Email-JSON, etc ... .
- SNS offers methods of controlling specific access to your topics through a topic policy. The policy themselves follow the same format as IAM policies.
By integrating SNS and SQS with each other, together, a solution can be designed to send messages to subscribers through a push method. To do this, you'll need to have your SQS queue subscribed to the SNS topic. Integration with AWS Lambda allows SNS notifications to invoke existing Lambda functions. Like SQS, the Lambda function has to be subscribed to the topic.
For more information on SNS,
- https://aws.amazon.com/blogs/mobile/invoking-aws-lambda-functions-via-amazon-sns/
- https://aws.amazon.com/blogs/compute/simplify-pubsub-messaging-with-amazon-sns-message-filtering/
E. Simple Email Service (SES) makes it possible to handle an automated email system to communicate with your customers.
- When receiving emails, you can architect your application to respond automatically to incoming emails, such as a request to unsubscribe from newsletter.
- When sending emails, you can architect your application to emails automatically for an event, such as email confirmation.
AWS will automatically scan for spam and viruses, and reject any messages from untrusted sources. When receiving an email, there are two ways to configure SES to instruct it as to what to do with the email.
- Recipient-based control, you set up configurations to direct the email based on the recipient. These recipient lists are classed as conditions. Receipt rules are used to control what action is taken when a condition is met.
- IP address-based control defines what happens to your email based on its source IP address. This method allows you to select if email should be accepted or rejected based on its originating IP address.
For more information on event-driven architecture,
Top comments (1)
Great write-up. We also recently published an article on how to bridge Backend and Data Engineering teams using Event Driven Architecture - packagemain.tech/p/bridging-backen...