Alarm and monitoring systems are a key part of mature products and applications. If you worry about your customers, it’s better to be notified when something goes wrong then be in the dark. If you host your infrastructure on AWS, the only one way to get the full metrics of your used services is to use CloudWatch. AWS CloudWatch doesn’t only give you access to metrics, however, it also creates alarms for specific cases. But, if you worked with AWS before you no doubt noticed that the interface and certain functions are a far cry from “user friendly,” which means that sending a notification with alarms to Slack not a trivial task.
This article reveals the secrets you should know to set up the alarms you need in five minutes. For those of you who would like to see all in action, please watch our video tutorial.
There are a few steps we will need to go through to make this happen:
- Create an AWS Access Key and Secret
- Create a Role
- Deploy the Lambda function
- Create an SNS topic and subscription
- Create a Cloudwatch Alarm
1. Create AWS Access Key and Secret
If you already have your AWS Access Key and Secret, you can skip this step. Otherwise, follow this link. Choose your user, then go to the Security Credentials tab and click Create Access Key.
Copy the AWS Access Key and Secret; we will use them during configuring deployment of Lambda Function.
Create a Role
The Role will be used by your Lambda function and requires permission to do certain things.
Follow this link then click Create Role.
Chose Lambda from the list.
Find AWSLambdaBasicExecutionRole and select it.
Finally, type a role name, such as “cloudwatch-to-slack-role,” click on the newly created role in the list and copy the ARN name. We will use it in step 3 during deployment of the Lambda function.
3. Deploy the Lambda function
To deploy the AWS Lambda function, you need to clone the repository and have Node.js installed on your local machine. Especially for that guide, we contributed significantly in the repository to get a better configuration process.
git clone git@github.com:assertible/lambda-cloudwatch-slack.git
cd lambda-cloudwatch-slack
cp .env.example .env
Open your .env file and fill in the environment variables.
UNENCRYPTED_HOOK_URL
That variable you should fill with the Slack Incoming Webhook; create a new one for CloudWatch on that page for your organization
Follow the practice of using one hook for one specific service. Once you create a webhook, choose a channel, then post notifications, create a username(the title for messages) and, of course, add a nice picture. Download the Cloudwatch logo from the link here
AWS_FUNCTION_NAME
The name of your function which will be declared in your AWS Lambda function page. Let’s call it “cloudwatch-to-slack.”
AWS_REGION=eu-west-1
Chose the region of the Lambda function.
AWS_ROLE="arn:aws:iam::123456789123:role/lambda_exec_role"
Copy the arn id from the second step here.
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
Fill in the AWS Access Key and Secret with your keys from the first step.
Now we’re ready to deploy; simply execute the following command in your terminal from the folder with the repository we cloned:
npm install
npm run deploy
If everything went well, you will be able to find your fresh installed function on the Lambda page in AWS.
4. Create an SNS topic and subscription
Browse in AWS to the Simple Notification Service. Ironically, this service is anything but simple, especially the terminology they use and steps you have to go through.
Go to Topics and click Create New Topic. I chose the name “cloudwatch-alarms” for the notification that will be sent.
Open the created topic and create a subscription.
Select AWS Lambda as your Protocol and pick the endpoint with the name of your function.
That’s it; now we are ready to create an alarm.
5. Create Cloudwatch Alarm
Navigate through Cloudwatch to your AWS account and click Create Alarm.
Chose the metric you would like to monitor, define the alarm configuration and select your SNS topic name in the Send Notification To section.
I recommend that you specifically define conditions that will immediately throw alarms for the purposes of testing your function. If everything is configured properly, you will see the message in your Slack channel.
I hope this guide helps you save time and set up alarms for your current or future projects.
Additional notes:
By default, AWS Lambda has blueprints that allow you to do different things with a simple configuration and, in those blueprints, you may even find a CloudWatch-to-Slack function. Unfortunately, this send a very poor, unstructured data to the channel.
Top comments (10)
Hi Alex, thanks for sharing. We took the same approach to a hackathon a few years ago and even built a product out of it: marbot receives CloudWatch alarms and all other kinds of notifications from your AWS infrastructure and escalates them among your team.
I'm not surprised :) with whole complexity and irrationality of AWS there are plenty option to build own startups on top of it
@alex Thanks for the writing. I'm facing issue while deploying.
'User: arn:aws:iam::273191640429:user/cloudwatch-to-slack-user is not authorized to perform: iam:PassRole on resource: arn:aws:iam::273191640429:role/cloudwatch-to-slack-role'I know its a permission issue. But due to limited knowledge on AWS couldn't able to solve it by meself.
And let know which are all other permissions needed.
I have created an IAM user with administrative permissions. And works good now.
Took me a while to get the permissions set on the IAM user. Now I am stuck. I have verified that I have npm v12.16.1.
I get this error after the message "Uploading zip file to AWS Lambda us-east-1 with parameters:"
The runtime parameter of nodejs8.10 is no longer supported for creating or updating AWS Lambda functions. We recommend you use the new runtime (nodejs12.x) while creating or updating functions.
=> Retrying
InvalidParameterValueException: The runtime parameter of nodejs8.10 is no longer supported for creating or updating AWS Lambda functions. We recommend you use the new runtime (nodejs12.x) while creating or updating functions.
at Object.extractError (/AWSCloudWatchToSlackLamba/lambda-cloudwatch-slack/node_modules/aws-sdk/lib/protocol/json.js:48:27)
at Request.extractError (/AWSCloudWatchToSlackLamba/lambda-cloudwatch-slack/node_modules/aws-sdk/lib/protocol/rest_json.js:52:8)
at Request.callListeners (/AWSCloudWatchToSlackLamba/lambda-cloudwatch-slack/node_modules/aws-sdk/lib/sequential_executor.js:109:20)
at Request.emit (/VSCode/AWSCloudWatchToSlackLamba/lambda-cloudwatch-slack/node_modules/aws-sdk/lib/sequential_executor.js:81:10)
at Request.emit (/AWSCloudWatchToSlackLamba/lambda-cloudwatch-slack/node_modules/aws-sdk/lib/request.js:683:14)
at Request.transition (/AWSCloudWatchToSlackLamba/lambda-cloudwatch-slack/node_modules/aws-sdk/lib/request.js:22:10)
at AcceptorStateMachine.runTo (/AWSCloudWatchToSlackLamba/lambda-cloudwatch-slack/node_modules/aws-sdk/lib/state_machine.js:14:12)
at /AWSCloudWatchToSlackLamba/lambda-cloudwatch-slack/node_modules/aws-sdk/lib/state_machine.js:26:10
at Request. (/AWSCloudWatchToSlackLamba/lambda-cloudwatch-slack/node_modules/aws-sdk/lib/request.js:38:9)
at Request. (/AWSCloudWatchToSlackLamba/lambda-cloudwatch-slack/node_modules/aws-sdk/lib/request.js:685:12) {
message: 'The runtime parameter of nodejs8.10 is no longer supported for creating or updating AWS Lambda functions. We recommend you use the new runtime (nodejs12.x) while creating or updating functions.',
code: 'InvalidParameterValueException',
time: 2020-03-22T16:21:06.899Z,
requestId: 'bc115ac4-f63b-4d39-840f-87222e751928',
statusCode: 400,
retryable: false,
retryDelay: 22.01625183426701
}
This happened to me too.
To fix it upgrade the dependencies in the package.json file
I upgraded to these versions and it worked.
"dependencies": {
"aws-sdk": "^2.673.0",
"https": "^1.0.0",
"lodash": "^4.17.15",
"url": "^0.11.0"
},
"devDependencies": {
"node-lambda": "0.16.0"
}
Hello, I have a question, I have seen the code, in this case, in the Cloudwatch function, I would like to pull the values of Dimensions seen in Cloudwatch, how would that form be? Because I want to specify the Ec2 server ID
Thank you for the post. I was wondering if there was a way to do this with nicely formatted messages in Slack. #DevChatOps
I'm just wondering that maybe one day AWS will decide to make it more "user friendly", because now it's really a lot of actions required to do a one simple thing.
Hi Alex
Can we use this code to create an alarm when CPU utilization is greater than a certain value and when it happen, then send the slack notification