DEV Community: Akhilesh Yadav

AWS CDK - Building Telegram bot with AWS Lambda and API Gateway Proxy Integration - Part 2

Akhilesh Yadav — Sun, 20 Feb 2022 13:18:30 +0000

Welcome to part 2 of this series. This would be the final series of AWS CDK - Building Telegram bot with AWS Lambda and API Gateway Proxy Integration . Regret for posting this lately.

You can find the sourcecode on Git repository by following below link. Checkout branch part2
https://github.com/arki7n/aws-cdk-telegram-bot-example.git

Commands for local usage:

git clone https://github.com/arki7n/aws-cdk-telegram-bot-example.git
git checkout part2

So let me start about all the update. Incase you have the main branch of the repository, you can follow changes and execute one by one.

Let's update few packages to make sure existing code doesn't break with AWS CDK feature updates.

npm install @aws-cdk/core@1.137
npm install -g aws-cdk

Setting up a new Telegram Bot

Head to your Web browser.
Step 1: Open below link and signin in with your telegram account.

https://web.telegram.org/k/

Step 2: After successful login, search for "BotFather" bot in Telegram search bar.

Step 3: Type /help and then bot would reply with its menu. Click on /newbotto begin setting up new bot.

Step 4: I will be creating up a bot with the intention of saving bookmark links or texts to some database. And then be able to view the database.

Step 5: Save the API KEY token which would be required to access telegram APIs.

Example Telegram Bot: https://t.me/BookmarkmeBot

Telegram API Resources

There are 2 ways to setup bot with Telegram and hook our custom lambda function with it.

Using HTTP long-polling on Telegram API: Active Server driven and could be expensive to keep the server running and poll for new user messages to our bot.
Webhook: As soon as the bot receives new message, Telegram Server sends the message to our custom HTTP URL using POST method. We would use API Gateway URL and lambda function would do the rest of the work of processing data and sending response back to the telegram user.

Setting up new API Gateway URL path for Webhook in AWS CDK code.

There are some new addition in the earlier code of Part 1.

Used npm 'path' package to get relevant directory of lambda function.
Added dynamic description for Lambda to always upload code on new deployments irrespective of any changes in code.
Added Lambda Versioning to track changes in AWS Lambda UI Console.
Enabling CORS at API Gateway side to let telegram server push Webhook message without getting blocked (403 Forbidden Error).
New Resource path /bot/webhookadded with POST method integration with lambda. Keeping /bot path for manual health check and see new lambda version information is available.
Output URL of API URL and cloudwatch log UI.

Find the code below for aws-cdk-telegram-bot-example\cdk-tool\lib\cdk-tool-stack.js file. Make sure to replace the BOT_TOKEN with yours.

const cdk = require("@aws-cdk/core");
const lambda = require("@aws-cdk/aws-lambda");
const apigw = require("@aws-cdk/aws-apigateway");
const path = require('path');
const BOT_TOKEN = '5118686429:AAHtgBvYLyrTSIUJ-iNRmV5MiuTYcSfAXIYeysdf'; // PASTE Telegram API BOT TOKEN here

class CdkToolStack extends cdk.Stack {
  /**
   *
   * @param {cdk.Construct} scope
   * @param {string} id
   * @param {cdk.StackProps=} props
   */
  constructor(scope, id, props) {
    super(scope, id, props);

    // All constructs take these same three arguments : scope, id/name, props
    const lambdaTelegram = new lambda.Function(this, "telegramBotHandler", {
      runtime: lambda.Runtime.NODEJS_14_X,
      handler: "index.handler",
      code: lambda.Code.fromAsset(path.join(__dirname, '../../assets/lambda/telegram-bot')), // Get relevant path to lambda directory.
      architecture: lambda.Architecture.ARM_64,
      environment: {
        'CURRENT_ENV': 'dev',
        'BOT_TOKEN': BOT_TOKEN
      },
      description: `Generated on: ${new Date().toISOString()}`  // added to keep pushing latest code on AWS lambda on each deployment.
    });

    /*Versioning every new changes and keeping track of it. Check AWS Lambda UI Console*/
    const version = new lambda.Version(this, 'Ver'+new Date().toISOString(), {
      lambda: lambdaTelegram,
    });

    // All constructs take these same three arguments : scope, id/name, props
    // defines an API Gateway REST API resource backed by our "telegrambot-api" function.
    const restApi = new apigw.RestApi(this, "telegrambot-api", { 
        deploy: false,
        defaultCorsPreflightOptions: { // Enable CORS policy to allow from any origin. Customize as needed.
          allowHeaders: [
            'Content-Type',
            'X-Amz-Date',
            'Authorization',
            'X-Api-Key',
          ],
          allowMethods: ['OPTIONS', 'GET', 'POST', 'PUT', 'PATCH', 'DELETE'],
          allowCredentials: false,
          allowOrigins: apigw.Cors.ALL_ORIGINS,
        }
    });

    // Let's keep this as it as and use it for normal 'Hello World' Response with GET method integration with lamhda.
    restApi.root
      .addResource("bot")
      .addMethod("GET", new apigw.LambdaIntegration(lambdaTelegram, { proxy: true }));

    // Lets add nested resource under /bot resource path and attach a POST method with same Lambda integration.
    restApi.root
      .getResource("bot")
      .addResource("webhook")
      .addMethod("POST", new apigw.LambdaIntegration(lambdaTelegram, { proxy: true }));

    // All constructs take these same three arguments : scope, id/name, props
    const devDeploy = new apigw.Deployment(this, "dev-deployment", { api: restApi });

    // All constructs take these same three arguments : scope, id/name, props
    const devStage = new apigw.Stage(this, "devStage", {
      deployment: devDeploy,
      stageName: 'dev' // If not passed, by default it will be 'prod'
    });

    // All constructs take these same three arguments : scope, id/name, props
    new cdk.CfnOutput(this, "BotURL", {
      value: `https://${restApi.restApiId}.execute-api.${this.region}.amazonaws.com/dev/bot`,
    });

    new cdk.CfnOutput(this, "BotWebhookUrl", {
      value: `https://${restApi.restApiId}.execute-api.${this.region}.amazonaws.com/dev/bot/webhook`,
    });

    new cdk.CfnOutput(this, "Lambda Cloudwatch Log URL", {
      value: `https://console.aws.amazon.com/cloudwatch/home?region=${this.region}#logsV2:log-groups/log-group/$252Faws$252Flambda$252F${lambdaTelegram.functionName}`
    });
  }
}

module.exports = { CdkToolStack };

Update Lambda code

As we have gone with Webhook Approach, Telegram Server would push new user messages to our set Webhook URL. (Will show how to set webhook URL).

A normal lambda event object would like below. You can find resource path info, used method and stringified JSON object of telegram bot user message within body field.

{
    "resource": "/bot/webhook",
    "path": "/bot/webhook",
    "httpMethod": "POST",
    "headers": {
        "Accept-Encoding": "gzip, deflate",
        ....
    },
    "multiValueHeaders": {
        "Accept-Encoding": [
            "gzip, deflate"
        ],
        .....
    },
    "queryStringParameters": null,
    "multiValueQueryStringParameters": null,
    "pathParameters": null,
    "stageVariables": null,
    "requestContext": {
        "resourceId": "93ctxg",
        "resourcePath": "/bot/webhook",
        "httpMethod": "POST",
        "extendedRequestId": "N1EZWE8FIAMFimA=",
        "requestTime": "20/Feb/2022:07:02:06 +0000",
        "path": "/dev/bot/webhook",
        "accountId": "285535506992",
        "protocol": "HTTP/1.1",
        "stage": "dev",
        .......
        "domainName": "tq9rr56bhc.execute-api.us-east-1.amazonaws.com",
        "apiId": "tq9rr56bhc"
    },
    "body": "{\"update_id\":192810399,\n\"message\":{\"message_id\":15,\"from\":{\"id\":198940317,\"is_bot\":false,\"first_name\":\"Vikit\",\"username\":\"redblueshine\",\"language_code\":\"en\"},\"chat\":{\"id\":198940317,\"first_name\":\"Vikit\",\"username\":\"redblueshine\",\"type\":\"private\"},\"date\":1645340526,\"text\":\"hi\"}}",
    "isBase64Encoded": false
}

Let's parse the stringified JSON object using JSON.parse(PASTE_STRINGIFIED_DATA) method. You will find from field containing id (Telegram UserID) and text field containing message. We will be requiring this 2 field information for replying towards the message sent by bot user.

File Path: \aws-cdk-telegram-bot-example\assets\lambda\telegram-bot\index.js

Lets add few libraries in our lambda code. e.g axios

const axios = require('axios');
const telegramLink = `https://api.telegram.org/bot${process.env.BOT_TOKEN}/sendMessage`;

exports.handler = async function(event) {
    console.log("request:", JSON.stringify(event, undefined, 2));

    if(event.path==="/bot" || event.path==="/bot/"){
      return {
        statusCode: 200,
        headers: { "Content-Type": "text/plain" },
        body: `Hello, CDK! You've hit ${process.env.AWS_LAMBDA_FUNCTION_NAME} with ${process.env.AWS_LAMBDA_FUNCTION_VERSION}\n`
      };
    }

    try {
      if(event.body){
        const jsonData = JSON.parse(event.body).message;
        await sendReply(jsonData.from.id, 'Processing data:'+jsonData.text);
      }
    } catch(e){
      console.log('Error occured:',e);
    }
    return {
      statusCode: 200,
      headers: { "Content-Type": "text/plain" },
      body: `Success`
    };
  };

function sendReply(chatId, textReply){
  var data = JSON.stringify({
    "chat_id": chatId,
    "text": textReply,
    "disable_notification": true
  });

  const config = {
    method: 'post',
    url: telegramLink,
    headers: { 
      'Content-Type': 'application/json'
    },
    data : data
  };

  return axios(config)
  .then(function (response) {
    console.log(JSON.stringify(response.data));
  })
  .catch(function (error) {
    console.log(error);
  });
}

Deploy CDK changes

Save all the changes and hit below command from directory path aws-cdk-telegram-bot-example/cdk-tool

cdk deploy --require-approval never

End ouput:

Save above 3 links(BotURL, BotWebhookURL, LambdaCloudwatchLogURL) in notepad as we would need it later on.

If incase there is error, you can destroy and recreate by executing below commands.

cdk destroy
cdk deploy --require-approval never

You can confirm changes by moving to API Gateway UI, Lambda UI and associated Cloudformation template.

API Gateway Console UI
Cloudformation Console UI
Lambda Console UI

Setting up Telegram Webhook

Telegram API Documentation can be found at: https://core.telegram.org/bots/api

Set webhook url for the given Telegram Bot.

bot_token=Collected while creating new telegram bot.
url_to_send_updates_to = BotWebhookURL from last step.

You can simply paste below links in web browser.

Set Webhook https://api.telegram.org/bot{bot_token}/setWebhook?url={url_to_send_updates_to}
Get Webhook Info which will contain info on pending replies count and last response. https://api.telegram.org/bot{bot_token}/getWebhookInfo?url={url_to_send_updates_to}
Get some more info. https://api.telegram.org/bot{bot_token}/WebhookInfo?url={url_to_send_updates_to}
Delete webhook and stop pushing message to API Gateway URL. https://api.telegram.org/bot{bot_token}/deleteWebhook?url={url_to_send_updates_to}

So make sure to set webhook url and check the webhook info.

Final Testing

Head over to your created Telegram bot and send some text.
Example: https://t.me/BookmarkmeBot

and testing the normal /bot path from browser.

What's Next

I am not going to make this post lengthy. So you can write additional javascript code to save all your received data to AWS dynamoDB table or you can use Airtable API and see data with Airtable Site UI. Or build some jokes api, dictionary or anything basd on your requirements.

You can find the sourcecode on Git repository at below link. Checkout branch part2
https://github.com/arki7n/aws-cdk-telegram-bot-example.git

Commands for local usage:

git clone https://github.com/arki7n/aws-cdk-telegram-bot-example.git
git checkout part2

Do not forget to destory the cloudformation stack after testing.
Run cdk destroy to wipe out all the created resources.

If you need some more help related to Airtable or building some additional logic, you can follow me at Twitter and I will help you to give some solution.

Follow Twitter: https://twitter.com/arki7n

Akhilesh YadavFollow

Hi, I am Full time Software Engineer and hold varied experience around different Programming Languages and framework.

AWS CDK - Building Telegram bot with AWS Lambda and API Gateway Proxy Integration - Part 1

Akhilesh Yadav — Tue, 02 Nov 2021 09:31:59 +0000

This is the first part of the Building Telegram bot with AWS Lambda and API Gateway Proxy Integration Series and this article would feel easy incase you come from Cloudformation or Terraform background. For beginners, this would make more sense if you follow this article along with AWS CDK Official Guide.

As per official documentation, The AWS Cloud Development Kit (AWS CDK) is an open source software development framework to define your cloud application resources using familiar programming languages.

It means that now you can use TypeScript, JavaScript, Python, Java, or C# to create Infrastructure instead of JSON/YAML used in Cloudformation template.

Advantage of AWS CDK over Cloudformation:

Use logic (if statements, for-loops, etc) when defining your infrastructure
Use object-oriented techniques to create a model of your system
Define high level abstractions, share them, and publish them to your team, company, or community
Organize your project into logical modules
Share and reuse your infrastructure as a library
Testing your infrastructure code using industry-standard protocols
Use your existing code review workflow Code completion within your IDE

With AWS CDK you can design your own reusable components that meet your organization’s security, compliance, and governance requirements. It also provides high-level components that preconfigure cloud resources with proven defaults, helping you build on AWS without needing to be an expert.

Which means Construct props can have default values for some key as mentioned in document for API-Gateway Stage. Do check the Construct Props section and scroll down to see default values for some properties.

https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-apigateway.Stage.html

new Stage(scope: Construct, id: string, props: StageProps)

This is part 1 of building Telegram bot using AWS SDK Series.

The AWS CDK Toolkit provides the cdk command-line interface that can be used to work with AWS CDK applications.

How to setup AWS CDK Project

https://docs.aws.amazon.com/cdk/latest/guide/getting_started.html

Install CDK using Nodejs Package Manager
- Lets install : npm install -g aws-cdk
- Check version using : cdk --version
Configure AWS Credential on your local Machine
- Copy and Paste Access Keys and Secret Keys : aws configure
- Confirm if you gained the AWS Identity. aws sts get-caller-identity

Many AWS CDK stacks that you write will include assets: external files that are deployed with the stack, such as AWS Lambda functions or Docker images. The AWS CDK uploads these to an Amazon S3 bucket or other container so they are available to AWS CloudFormation during deployment. Deployment requires that these containers already exist in the account and region you are deploying into. Creating them is called bootstrapping.

To bootstrap, paste this :

cdk bootstrap aws://$(aws sts get-caller-identity --output text --query Account)/$(aws configure get region)

mkdir telegram-bot
cd telegram-bot

We will create separate directory for managing aws-cdk file and lambda file.

mkdir cdk-tool
mkdir lambda

Now go to cdk-tool empty directory and initialize the app using the cdk init command, specifying the desired template ("app") and programming language. That is:

cd cdk-tool
cdk init app --language javascript

Default directory structure would look like this:

lib/cdk-tool-stack.js : is where your CDK application's main stack is defined. You will write the infrastructure configuration over here in this file.
bin/cdk-tool.js : This is the entrypoint of the CDK application. It will load the stack defined in lib/cdk-tool-stack.js
package.json : is your npm module manifest. It includes information like the name of your app, version, dependencies and build scripts.
cdk.json: tells the toolkit how to run your app.
.gitignore and .npmignore tell git and npm which files to include/exclude from source control and when publishing this module to the package manager.
node_modules is maintained by npm and includes all your project’s dependencies.

Creating Sample Lambda Function

Do you remember you created one more directory to store lambda function, ie. lambda directory. Lets switch to that directory from parent directory and create index.js file into telegram-bot directory containing nodejs function sourcecode.

cd lambda
mkdir telegram-bot
touch index.js

Paste this into index.js file.

exports.handler = async function(event) {
    console.log("request:", JSON.stringify(event, undefined, 2));
    return {
      statusCode: 200,
      headers: { "Content-Type": "text/plain" },
      body: `Hello, CDK! You've hit ${event.path}\n`
    };
  };

Now, lets go back to cdk-tool directory from parent directory and link the lambda function sourcecode with AWS Lambda Construct and later bind it with API Gateway route.

Before that, you would need to install lambda and apigateway cdk constructs using

npm install @aws-cdk/aws-lambda
npm install @aws-cdk/aws-apigateway

Now move to cdk-tool directory :

cd cdk-tool

Go to lib/cdk-tool-stack.js and paste this imports at top:

const lambda = require("@aws-cdk/aws-lambda");
const apigw = require("@aws-cdk/aws-apigateway");

A Lambda CDK construct would look lik below.

    const lambdaTelegram = new lambda.Function(this, "telegramBotHandler", {
      runtime: lambda.Runtime.NODEJS_14_X,
      handler: "index.handler",
      code: lambda.Code.fromAsset("../lambda/telegram-bot"), // from parent directory containing package.json
      architecture: lambda.Architecture.ARM_64,
      environment: {
        'CURRENT_ENV': 'dev',
      }
    });

As seen above, we have mapped lamba sourcecode using lambda.Code.fromAsset and have setup runtime environment, entry of the function, architecture and Lambda Environment variables.

We would also require API Gateway Proxy integration to map it with this lambda function and get invokable API Link using apigw.RestApi

    const restApi = new apigw.RestApi(this, "telegrambot-api", { deploy: false });

Create a new resource path and setup GET method with it and attach it with lambda function construct.

    const method = restApi.root
      .addResource("bot")
      .addMethod("GET", new apigw.LambdaIntegration(lambdaTelegram, { proxy: true }));

Since, we have passed { deploy: false } inside api.RestApi construct props, we would need to create manual deployment and stage by ourselves. Incase, if you do not pass that option, it would create deployment and default prod stage for you and you can ignore next 2 snippet as well.

    const devDeploy = new apigw.Deployment(this, "dev-deployment", { api: restApi });

Creating our own Stage with customized properties using apigw.Stage

    const devStage = new apigw.Stage(this, "devStage", {
      deployment: devDeploy,
      stageName: 'dev' // If not passed, by default it will be 'prod'
    });

Emitting the output values after deployment using cdk.CfnOutput.

    new cdk.CfnOutput(this, "BotURL", {
      value: `https://${restApi.restApiId}.execute-api.${this.region}.amazonaws.com/dev/bot`,
    });

Final code of lib/cdk-tool-stack.js should look as below.

Now, lets validate the written constructs and see if it generates valid cloudformation template.

Enter command in root directory

cdk synth
and then you will see some new directories in your folder structure which contains cloudformation templates and metadata.

Lets deploy this application using:

cdk deploy

You can see in above screenshot with resources that would be created. Confirm by pressing y.

Now, call the API on browser and check the response.

To destroy the application:

cdk destroy

Confirm deletion by pressing y.

You can find the sourcecode on Git repository at :
https://github.com/arki7n/aws-cdk-telegram-bot-example.git

That's it! Hope you have get to complete all of the above steps and achieved the desired output. You can customize the lambda function source code and experiment things on your own as well. I shall be back with part 2 of this series in next blog post. Feel free to follow me to remain updated with next part 2 post.

Akhilesh YadavFollow

Hi, I am Full time Software Engineer and hold varied experience around different Programming Languages and framework.

Using Browser Devtools Advance Network Search filters Effectively

Akhilesh Yadav — Sun, 29 Mar 2020 14:18:17 +0000

This is my first article on dev.to and just felt to share some DevTools network search filter tips.
Many of us work on Web Development stuffs and have to use browser devtools very often to figure out many things.

I am just going to share tricks and tips related to Filter Search given inside Firefox and Chrome Browser Devtools.

You can quickly open the DevTools by simply hitting the F12 key. In most cases though, you also want to inspect an element on a page. There's a special shortcut that gets you there even faster: hitting Ctrl+Shift+C will open the DevTools and invoke the inspector tool.

Using Network Filter Direct Search

Open the Network panel then press Command+F (Mac) or Control+F (Windows, Linux, Chrome OS) to open the new Network Search pane.

Filter requests by properties

Go to Network Tab and click on filter search box.

Use the Filter text box to filter requests by properties, such as the domain or size of the request.

Keyword	Meaning	Examples	Browser
domain	Shows resources coming from a specifc domain.	domain:mozilla.org	Both
has-response-header	Shows resources that contain the specified HTTP response header.	has-response-header:cache-control	Both
is	is:cached and is:from-cache shows only resources coming from cache. is:running shows only resources, which are currently being transferred.	is:cached	Both
larger-than	Shows resources that are larger than the specified size in bytes. k can be used as suffix for kilobytes and m for megabytes, e.g. the value 1k is equivalent to 1024 .	larger-than:2000	Both
method	Shows resources that have were requested via the specific HTTP request method.	method:post	Both
mime-type	Shows resources that match the specified MIME type.	mime-type:application/javascript	Both
set-cookie-domain	Shows the resources that have a Set-Cookie header with a Domain attribute that matches the specified value.	set-cookie-domain:.mozilla.org	Both
set-cookie-name	Shows the resources that have a Set-Cookie header with a name that matches the specified value.	set-cookie-name:_ga	Both
set-cookie-value	Shows the resources that have a Set-Cookie header with a value that matches the specified value.	set-cookie-value:true	Both
status-code	Shows resources that have the specific HTTP status code.	status-code:304	Both
remote-ip	Shows resources coming from a server with the specified IP.	remote-ip:63.245.215.53 ,OR, remote-ip:[2400:cb00:2048:1::6810:2802]	Firefox
cause	Shows resources matching a specific cause type. The types can be found in the description of the cause column .	cause:js , cause:stylesheet , cause:img	Firefox
transferred	Shows resources having a specific transferred size or a transferred size close to the one specified. k can be used as suffix for kilobytes and m for megabytes, e.g. the value 1k is equivalent to 1024 .	transferred:1k	Firefox
size	Shows resources having a specific size (after decompression) or a size close to the one specified. k can be used as suffix for kilobytes and m for megabytes, e.g. the value 1k is equivalent to 1024 .	size:2m	Firefox
scheme	Shows resources transferred via the given scheme.	scheme:http	Firefox
regexp	Shows the resources having a URL that matches the given regular expression.	regexp:\d{5} ,OR, regexp:mdn	mozilla
mixed-content	Show all mixed content resources (mixed-content:all) or just the ones that are currently displayed	mixed-content:displayed	Chrome

Extra Tricks:

You can use multiple properties simultaneously by separating each property with a space. For example, mime-type:image/gif larger-than:1K displays all GIFs that are larger than one kilobyte. These multi-property filters are equivalent to AND operations. OR operations are currently not supported.
You can also filter requests only of method type POST, by using: -- method:POST would show only requests with POST method -- -method:GET, here - inverts the filter, and shows request with all type except GET. -- You can also type - to get a hint of types available and use it.

Some more examples
domain:
-- Use a * character to include multiple domains.
-- Ex: *.com, domain:google.com, -domain:bing.com

has-response-header:
-- Filter resources with the specified HTTP response header.
-- Ex: has-response-header:Content-Type, has-response-header:age

is:
-- is:running finds WebSocket resources
-- is:from-cache,
-- is:service-worker-initiated
-- is:service-worker-intercepted

larger-than:
-- Note: larger-than:1000 is equivalent to larger-than:1k
-- Ex: larger-than:420, larger-than:4k, larger-than:100M

method:
-- method:POST, -method:OPTIONS, method:PUT, method:GET

mime-type:
-- Ex: mime-type:application/manifest+json, mimetype:image/x-icon

mixed-content:
-- mixed-content:all (Show all mixed-content resources)
-- mixed-content:displayed (Show only those currently displayed) (never used this personally)

scheme:
-- Ex: scheme:http, scheme:https,
-- Note that there are also scheme:chrome-extension, scheme:data

set-cookie-domain:
-- Ex: set-cookie-domain:.google.com

set-cookie-name:
-- Match Set-Cookie response headers with name
-- Ex: set-cookie-name:WHATUP

set-cookie-value:
-- Match Set-Cookie response headers with value
-- Ex: set-cookie-value:AISJHD98ashfa93q2rj_94w-asd-yolololo

status-code:
-- Match HTTP status code
-- Ex: status-code:200, -status-code:302

Reference:
https://developers.google.com/web/tools/chrome-devtools/network/reference

Lets connect:
https://ayzom.com
https://www.linkedin.com/in/arki7n
https://www.twitter.com/in/arki7n