DEV Community: TheNiqabiCoderMum

🔥 Chatfuel Image Gallery from Google Sheets with Google Apps Script

TheNiqabiCoderMum — Wed, 10 Jul 2019 12:28:38 +0000

Let’s say you are making a bot for a restaurant on Chatfuel. The bot can tell the users the daily specials. The daily specials would change, well, daily and you want the restaurant staff to update them. Changing things daily inside Chatfuel by people who may not be tech savvy, the restaurant staff for example, is not the recommended thing to do. And things can get further complicated if you are making the bot for a client.

So, we want a place where the staff can update the specials every day and the bot reads the specials before sending the information to the user. Typically, this is a job for a database, but Google Sheets can be used as a light weight and easy to use alternative to databases in this scenario.

The only problem is that Chatfuel doesn’t offer the functionality to read data from Google Sheets out of box. Using third party integrations, Zapier or Integromat for instance, does the trick but it adds to the overhead costs.

The good news is that if you know some basic JavaScript, you can use Google Sheets like a regular database and integrate it with your bot. Enter Google Apps Script!

Google Apps Script lets you do new and cool things with Google Sheets. You can use Apps Script to add custom menus, dialogs, and sidebars to Google Sheets. It also lets you write custom functions for Sheets, as well as integrate Sheets with other Google services like Calendar, Drive, and Gmail.

But Google Apps Script can do a lot more than that. And in this tutorial, we will use Google Apps Script to read the daily specials for a pizza shop from a Google Sheet and send the data as gallery cards to Messenger through our bot.

If you are not familiar with Google Apps Script, it is a scripting language for light-weight application development in the Google ecosystem. It is based on JavaScript. So, if you are familiar with JavaScript, using Google Apps Script is fairly simple.

Let’s get started.

Scaffolding

Go to Google Sheets and create a new blank sheet.

To follow along this tutorial, make columns for name, description and image URL. Here is a screen shot of my sheet with some fake data. Make sure your images are hosted somewhere on the web and they have the right permissions.

Once your sheet is set up as you want, let’s write our script.

Introducing Google Apps Script

There are different types of Google Apps scripts and in this tutorial, I will create a container bound script. You can read more about the different kinds of scripts here. But basically, what it means is that a script which is bound to a Google Sheet cannot be detached from the file they are bound to, and they gain a few special privileges over the parent file.

To create a bound script, in your Google Sheet, select tools from the menu and then select Script Editor.

It will open the Google Apps Scripts project page.

A project represents a collection of files and resources in Google Apps Script, sometimes referred to simply as "a script". A script project has one or more script files which can either be code files (having a .gs extension) or HTML files (a .html extension). You can also include JavaScript and CSS in HTML files.

You can read more about Google Apps Scripts projects here.

Give your project a suitable name.

As you can see there is a code editor where we will write our code. Currently there is just an empty function here.

function myFunction() {  
}

Google Apps Script has a basic logging mechanism using the Logger class. So we can use Logger.log in place of JavaScript’s console.log. Let’s log a simple “Hello, world!”.

function myFunction() {
    Logger.log('Hello World!');
}

Click save and then run your script.

Now click on View > Logs or simply hit Ctrl + Enter and you should see Hello World! displayed on the logs.

Deploying The Script As A Web App

At the moment this is just a script. We will need to turn this script into a web app so our bot can communicate with it.

Any script can be published as a web app if it meets these requirements:

It contains a doGet(e) or doPost(e) function.
And the function returns an HTML service HtmlOutput object or a Content service TextOutput object.

Armed with this information, lets change our function.

function doGet() {
  Logger.log("Hello World!");
  return ContentService.createTextOutput('Hello, world!');
}

Let’s save this script again and then deploy it as a web app. Hit Publish and select "Deploy as web app".

Change the “Who has access to this app” to “Anyone, even anonymous” and click deploy.

Note the web app URL from the next screen.

Integrating with Chatfuel

Now go to your Chatfuel bot and add a JSON API card where you want to display the specials. Change the Type of the request to GET and in the URL paste the URL you copied from the Google Apps Script project page. Test the request. In the response section, under Response Body, you should see our Hello, world! text.

Now that our bot is connected to our web app running on Google Apps Script project, let’s see how we can read data from the Google Sheets.

Reading Data From Google Sheets Programmatically

To retrieve the data from the spreadsheet, you must get access to the spreadsheet where the data is stored, get the range in the spreadsheet that holds the data, and then get the values of the cells.

Apps Script facilitates access to the data by reading structured data in the spreadsheet and creating JavaScript objects for them.

Since we are using a bound script the above process is fairly easy. We will just call a few methods on the JavaScript object created for us. You can read about all available methods here. Change the code to following.

function doGet() {
  var sheet = SpreadsheetApp.getActiveSheet();
  var data = sheet.getDataRange().getValues();
  for (var i = 0; i < data.length; i++) {
    Logger.log('Item name: ' + data[i][0]);
    Logger.log('Item description: ' + data[i][1]);
  }
  return ContentService.createTextOutput('Hello, world!');
}

Hit save. Google will ask you permission to access data and it may tell you that this web app is not safe. Proceed anyway and then run your function. Check the logs and you should get something like this.

As you can see it is also reading the header row with the data. But that can be fixed easily by initializing our loop variable with 1 instead of 0.

This is an extremely simple script and we are just scratching the surface of all the possibilities offered to us. Feel free to play around with the code and build more complex functionality. But for the purpose of this tutorial, this script will do.

Making Image Gallery From The Data

Now that we know how we can read and parse data from our sheet programmatically, let's see how we can send this data back as a gallery.

Chatfuel documentation gives us all the information we need. Go to the JSON API section and scroll down to "Sending galleries". The page looks like this.

As we can see, we need to send the actual data in the form of a list of objects.

So let's first create a list of objects or elements from our data.

Change the code to following.

function doGet() {
  var sheet = SpreadsheetApp.getActiveSheet();
  var data = sheet.getDataRange().getValues();
  elements = create_elements(data)
  Logger.log(elements);
  return ContentService.createTextOutput('Hello, world!');
}

function create_elements(data) {
  var elements = [];
  for (var i = 1; i < data.length; i++) {
    var object = {
      title : data[i][0],
      image_url : data[i][2],
      subtitle : data[i][1],
      buttons : [
        {
          "type":"web_url",
          "url":"https://blog.naveeraashraf.com/",
          "title":"View Item"
        }
      ] 
    }
    elements.push(object)
  }
  return elements  
}

We are looping through our rows and adding the data to a JavaScript object, which is then pushed to a list. We also moved the code to create our objects in a separate function to keep our code clean. You can check your logs to see if your code is working properly.

So far we are only logging the objects and not sending them to our bot. Let's change that. First we will use our objects to create the response which will build a gallery. Add the following function to your code. You can copy the response from Chatfuel docs if you wish and make necessary changes.

function buildImageGallery(elements) {
  var output = JSON.stringify({
    messages: [
      {
        attachment: {
          type: "template",
          payload: {
            template_type:"generic",
            image_aspect_ratio: "square",
            elements: elements
          }
        }
      }
    ]
  });

  return ContentService.createTextOutput(output).setMimeType(ContentService.MimeType.JSON);
}

We are simply replacing the elements list in the docs with the list we have created in the previous step.

We will also add some functionality to our code for when there is no data in the sheet. This way our code won't break in case the restaurant staff forgot to add the new specials but deleted the old ones.

Your final code should look like this now.

function doGet() {
  var sheet = SpreadsheetApp.getActiveSheet();
  var data = sheet.getDataRange().getValues();
  elements = create_elements(data)
  if (elements.length != 0) {
     return buildImageGallery(elements);
  } else {
    return notFound()  
  }
}

function create_elements(data) {
  var elements = [];
  for (var i = 1; i < data.length; i++) {
    var object = {
      title : data[i][0],
      image_url : data[i][2],
      subtitle : data[i][1],
      buttons : [
        {
          "type":"web_url",
          "url":"https://blog.naveeraashraf.com/",
          "title":"View Item"
        }
      ] 
    }
    elements.push(object)
  }
  return elements  
}

function buildImageGallery(elements) {
  var output = JSON.stringify({
    messages: [
      {
        attachment: {
          type: "template",
          payload: {
            template_type:"generic",
            image_aspect_ratio: "square",
            elements: elements
          }
        }
      }
    ]
  });

  return ContentService.createTextOutput(output).setMimeType(ContentService.MimeType.JSON);
}

function notFound() {
  var output = JSON.stringify({
    messages: [
      {
        text: "There are no items in this category"
      }
    ]
  });
  return ContentService.createTextOutput(output).setMimeType(ContentService.MimeType.JSON);
}

Test your bot in Messenger and you should get the data from your sheet displayed as an image gallery.

That's it! I hope you enjoyed this tutorial. If you did don't forget to share it.

Making a Static Site Generator with Python - part 2

TheNiqabiCoderMum — Fri, 03 May 2019 09:57:10 +0000

Originally published at my blog: blog.naveeraashraf.com

In the first part of this tutorial we explored the two main components behind making a static site generator or SSG - the Markdown parser, Markdown2, and the templating engine, Jinja2. We saw how to use these components together to create HTML files from Markdown files and pre-created templates.

In this part we will create our own static site generator. This is the final product

So let's get started.

Structure

In this tutorial I will create a recipe blog. Since this tutorial is for educational purpose, I will only create two main views: the home page of the recipe blog with all the posts listed on it, and the individual posts pages. You can find the complete code in the github repository

Scaffolding

Create a new folder where you want your code to live:

mkdir recipe-ssg

In this folder, create the content folder where we will write our blog posts in markdown files:

cd recipe-ssg & mkdir content

We will also need posts written in markdown. If you want to follow along, you can copy the recipes written in markdown from here. By the way these are not just place holder recipes. These are my tried and tested recipes. If you are a baker, do give these recipes a try!

We will also need some images. Feel free to use my images from here. Make a folder in your root project called output. Inside the output folder make another folder called img and add all the image files to this folder.

You will also need to install Markdown2 and Jinja2, using pip or pipenv, ideally inside a virtual environment.

pipenv install markdown2
pipenv install jinja2

Writing the Python Script

Create a new file in your project root and call it main.py. First we will import all the packages we will need:

import os
from datetime import datetime
from jinja2 import Environment, PackageLoader
from markdown2 import markdown

Next we will parse our markdown files. This is how we did it in the first part:

from markdown2 import markdown

with open('content/turkish-pide.md', 'r') as file:
    parsed_md = markdown(file.read(), extras=['metadata'])

But since we have more than one file now, we will change the code to loop over all the files in the content folder like so:

POSTS = {}
for markdown_post in os.listdir('content'):
    file_path = os.path.join('content', markdown_post)

    with open(file_path, 'r') as file:
        POSTS[markdown_post] = markdown(file.read(), extras=['metadata'])

Next we will sort these posts in reverse order so the newest posts show first but the dates are in a string format. So we will first need to convert them to datetime:

POSTS = {
    post: POSTS[post] for post in sorted(POSTS, key=lambda post: datetime.strptime(POSTS[post].metadata['date'], '%Y-%m-%d'), reverse=True)
}

So far so good. You can check if your script is working so far by printing some metadata to the console.

Creating Templates

Next we will write our templates. Create a templates directory in the project root. We will write two templates, one for the main home page and the other for individual posts. Let's write the individual posts one first:

<!DOCTYPE html>

<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <meta http-equiv="X-UA-Compatible" content="ie=edge">
    <title>{{ post.title }}</title>
</head>
<body>

    <h1>{{ post.title }}</h1>
    <small>{{ post.date }}</small>
    <p>
        {{ post.content }}
    </p>
</body>
</html>

Save the above code into post.html inside the templates directory. Create another file home.html inside templates directory and paste the following code:

<!DOCTYPE html>

<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <meta http-equiv="X-UA-Compatible" content="ie=edge">
    <title>My Recipes</title>
</head>
<body>

    <h1>My Recipes</h1>
    {% for post in posts %}
    <p>
        <h2>{{loop.index}}: <a href="posts/{{ post.slug }}/">{{post.title}}</a> <small>{{post.date}}</small></h2>
        {{post.summary}}
    </p>
    {% endfor %}
</body>
</html>

This template is a little less straight forward than the individual post one. But we are simply using Jinja2's for loop to loop through all the posts and filling in the place holder with data from those posts. The data will be handed over to this template in a list so we can easily loop over it. When we will go to our main.py again and write the script to pass data along to these templates, this will make much more sense.

Using Template Inheritance

But before we do that, you may have noticed that there is quite a bit of repetition. We had to write the same scaffolding HTML code for both templates. While this is not a big deal in this case because we have just two templates, but imagine having to rewrite the same code for larger projects.

For sites with navigation menus and footers a lot of code will need to be written over and over again. And if you had to make a change, let's say in your navigation menu, you will need to make the change on each and every template.

This can get incredibly monotonous, not to mention increasing the chance of errors and bugs.

Fortunately all templating languages offer the solution to this problem through template inheritance. You can read more about Jinja2's template inheritance from here.

Templates usually take advantage of inheritance, which includes a single base template that defines the basic structure of all subsequent child templates. You use the tags {% extends %} and {% block %} to implement inheritance. ~ Real Python

Create a new file in templates directory and call it layout.html. In this file we will put all the code that needs to be repeated on every template:

<!DOCTYPE html>

<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <meta http-equiv="X-UA-Compatible" content="ie=edge">
    <title>My Recipes</title>
</head>
<body class="container">

    {% block content %} {% endblock %}

    <br>

</body>
</html>

Now let's make suitable changes to the other two templates as well. In the home.html template paste the following code:

{% extends "layout.html" %}

{% block content %}

<h1>My Recipes</h1>

<div>

{% for post in posts %}

    <p>
        <h2>{{loop.index}}: <a href="posts/{{ post.slug }}.html">{{post.title}}</a> <small>{{post.date}}</small></h2>
        {{post.summary}}
    </p>

{% endfor %}

</div>

{% endblock %}

And in the post.html change the code to:

{% extends "layout.html" %}

{% block content %}

<h1>{{post.title}}</h1>
<p>
    <small>{{post.date}}</small>
    {{post.content}}
</p>

{% endblock %}

Now we have much cleaner code. Now let's work on rendering these templates.

Rendering Home Page

In your main.py file, get the templates with Jinja2 just like we did in the first part. But this time we will get two templates:

env = Environment(loader=PackageLoader('main', 'templates'))
home_template = env.get_template('home.html')
post_template = env.get_template('post.html')

Now let's pass the data to our home page template from our POSTS list. Since the home page only needs the metadata we will pass it metadata only:

posts_metadata = [POSTS[post].metadata for post in POSTS]
home_html = home_template.render(posts=posts_metadata)

This will pass a list of metadata through the variable posts to our home page template. This is the same posts variable over which we looped in the template.

One more thing that I want to do is to have the tags of each post in some kind of list so I can loop through them as I may want to make each tag into a clickable link. Currently all the tags are being passed as a single string.

To change that I will create a new list from the post_metadata['tags'] variable and pass it along with other data to my home template.

posts_metadata = [POSTS[post].metadata for post in POSTS]
tags = [post['tags'] for post in posts_metadata]
home_html = home_template.render(posts=posts_metadata, tags=tags)

Now let's write this HTML to a file:

with open('output/home.html', 'w') as file:
    file.write(home_html)

Run your main.py and you will get a home.html in your output directory. Open the file in browser and it will look like this:

Rendering Individual Posts

If you click on the title of any of the post in your browser, there will be a not found error. Because the individual post pages haven't been rendered yet. To do so add the following code to your main.py:

for post in POSTS:
    post_metadata = POSTS[post].metadata

    post_data = {
        'content': POSTS[post],
        'title': post_metadata['title'],
        'date': post_metadata['date'],
    }

    post_html = post_template.render(post=post_data)

    post_file_path = 'output/posts/{slug}.html'.format(slug=post_metadata['slug'])

    os.makedirs(os.path.dirname(post_file_path), exist_ok=True)
    with open(post_file_path, 'w') as file:
        file.write(post_html)

Now run the main.py again and now when you click on a link it will take you to the corresponding page.

Extra Bits

Our static site generator is done at this point. But I want to show you a few extra things.

Let's say we want to add some css to make our site look nicer. This is where we will be thankful to the template inheritance. Just add the following styling to your layout.html in the templates directory:

<style>
    .container {
        width: 80%;
        margin: auto;
        margin-top: 3em;
    }
</style>

Run your main.py again and you will see the styles applied to all pages. Great!

Let's add a light-weight css framework to our site. I am using Picnic.

Add this link to the head of your layout.html:

<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/picnic@6.5.0/picnic.min.css">

Re run main.py and voila!

Remember those tags we passed as an individual list? Using Jinja2's build in filters we can now iterate over that list from within our template and put each tag inside it's own span or button element.

{% set list_of_tags = post.tags.split(",") %}
{% for tag in list_of_tags %}
    <button class="shyButton mybutton">{{ tag }}</button>
{% endfor %}

And after adding some CSS classes to my code here is the final product

and

You can find the complete code in the github repository.

That's it! I hope you enjoyed this tutorial. If you did don't forget to share it.

⚡️ Making a Static Site Generator with Python - part 1

TheNiqabiCoderMum — Thu, 02 May 2019 10:51:57 +0000

Static site generators are a secure and fast alternative to traditional content management systems like Wordpress.

Static Site Generators or SSG are pretty simple to use once you get the hang of them. But initially there may be a learning curve. One way of learning (my preferred way) is to look under the hood and see how things work and try to recreate them. With this end goal in mind we will create our own static site generator using Python. Plus it is a lot of fun to make.

If you are new to static site generators, you can first read about what is a static site generator, when to use one, and when not to use one. Also have a look at a list of many static site generators written in different programming languages.

So without further ado, let’s start.

Understanding how static site generators work

The concept behind statistic site generators is pretty simple. You write your posts in markdown and the script in the static site generators turn your markdown into HTML using pre-created templates. These static HTML files can be uploaded to any web server and served to your readers.

Let’s say you are making a recipe blog. You write your recipe in markdown like this:

title: Turkish Pide
date: 17-04-19

**Ingredients for the dough:**
500 gm bread flour
1 tblspoon yeast
1 tblspoon honey
1 tsp salt
2 tblsp olive oil
250 ml or as req water.

**Ingredients for the filling:**
1 cup grated Mozerella
½ cup crumbled feta 

Combine the ingredients of the dough and knead until smooth and elastic. Let rest 45 minutes. Divide in two. Roll each in a rectangle. Spread the cheese filling over top. And roll the sides in to form a boat shape. Let rise for 10-15 minutes. Brush the sides with olive oil and bake at 230C for 10-15 minutes or until nicely browned.

And you have a template like this:

<html>
    <head>
        <title>{{ title }}</title>
    </head>

    <body>
        <p>
            Published at: {{ date }}
            {{ content }}
        </p>
    </body>
</html>

The static site generator will parse the markdown text and insert each component where it belongs, so you will end up with something like this:

<html>
    <head>
        <title>Turkish Pide</title>
    </head>

    <body>
        <p>
            Published at: 17-04-19
            <strong>Ingredients for the dough:</strong><br>
            <ul>
                <li>500 gm bread flour</li>
                <li>1 tblspoon yeast</li>
                <li>1 tblspoon honey</li>
                <li>1 tsp salt</li>
                <li>2 tblsp olive oil</li>
                <li>250 ml or as req water.</li>
            </ul>

            <strong>Ingredients for the filling:</strong>
            <ul>
                <li>1 cup grated Mozerella</li>
                <li>½ cup crumbled feta</li>
            </ul>

            Combine the ingredients of the dough and knead until smooth and 
elastic. Let rest 45 minutes. Divide in two. Roll each in a rectangle. Spread the
cheese filling over top. And roll the sides in to form a boat shape. Let rise for
10-15 minutes. Brush the sides with olive oil and bake at 230C for 10-15 minutes
or until nicely browned.
        </p>        
    </body>
</html>

So we need to find a way to parse the text from markdown to HTML and to insert HTML into pre-created templates.

Converting Markdown to HTML

Create a new folder where you want your code to live:

mkdir recipe-ssg

SSG stands for Static Site Generator by the way. And as good as it is for the SEO of my site, I am pretty tired of typing Static Site Generators over and over so I will refer to Static Site Generators as SSG from now on. Phew!

In this folder, create the content folder where we will write our blog posts in markdown files:

cd recipe-ssg & mkdir content

In the content folder create a markdown file. You can copy the contents of this file and call it turkish-pide.md.

You will notice that the file is divided into two parts. There is some metadata between the two dash separators. And the actual content is below the separator.

Next we want a way to parse this markdown, convert it into HTML and store this data so it can be consumed by Python. But we don't need to reinvent the wheel. The awesome Python community has already written a Markdown Parser for us. We will install it and play around a bit to see how it works.

Within your virtual environment install markdown2 using pipenv or pip.

pipenv install markdown2

In the interactive Python shell try:

from markdown2 import markdown
markdown("**This is a very important message**")

# outputs '<p><strong>This is a very important message</strong></p>\n'

Great! Now we can create a simple script to convert our markdown format files into HTML files.

Create a main.py file in the root of your project (not inside the content) and type the following:

from markdown2 import markdown

with open('content/turkish-pide.md', 'r') as file:
    parsed_md = markdown(file.read(), extras=['metadata'])


print('Metadata: ', parsed_md.metadata)
print('Content: ', parsed_md)

Run main.py and you will see the contents of the turkish-pide.md file printed on the console with HTML tags. Also notice that the metadata is converted into a Python dictionary. We can now use these variables to create pages using templates.

But first we need to create a template.

Creating and Using Templates

Templates are ordinary HTML files with placeholders for the variable data. We will use Jinja2 templating language for Python which will simply insert the data from our variables in the HTML template. If you are new to templating languages or Jinja2, you can first read more about it here.

Install Jinja2 within your virtual environment using pipenv or pip:

pipenv install jinja2

To try Jinja2, create a folder inside your project root and call it templates. Now create a test.html inside templates and put the following code in it:

<!DOCTYPE html>

<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <meta http-equiv="X-UA-Compatible" content="ie=edge">
    <title>{{ post.title }}</title>
</head>
<body>

    <h1>{{ post.title }}</h1>
    <small>{{ post.date }}</small>
    <p>
        {{ post.content }}
    </p>
</body>
</html>

Great! Now we have a template and we just have to extract our variables from the parsed markdown and hand them over to Jinja2.

In your main.py add the following code:

from jinja2 import Environment, PackageLoader

env = Environment(loader=PackageLoader('main', 'templates'))
test_template = env.get_template('test.html')

data = {
    'content': parsed_md,
    'title': parsed_md.metadata['title'],
    'date': parsed_md.metadata['date']
}

print(test_template.render(post=data))

After importing the Jinja2, we create an environment to show Jinja2 where the templates folder is located. Make sure that the first argument for PackageLoader is the name of your python file, in my case it is main. Next we get the template we need and after extracting data from our parsed file we hand it over to Jinja2 while calling the render() function.

Your final code should look like this now:

from markdown2 import markdown
from jinja2 import Environment, PackageLoader

with open('content/turkish-pide.md', 'r') as file:
    parsed_md = markdown(file.read(), extras=['metadata'])

    env = Environment(loader=PackageLoader('main', 'templates'))
    test_template = env.get_template('test.html')

    data = {
    'content': parsed_md,
    'title': parsed_md.metadata['title'],
    'date': parsed_md.metadata['date']
    }

    print(test_template.render(post=data))

If you run your main.py now you should get all the code from your test.html file with the contents from your turkish-pide.md file inserted in suitable places.

Now that we know how to render HTML pages using Jinja2, in the next part we will put all the pieces together and make our SSG.

Special thanks to Jahangir Rahmonov from where I learned the concepts and his article served as the base for this tutorial.

🚀 Building a Telegram Bot with AWS API Gateway and AWS Lambda

TheNiqabiCoderMum — Tue, 27 Nov 2018 06:46:23 +0000

Over the past year I have made my fair share of bots. I have made bots using services like Chatfuel and Flowxo, I have made bots from scratch in Python and Nodejs and I have used frameworks like Microsoft Bot Framework. I have also experimented with different deploying options for my coded-from-scratch bots including VPS, web servers, and serverless architecture.

Out of all of these options, I keep coming back to serverless deployment. Not only can using a serverless computing architecture be extremely economical in many use cases, it is also very low maintanance. You don't have to worry about maintaining a server at all. Just write your code, upload it to the cloud and then you can pretty much forget about it.

Echo Telegram bot with AWS API Gateway and AWS Lambda

In this guide I will demonstrate making a simple serveless Telegram bot that echoes everything a user inputs, using AWS API Gateway and AWS Lambda. But first a little about the services we will use.

What is API Gateway

Amazon API Gateway is a publicly available endpoint for our code that runs on AWS Lambda, Amazon EC2, or other publicly addressable web services. It helps us deliver mobile and web application backends in a secure and scalable way.

What is AWS Lambda

AWS Lambda is a service that lets us create stateless functions (or blocks of code) in the cloud which can be invoked (or run) through a multitude of triggers. There are many other services beside AWS which let us create cloud functions, like Google Cloud Functions and Azure Functions.

Bots or chatbots are an ideal candidate for this kind of computing service since in essence chatbots are just a piece of code which is triggered through a user interaction. Many popular chatbot making platforms are already using cloud functions as the backbone of their service.

With all that introduction out of the way, let's get started!

Registering a bot on Telegram

To follow along this guide, you will need a Telegram bot. Registering a new bot in Telegram is really simple. Just send a "/newbot" command to Telegram's own bot BotFather and follow the instructions. You will receive a bot token, copy this token and keep it handy.

You can customize the descriptions and about section of your bot, as well as change the profile picture for the bot through BotFather. Once you are happy with your bot's settings, you are ready to proceed.

The way a Telegram bot works is that when a user sends a message to our bot's account, Telegram sends that message to our code. Our code processes that message, and after doing whatever it needs to do, it sends back a message to Telegram servers which is then delivered to the user.

There are two ways we can get new updates from Telegram servers.

Long polling in which our code keeps asking Telegram servers if there is a new update.
Setting webhooks in which we ask Telegram servers to send any new message/update received to us through a HTTP request.

Setting webhooks is the most resource friendly and scalable way to receive new updates. And this is what we are going to do. But for that we will need a secure URL where Telegram servers can push all updates. Enter AWS API Gateway.

Creating an API Gateway

If you don't have an Amazon account, sign up for one. You will need to provide your credit card details but Amazon has a very good free tier and to follow along this guide shouldn't cost you anything.

Click on 'Create API'

Give your API a name and description and then click Create API.

Before defining any resources for this API, let's first create our Lambda function.

Creating a lambda function

Choose 'Author from scratch' option for now. Once you have followed this guide till the end, you can try exploring the different blueprints provided by Amazon.

Give you function a name, choose your favorite runtime (in the case of this guide choose Python3.6).

Each Lambda function has an IAM role (execution role) associated with it. You specify the IAM role when you create your Lambda function. Permissions you grant to this role determine what AWS Lambda can do when it assumes the role. You can read more about managing permissions here.

For the purpose of this guide, we will select 'Create a new role from templates' from the drop down menu. Give your role a name and then search 'basic' in the 'Policy templates' dropdown list. Select 'Basic Lambda @ Edge permissions'. Then click 'Create function'.

This will take you to your function. Scroll down and familiarize yourself with different options. You need to pay attention to a few options in particular:

Environment variables - any environment variables that your code uses must be declared here.
Basic settings from where you can change the allocated memory and timeout. Please note that changing these settings may affect the costs. You can learn more about Lambda pricing here.

Before we add any code to our function, let's first add a trigger to it. We want that everytime someone sends a message to our bot on Telegram, the Telegram servers should push it to our URL wher our code is hosted. In other words we want a HTTP GET/POST request to trigger our code and we will do this through our API gateway.

You can add the trigger to your Lambda function from Lambda's own page as well but for now we will head over to API Gateway page to connect our Lambda function with the API and to configure the API.

Connecting Lambda function to API Gateway

Select the API which we created earlier. We will need to add some methods to this API. From the dropdown 'Actions' list select 'Create Method'

You will see another dropdown list appear under "/". From here you can add methods such as "GET" and "POST". Since we are making a really simple bot, we don't want a bunch of different methods. So for simplicity's sake we will just choose "ANY" method. It means that no matter what kind of request is sent to this URL, it will always trigger the same code.

Hit the small check sign next to "ANY" and it will take you to a new screen where you can choose what to do once this endpoint receives any kind of request. From here choose "Lambda Function" for integration type and the name of your Lambda function from the list. Also check the "Use Lambda Proxy Integration" option which will enable us to access the request details inside our Lambda function.

Click save. Test your API on the next screen to check if everything is working the way it should.

Then click "Deploy API" from the "Actions" list.

Choose a stage name like "V1" or "beta" for the stage name. Once your API is deployed you will get an invoke URL. If you click on this URL, you should get a simple web page with "Hello from Lambda!". Note down this URL.

Setting Webhooks

So now we have a secure URL which is connected to our code living in a cloud function. Cool! We can go ahead and tell Telegram to send any future messages received by our bot to this address.

Setting a webhook is as simple as typing the following in your browser and hitting enter. (Make sure to substitute the place holder text)

"https://api.telegram.org/bot<your-bot-token>/setWebHook?url=<your-API-invoke-URL>"

You will get a confirmation message from Telegram and from now on any messages sent to your bot will be pushed to this URL.

Adding code to Lambda function

Go ahead and try sending a message from your own Telegram account to the bot.

So apparently nothing happens 🤔 and that is because our code is not yet doing anything. But if you want to make sure that your function was triggered by receiving the webhook, you can do so by checking the CloudWatch logs. Select "Monitoring" tab and then "View logs in CloudWatch" on the Lambda function's page.

We will write some very basic code which will just return the message sent by the user. Replace the code in your Lambda function's visual editor with the following:

import json
from botocore.vendored import requests

TELE_TOKEN='<YOUR-BOT-TOKEN>'
URL = "https://api.telegram.org/bot{}/".format(TELE_TOKEN)

def send_message(text, chat_id):
    final_text = "You said: " + text
    url = URL + "sendMessage?text={}&chat_id={}".format(final_text, chat_id)
    requests.get(url)

def lambda_handler(event, context):
    message = json.loads(event['body'])
    chat_id = message['message']['chat']['id']
    reply = message['message']['text']
    send_message(reply, chat_id)
    return {
        'statusCode': 200
    }

This is a very simple piece of code as you can see. We are just extracting the message and chat id from the event object and sending it back as a query string along with the chat id in a HTTP GET request to https://api.telegram.org with our token. We also return a JSON object with "statusCode" as "200" so telegram servers can know that the request was received successfully.

You can read more about interacting with the Telegram API here.

Go ahead and send your bot some messages and it should return the same messages back.

Of course, for a production bot we will do a few things differently, for example adding the bot token as an environment variable and adding security layers. But that is beyond the scope of this simple tutorial.

Conclusion

This brings us to the end of this guide. We learned how to create a simple echo bot for Telegram but more than that we learned how to host it completely serverless in the cloud.

Traditionally to run this kind of bot, we would need to write our code not as a simple script but within a micro service framework like Flask. We would need basic scaffolding like a web server to serve the code, a SSL certificate to make our connection secure and so on. But thanks to the powerful Lambda functions and the API Gateway our task was made significantly simpler.

Image credit Designed by Freepik