DEV Community: Phoebe

How to find and read API/ or Technical Docs

Phoebe — Tue, 17 Nov 2020 02:16:58 +0000

In this article, I will share my strategy on how to find and read technical documentation. While most articles help you read well-written, well-maintained docs, I will share my approach on how to navigate a not-so-friendly docs (Monaco Code editor with an example from my project query.cafe.
For this topic, I also made a Youtube video to show you some examples!
.....
Unfortunately, I can only see a few comprehensive articles and videos to learn the “reading strategy”.
So, now there is one more article to help beginner developers. Let’s dive in!

Why “read the manual” skill is essential but underestimated?

Most people can read, and it is the brain assuming it can read anything easily. Therefore, most beginner developers don’t even realise they don’t know how to read the technical documentation effectively. Reading is handled by default by our System 1 because it is a habit and a primary skill.

Also, generally, technical documentation is dry and not visually attractive; even though I believe the team behind the documentation devote lots of effort to make it as straightforward as possible. Because it is boring to read those materials, we tend to skip them. (My favourite Docs are MDN Web Docs and Google API Docs).

That is why we ignore the “read the manual” skill. However, only in real work, we know its importance. Those documents are the most reliable, updated source of information regarding the technology (language, framework, modules, library) we are working on. If the technology is mature, there are many tutorials and unofficial articles to go to; so we can compare and don’t rely much on the docs. Yet, if the technology is newly developed, the official documentation is the only choice.

My strategy to find and read technical documentation

This is not the “one size fits all” strategy because different materials are written differently. Yet, there are some common things and patterns you can rely on to navigate the docs. Moreover, most docs assume that readers have a reasonable level of understanding of technology and syntax.

KNOW THE DOCS

Similar to reading a map, you must get yourself familiar with the legends and navigation. There are many things in the docs homepage, some are descriptive, some aren’t, but I usually care about a few things first.

Scan the page and remember the navigation

As mentioned, there are many things on the homepage with tabs, links, sections. They are very descriptive; so to save time for your next visit; you should let your mind wander and scan through those sections to remember briefly what each section is about.

Version

Most cases the version you downloaded are the latest version, so the docs by default support the latest version. However, for example, Python 2 and Python 3 are equally popular, so you should check your project specs.

Tab “Docs”, “API Docs”, “Getting Started”, “What is…”, “About”

These are things you come to the page for. You want to know what this doc is about and if there are solutions for what you are working on. They also show some use-cases in tutorials so you can get some hints on what function you should look for.
You can’t miss these tabs, and they are usually used universally across docs.

“Tutorials”, “Playground”

In these tutorials, you can build small applications using technology. These tutorials cover the basics, but you can learn the documentation pretty fast just by completing the tutorials.

Search bar of the page and Brower search (Ctrl + F)

The simple yet powerful tool! I usually have a specific thing to look for in the documentation page. I don’t want to be dragged away by the details on the page.

Firstly, I use the Browser search to see if what I look for is available on the main page (most likely not but it’s worth trying). Then, I use the Search bar of the page to find from the page directory.

For example, on Monaco code editor, I want to find a function to help me with “value” displayed in the code editor on query.cafe. So the keyword is “value” or “setvalue” or “set” or “update”.
With search bar, you should be creative and play the keyword game like you do with Google.

STRATEGY

1. Read with a purpose. Know what is the expected behaviour/ output of the code you want to write.

What problem are you trying to solve? There are two scenarios here to consider before diving deeper into the documentation.

If you are looking for feature-oriented or problem-oriented solutions, you should go to complimentary (Google, Stackoverflow, Github Repo, Youtube tutorials) sources first. For example, you want to build a code editor in your web application. Documentation can’t guide you through the process on how to build it, where to start.

Documentation is only useful when you have a bug to fix, or you look for technical instruction to write a function to do a certain thing.

2. Know what you are working on

A string, an array, displaying value issue, handling output value of a method, request from/to the server.

3. Some common categories “Event”, “Functions”, and “Classes” are used to group the information.

4. For Functions, you should look for what Parameters you need to give, Syntax, what Output and what Type of Output it gives back.

It is essential to know the output from a function because it will link to what you can do next with it.
Also, remember to look for “Events”, “Classes”, “Functions” tab referred from the current section you are in.

If you have other tips on how to read a technical document, please comment. I think others will find it helpful too.

How I’m learning about APIs by building a Slackbot — Part 2

Phoebe — Sun, 27 Sep 2020 12:52:10 +0000

Continuing on my HelloWorld version, I completed my HelloGalaxy version. This journal article is to share the development process of heybot’s HelloGalaxy version, and one of my favourite challenges which there was no official documentation and I figured out my own solution (using private_metadata).

My full code is available here.

The Roadmap

>>Project Overview

>> Part 1: Hello World version

>> Part 3: Final version (coming up next)

Recalling from my original roadmap, this is where we stand.

Demo

Objectives

In this version, my goals were:

To connect to BambooHR API by creating a trial account and Token key.
When I say “Hello”, heybot replies with action options (buttons).
When I click on a button to choose the service, heybot askes for employee ID via a Modal. Heybot returns answers to me.

Resources

BambooHR API

The Documentation is clear and well organized; APIs are grouped by activities (Report, Login, Employees).
I wanted to build heybot for one single customer, so all I needed was (1) an account (NOTE: a trial account is valid in 7 days only), (2) subdomain is “mycompany” which I created when I opened my trial account (My company is monsterinc :D), (3) an API key which I made from my trial account “Settings”.
How I create an API key:

I recommend copying the API key and saving it in a note because we have to use it frequently.
One more important thing that you should note is the AUTHORIZATION_TOKEN ("Basic NDJim******** *) in *headers.

Why do we need this?

We can't use the API key to send an HTTP Request to BambooHR directly. An API key will have to be converted into an authorization token.
Before running the Flask app, we have to "export AUTHORIZATION_TOKEN= "NDJim**********" to be an environment variable (similar to what I did to SLACK_TOKEN and SLACK_SIGNING_SECRET which I explained in Step 3.1 here).

How to use BambooHR API Key to find AUTHORIZATION TOKEN?

Go to the API Reference page and choose an API and "Try It".

Challenges and Solutions

Find the right documentations on Slack API

Slack has been upgrading its API, and because of that, some features or methods were either discontinued or were still in use, but their support resources were deprecated.
In HelloGalaxy version, I needed to build:

rich-text messages containing buttons/ menu/ date picker
a modal. For #1, Slack introduced Blocks (supported by Block Kit Builder to replace attachments according to this post. The purpose of this upgrade is to support more interactive features (vote, buttons, menu) or rich contents (video, link, image).

This is my action message built with Blocks and sent to the user by chat_postMessage() method.

You can see my Block payload here under def understood_greeting(self, user).

For #2, Modal replaced Dialogs according to this post. This change led to the retirement of dialog.open() method and the introduction of views.open() method to open Modal (we can't use chat_postMessage() or other regular methods to open a Modal).

Get content from request body with `Content-type = application/x-www-form-urlencoded`

Normally, working with Content-type = application/JSON is very easy and straightforward. We can use request.data, request.get_data, request.json, request.get_json.

application/x-www-form-urlencoded is a bytestring type file. So, if we use request.data (the common solution), we will get an empty string.
This StackOverflow post is a comprehensive answer. My full answer with details was in here too. Below is my solution for your quick look.

@ app.route('/slack/request_handler', methods=['POST'])
def request_handler():
   # return an ImmutableMultiDict with 1 pair
   payload = request.form 

   # get value of key 'payload'
   a1 = payload['payload']  

   # Note that if you have single quotes as a part
   # of your keys or values, this will fail due to
   # improper character replacement.
   # Convert a string (representation of a Dict))
   # to a Dict
   a2 = json.loads(a1)

   # Get value of key "channel"
   channel = a2["channel"] 
   print(channel) 
   # {'id': 'D01ACC2E8S3', 'name': 'directmessage'}

Using private_metadata to solve view_submission payload's limitations.

NOTE: When a button clicked, it sends back a block_actions payload8 containing channel_id, action_id. When a modal is closed (made submission), a view_submission payload is sent.
I save samples of block_actions payload and view_submission payload here.

Context:

A Modal appears to get Employee ID input from a user. Once the user provided the ID and clicked "Submit", a view_submission payload was sent to my server.

I used that ID to get the answer from BambooHR.
I wanted to reply to the user.

Problem:

To call the correct function to send a GET request to BambooHR and build a right template message, need an action_id which I set when I created the Block message. view_submission payload doesn't have action_id.
In order to reply, a channel_id is a must. view_submission payload doesn't have channel_id.

Solution:

There was no official documentation on this from Slack. However, I utilized private_metadata to store the action_id received from the block_actions payload.
This Slack doc suggested storing channel_id in private_metadata.

Pseudocode

Code

# build view payload
def get_employee_id_modal(self, channel_id, action_id):
    private_metadata = {
       "channel_id": channel_id,
       "action_id": action_id
    }
    # jsonify a dict into a string 
    # (required type for private_metadata)
    json_private_metadata = 
    json.dumps(private_metadata)

    # return a block
    return ({
      "type": "modal",
      "private_metadata": json_private_metadata,
      "callback_id": "employee_id_modal",
      [....]
      })

Thanks for reading!
Phoebe

How I’m learning about APIs by building a Slackbot

Phoebe — Mon, 21 Sep 2020 05:35:14 +0000

Project Context

I have an HR background and with my experiences as an HR Partner in tech and banking industry, I see so much potential to improve the HR field with technology. Some areas to start with: communication, data management, data analytics, paperless process, automation workflow, knowledge management system and smart decision making using AI.

There have been so many great products to transform HR such as CultureAmp, Workday, Zendesk, Trello but I’ll say “We can do more!”.

I chose to build a Slackbot because a smart virtual assistant helps to improve productivity for both employees and HR team. For example, an employee is going to have a secondment wants to know about their journey (host-country’s culture, law, tax, company benefits, project details…). Traditionally, HR prepares lots of materials and training. But when employees want to check on certain things, either they have to read the materials again or ask HR again…and again. Both ways waste time and repeat communications.

Also, I finished my first project Eatthefrog, and it is a client-side web app. It runs on users’ browser and uses local storage to store tasks made by them, so it doesn’t “talk” to any server. With this Slackbot project (the bot’s name is heybot :), I will know how to answer those questions. Because I will:

Be a SERVER receiving/responding requests from a user (via heybot on Slack) aka CLIENT.
Be a CLIENT sending requests and receiving responses from a SERVER (BambooHR in this case).
Practice working with APIs

So, what is heybot?
heybot is a Slackbot that integrates with BambooHR, and it assists employees (1) to know information about their time off, (2) to request time off on BambooHR from Slack and get request update.

Github repo

The Roadmap

This project is more complicated, so I broke it down to multiple versions. For each version (milestone), I have separated post to give you more insights about that version’s development story.

>> Part 1: Hello World version

>> Part 2: Hello Galaxy version (coming up next)

>> Part 3: Final version (coming up next)

Here is the bird’ s-eye view of the project.

Technology Choices

Python or Nodejs

While choosing Nodejs is a rational choice because I don’t have to face “environment change” issue and not to mention Slack API’s well-supported libraries and tools, I chose Python.

Inspired by Alexa, I want heybot to have conversations as natural as possible with a user, and it can “understand” the user’s intention/ request without fixed commands. Therefore, Python is a better choice. It has several popular and commonly used (Natural Language Processing) NLP libraries.

Flask vs Django

These are two popular frameworks for Python. According to some references #1, #2, #3, and the fact that I don’t need to build a web interface for heybot, Flask is a suitable framework. It provides enough functionalities for me to run heybot. Also, slackclient — Slack Developer Kit uses Flask.

Thanks for reading!
Phoebe

How I’m learning about APIs by building a Slackbot — Part 1

Phoebe — Mon, 21 Sep 2020 05:07:45 +0000

This is my journal article for the development process of heybot’s Hello world version. I will write from a beginner’s perspective so it can be lengthy (a bit), but I hope other beginners will find it useful.

The project overview is here.

fullcode

Objective

In this version, my purposes were:

Connect heybot (the back-end is stored in my laptop) to Slack server
When I say “Hello”, heybot replies “Hello ”

Know the resources

These resources are all I need to build the Helloworld version and upgrade it to the final version.

General knowledge about API

Flask

I didn’t have a server, so using Flask I can create my webserver. With Flask I can handle web requests quickly (with no configuration).

ngrok

Thanks to Flask, I technically have a server on my Mac. But I am still invisible to other web services (servers) — in this case, they are Slack and BambooHR.

Wait! Why? Don’t you usually still access those services via your browser?
Ok! I can “talk” to them because I am one of their clients. They are visible in public web with a static address (domain). So, I can find them, ask them to give me the services whenever I want. They are going nowhere(unless they shut down their businesses).

Back to “But I am still invisible to other web services (servers)”, I’m having heybot as a service provider. According to the explanation above, if I want to “run my business”, I need to have a static address, too. This means I need to have a public address that can be found by other clients (can be another service provider (Slack, BambooHR) or a regular user).

In a nutshell, ngrok helps to create a tunnel so those web services can “see/ find” me. And, when they send me requests, thanks to ngrok’s tunnel, I can receive and respond to those requests.

Slack API

They have a few core APIs — Web API, Event API and other APIs.

Developer Kit for Python — slackclient

Slack prepares some tool kits for different programming languages to work with their server.

Slack Events API adapter for Python — python-slack-events-api

After hours of internet browsing and reading tutorials and docs, to build the Helloworld version, using Slack Events API adapter is a recommended and preferred approach. It is also easy to use too.
Hint: You might find some tutorials using RTM — Real Time Messaging API. According to this tutorial, “you can stream events through a websocket connection with our RTM API. The RTM API is only recommended if you’re behind a firewall and cannot receive incoming web requests from Slack. ⚠️ The RTM API isn’t available for default Slack apps. If you need to use RTM (possibly due to corporate firewall limitations), you can do so by creating a classic Slack app”.

Step-by-step

There are plenty of materials to help you create a simple Slackbot. So, I won’t repeat what they said. Instead, I follow the same steps, link to what materials I found useful, and I only write what I figured out.

⚠️Before starting, remember to set up a virtual environment. It is a MUST.

1. Create a Slack app

This step is straightforward on Slack API docs and also in this step of this tutorial.

2. Build the message

In step 2 of this tutorial, they prepared a complex message, but Helloworld version only needs heybot to say “Hello ”. So, I skipped that step.

3. Prepare to pass URL verification challenge from Slack

The URL generated by ngrok must pass this test.
You should print the request to see what it has. Trust me, it helps a lot, knowing what is sending to you.

def reply():
 # print(request.headers)
 # print(request.data)
 # print(request.args)
 # print(request.form)
 # print(request.endpoint)
 # print(request.method)
 # print(request.remote_addr)

 # get ready to receive and respond HTTP POST
 # request from Slack to verify bot's endpoint URL
 if request.method == 'POST':
  challenge_parse = (json.loads(request.data)) 
                   ['challenge']
  # print(challenge_parse)
  # respond URL verification from Slack  
  # with 'challenge' value
  response = {"challenge": challenge_parse}
  return response, 200

3.1 Respond to Slack Event

This step is the key to make heybot alive. It is in step 3 of this tutorial.
Explanation:

# import dependencies to obtain the environment variable values
import os 
import slack
from slackeventsapi import SlackEventAdapter
import json
from flask import Flask, request, jsonify

# Import environment var by retrieving exported token https://slack.dev/python slackclient/auth.html
SLACK_BOT_TOKEN = os.environ['SLACK_BOT_TOKEN']
SLACK_SIGNING_SECRET = os.environ['SLACK_SIGNING_SECRET']

What is os?

The OS module in Python provides functions for interacting with the operating system. In this case, I had to use SLACK_BOT_TOKEN and SLACK_SIGNING_SECRET. They are confidential, so we shouldn’t set them as variables in our code. I made them as environment variables (use export command) => if I share the code, no one can see unless I gave them the tokens. So, os helps to call out environment variables.

What is json module?

Data in a request that a server sends/ receives is a JSON object. I need to parse JSON (in Python, parse JSON means json.loads)
----------

# create the Flask server
app = Flask(__name__)

# Initialize a Slack Event Adapter for receiving actions 
slack_events_adapter = SlackEventAdapter(SLACK_SIGNING_SECRET, '/slack/events', app)

# Instantiate a Web API client
slack_web_client = slack.WebClient(
token=os.environ['SLACK_BOT_TOKEN'])

slackclient v2 separated WebClient and RTMClient. You can read about the differences in this Migration guide.
----------

# Routing
@app.route('/', methods=['POST', 'GET'])

All functions go under this route.

In Flask docs, you see they create multiple routes, which is for building a website where you have multiple pages, and each page has its path (route). We are building a chatbot, and we only need one end-point handling requests sent from Slack.
----------

# When a user sends a DM, the event type will be
# 'message'.
# Link the message callback to the 'message' event.
# Choose to use Event API (handled by
# SlackEventAdapter) instead of RTM API.

@slack_events_adapter.on("message")
def say_hello(event_data):
     message = event_data['event']

     # if the incoming message contains "hello" 
     # NOT CASE SENSITIVE, respond with a message
     # check if the message is from a bot or not.  
     # If Yes, do nothing. If No, reply.
     if message.get('bot_id') is None and 'hello' 
     in ((message.get('text')).lower()):
          channel_id = message['channel']
          user = message['user']
          message = "Hello <@%s>! :tada:" % user
          slack_web_client.chat_postMessage 
          (channel=channel_id, text=message)
     else:
          return

# Error events
@slack_events_adapter.on("error")
def error_handler(err):
     print("ERROR: " + str(err))

slack_events_adapter.start(port=5000)

The syntax for this code you can read from this example, chat_postmessage method.

Challenges and Solutions

heybot can’t distinguish between its messages and user’s messages

Because of that, it replies non-stop.

Solution on this StackOverflow post was the most relevant and well-described, but it mentioned a solution checking for subtype in the request’s event, which wasn’t there.

My solution was to check for bot_id from event.

# some code
if message.get('bot_id') is None .....:
# some code

[Errno 48] Address already in use

Occasionally, when you run ngrok, this error comes because sometimes the system didn’t kill the program process properly when you asked.

The solution is to use kill -9, which is not very recommended but it worked.

# list of processes using the port if any 
sudo lsof -i:5000   
# or 
ps -a 

# use the id on the PID column to terminate the process use 
kill (PID)

# if the above doesnt solve the problem, use this. Not very recommended :D. 
kill -9 (PID)

Phew! Thanks for reading this lengthy post. You are awesome! I explained the technical work with my own knowledge, so if they aren’t correct, please tell me.

Phoebe

DEV Community: Phoebe

How to find and read API/ or Technical Docs

Why “read the manual” skill is essential but underestimated?

My strategy to find and read technical documentation

KNOW THE DOCS

Scan the page and remember the navigation

Version

Tab “Docs”, “API Docs”, “Getting Started”, “What is…”, “About”

“Tutorials”, “Playground”

Search bar of the page and Brower search (Ctrl + F)

STRATEGY

1. Read with a purpose. Know what is the expected behaviour/ output of the code you want to write.

2. Know what you are working on

3. Some common categories “Event”, “Functions”, and “Classes” are used to group the information.

4. For Functions, you should look for what Parameters you need to give, Syntax, what Output and what Type of Output it gives back.

How I’m learning about APIs by building a Slackbot — Part 2

The Roadmap

>>Project Overview

>> Part 1: Hello World version

>> Part 3: Final version (coming up next)

Demo

Objectives

Resources

BambooHR API

Challenges and Solutions

Find the right documentations on Slack API

Get content from request body with Content-type = application/x-www-form-urlencoded

Using private_metadata to solve view_submission payload's limitations.

How I’m learning about APIs by building a Slackbot

Project Context

The Roadmap

>> Part 1: Hello World version

>> Part 2: Hello Galaxy version (coming up next)

>> Part 3: Final version (coming up next)

Technology Choices

Python or Nodejs

Flask vs Django

How I’m learning about APIs by building a Slackbot — Part 1

Objective

Know the resources

General knowledge about API

Flask

ngrok

Slack API

Developer Kit for Python — slackclient

Slack Events API adapter for Python — python-slack-events-api

Step-by-step

1. Create a Slack app

2. Build the message

3. Prepare to pass URL verification challenge from Slack

3.1 Respond to Slack Event

What is os?

What is json module?

Challenges and Solutions

heybot can’t distinguish between its messages and user’s messages

[Errno 48] Address already in use

Get content from request body with `Content-type = application/x-www-form-urlencoded`