DEV Community: Rahul Banerjee

How to Transform Data into Profit with Python and Flask

Rahul Banerjee — Wed, 05 Apr 2023 18:03:13 +0000

This article was originally posted on realpythonproject.com

Introduction

In today's digital landscape, APIs play an essential role in connecting services, sharing data, and enabling developers to create innovative applications. If you have unique data or functionality to offer, creating your own API and monetizing it can be a profitable venture. In this article, I will walk you through the process of building a custom API using Python and Flask, using MuscleWiki as an example. Additionally, we will explore how to monetize your API through rate-limiting and access control.

I recently scraped the data from MuscleWiki, created an API, and added it to RapidAPI. Below are some of the things I learned and some ideas I'd like to share.

GitHub Repo: https://github.com/rahulbanerjee26/MuscleWikiAPI

RapidAPI: https://rapidapi.com/rahulbanerjee26/api/musclewiki

Brainstorming and Identifying a Unique API Idea

To build a successful API, you must start with a unique and valuable idea. In the case of MuscleWiki, the API provides users with detailed information on exercises, muscles, and workout routines. This data can be integrated into fitness applications, websites, or other services.

Some tips for coming up with an API idea:

Look for websites with valuable data that can be accessed and used more efficiently through an API.
Identify pain points or areas where existing APIs could be improved or expanded upon.
Consider niche markets or specific industries that could benefit from specialized APIs

One approach is to look for websites or applications that offer valuable data or services but do not have an API. You can then scrape the website or reverse-engineer the application to create your own API that provides access to that data or functionality.

To come up with your own API idea, consider the following steps:

a. Identify your target audience: Consider the specific group of users or developers you want to target with your API. This could be a niche market or a broader segment of users with a particular interest. Defining your target audience will help you tailor your API features and functionality to meet their specific needs and preferences.

b. Research the market and competition: Examine existing APIs in the same domain or industry to ensure that your idea stands out from the competition. Assess the unique value proposition your API can offer and analyze any potential gaps in the market that your API can fill. You can use platforms like RapidAPI to explore existing APIs and gather insights into their features, popularity, and pricing models. Additionally, engage with online developer communities and forums to gather feedback on their pain points and unmet needs.

c. Evaluate demand and profitability: Conduct market research to estimate the potential demand for your API. Consider factors such as the size of the target audience, the potential for growth, and the expected revenue. To gather this information, you can use tools like Google Trends, keyword research tools, and industry reports to gauge the interest in your API's domain. Analyze user behavior, pain points, and needs through surveys, interviews, or focus groups. Based on your research findings, develop a revenue model and pricing strategy that aligns with the value your API provides and the market's willingness to pay.

By following these steps, you can come up with a unique and valuable API idea that addresses the needs of your target audience, fills a gap in the market, and holds the potential for profitability. This strong foundation will help ensure the success of your API as you move forward with development, deployment, and monetization.

In this tutorial, we will use Muscle Wiki. It is a website that provides valuable information about different exercises but does not have an API. We will scrape the website to collect data about each exercise and create an API that allows users to search for exercises based on different criteria.

Data Collection: Scraping and Organizing Data

If your API relies on external data sources, you will need to collect and organize this data. In the case of MuscleWiki, the data is scraped from the Muscle WIki website and compiled into a structured format. To achieve this, you can use Python libraries like Beautiful Soup and requests.

a. Choose your data sources: Identify the websites or data sources you will scrape to gather the information your API will provide. Make sure the data is accurate, reliable, and up-to-date.

b. Scrape the data: Use Python libraries like Beautiful Soup and requests to extract the desired data from the chosen websites. For example, you can fetch the HTML content of a page and parse it to extract specific elements.

Scraping Data from Muscle Wiki

To scrape data from Muscle Wiki, we will use Python's requests and BeautifulSoup libraries. We will define a function called get_exercise_data() that takes in a URL for a specific exercise on Muscle Wiki and retrieves relevant data about the exercise, such as its target muscles, difficulty level, and steps.

We will also define a function called get_musclewiki_data() that loops through all of the exercises listed on Muscle Wiki and calls get_exercise_data() for each exercise. This function saves all of the data to a JSON file for later use.

Here's the code snippet that shows how we scraped data from Muscle Wiki. I have removed some of the code, you can find the complete source code here

def muscleList(string):
    # Function to get targetted muscles for an exercise
    pass

def html_table_to_json(table,exerciseRowJson):
    # Function to convert html table to json
    pass

def getSteps(stepList):
    # Function to get the steps to do an exercise
    pass 

def getVideos(video):
    # Function to get the Videos of an exercise
    pass

def get_exercise_data(exercise_url,exerciseRowJson):
    muscleList()
    html_table_to_json()
    getSteps()
    getVideos()

def cleanData(exercise):
   # Function to clean the data
   pass

def organzieData(exercise):
  # Function to organize the data
  pass

# Function to get the workout data from muscle wiki
def get_musclewiki_data():
    url = "https://musclewiki.com/directory"
    response = requests.get(url)
    soup = BeautifulSoup(response.text, 'html.parser')

    exercises = soup.find_all('table', class_='wikitable')

    musclewiki_data = []
    count = 0
    for exercise in exercises:
        exerciseRowJson = get_exercise_data(exercise,exerciseRowJson)
        cleanData(exercise)
        organizeData(exercise)
        musclewiki_data.append(exerciseRowJson)
        count+=1

    with open('musclewiki_data.json', 'w') as f:
        json.dump(musclewiki_data, f,indent=4)

get_musclewiki_data()

API Design: Defining Endpoints, Routes, and Data Format

With your data ready, you can now begin designing your API. This involves:

a. Defining the endpoints and routes for your API: Determine the set of endpoints your API will offer, and design the routes that developers will use to access them. Keep the routes descriptive, consistent, and easy to understand.

b. Deciding on the data format: Choose a data format (e.g., JSON or XML) for your API responses. JSON is a popular choice due to its lightweight nature and compatibility with most programming languages.

c. Designing a logical and user-friendly API structure: Organize your API's endpoints, routes, and data format in a way that is logical and easy to understand for developers. This will enhance

Building the MuscleWiki API with Python and Flask

Python and Flask are excellent tools for building APIs. Flask is a lightweight web framework that enables you to quickly create RESTful APIs with minimal effort. In this section, we'll discuss the process of building your API using Python and Flask.

a. Set up your Flask application: First, create a new Flask application and define the necessary routes for your API endpoints.

from flask import Flask, jsonify

app = Flask(__name__)

@app.route('/apiexercises', methods=['GET'])
def get_exercises():
    # Replace this with your actual data fetching logic
    pass

if __name__ == '__main__':
    app.run(debug=True)

b. Implement the data fetching logic: Write the necessary Python functions to fetch the data from your chosen data storage solution, such as a database or an in-memory data structure.

def fetch_exercise_data():
    # Logic to fetch exercise data from your data storage
    pass

For the musclewiki API, I used a simple local JSON file for storage. However, if you plan to monetize your API, you'd have to use 3p services to store your data.

c. Process and serve the data: Process the fetched data and serve it through your API endpoints. This might involve filtering, sorting, or transforming the data before returning it to the user.

def get_exercises():
    exercise_data = fetch_exercise_data()
    # Process and filter the data as needed
    return jsonify({'exercises': exercise_data})

The API for MuscleWiki has the following endpoints

/: Home
/exercises: Get a list of exercises
/exercises/attributes: Get exercise attributes
/exercises/<int:exercise_id>: Get a specific exercise by ID

Below is the code for the endpoint to get all exercises. You can find the complete source over here

from flask import Flask, request,json

app = Flask(__name__)

# Load workout data
with open('workout-data.json', 'r') as f:
    workout_data = json.load(f)

@app.route('/exercises')
def get_exercises():
    # Get query parameters
    category = request.args.get('category')
    difficulty = request.args.get('difficulty')
    # Filter exercises based on query parameters
    filtered_exercises = []

    for exercise in workout_data:
        if category and category.lower() != exercise['Category'].lower():
            continue
        if difficulty and difficulty.lower() != exercise['Difficulty'].lower():
            continue
        filtered_exercises.append(exercise)

    response = app.response_class(
        response=json.dumps(filtered_exercises),
        mimetype='application/json',
        status=200
    )
    return response

Deploying the API to Vercel

Vercel is a platform that allows you to deploy your API for free. To deploy your Flask API on Vercel, follow these steps:

Sign up for a Vercel account and install the Vercel CLI.
Initialize your project by running vercel init in your project directory. This command will create a vercel.json file that you can configure to deploy your Flask app.
Configure the vercel.json file to use the correct Python runtime and set the entry point for your Flask app. Additionally, make sure to include any environment variables necessary for your application, such as API keys or database credentials. You can find MuscleWiki's vercel.json here
Deploy your API by running vercel deploy && vercel deploy --prod . Vercel will provide you with a unique URL to access your API. Make note of this URL, as you'll need it when registering your API on RapidAPI.

For more information on deploying Flask apps on Vercel, refer to Vercel's documentation.

Monetizing the API: Implementing Rate-limiting and Access Control

To monetize your API effectively, you must implement rate-limiting and access control. I haven't added rate-limiting or access-control to MuscleWiki. The only rate limiting is done by Vercel's free plan.

Rate-limiting helps protect your API from abuse and ensures fair usage among your users. Access control allows you to provide different levels of access based on the user's subscription plan.

a. Implement rate-limiting with Flask-Limiter: Flask-Limiter is a popular extension that enables you to set rate limits on your API endpoints easily. By using this extension, you can define limits based on the user's subscription plan, IP address, or custom criteria.

b. Implement access control using decorators: Create custom decorators to check the user's API key and determine their subscription level. Apply these decorators to your API endpoints to restrict access based on the user's subscription plan.

c. Integrate rate-limiting and access control into your API: Combine the rate-limiting and access control implementations to create a robust and monetizable API.

Below is some sample code to give you a better idea and to help you get started.

from flask import Flask, request, jsonify
from flask_limiter import Limiter
from functools import wraps

app = Flask(__name__)

limiter = Limiter(app, default_limits=["100 per day", "10 per hour"])

def check_subscription(subscription_level):
    def decorator(f):
        @wraps(f)
        def decorated_function(*args, **kwargs):
            api_key = request.headers.get('X-Api-Key')
            user_subscription = get_user_subscription(api_key)
            if user_subscription < subscription_level:
                return jsonify({'error': 'Unauthorized'}), 401
            return f(*args, **kwargs)
        return decorated_function
    return decorator

@app.route('/free_endpoint')
@limiter.limit("60 per hour")
def free_endpoint():
    return jsonify({'data': 'This is a free endpoint'})

@app.route('/premium_endpoint')
@check_subscription('premium')
@limiter.limit("1000 per hour")
def premium_endpoint():
    return jsonify({'data': 'This is a premium endpoint'})

if __name__ == '__main__':
    app.run(debug=True)

Launching and Promoting Your API

Once your API is ready and you have implemented rate-limiting and access control, it's time to launch and promote your API to developers. Consider adding your API to RapidAPI to get more visibility. RapidAPI is a popular API marketplace where developers can discover, test, and connect to APIs. By adding your API to RapidAPI, you can potentially monetize it by offering paid subscription plans. Follow RapidAPI's documentation to add your API

Here are some other key aspects to focus on:

a. Provide clear and concise documentation: Good documentation is critical to the success of your API. Include detailed explanations of each endpoint, the expected inputs, and the returned data. Also, provide examples of how to use your API to help developers understand its value and benefits.

b. Offer a free trial or limited free tier: Allowing developers to test your API before committing to a paid plan can encourage adoption and demonstrate its value. A free trial or limited free tier can help users evaluate your API and determine if it fits their needs.

c. Monitor usage and gather feedback: Track how developers are using your API and gather their feedback. Use this information to continuously improve and refine your API. An up-to-date and responsive API will increase its value and attract more users.

d*. Collaborate and network*: Reach out to other developers, bloggers, or influencers in your target market to promote your API. By collaborating with others and increasing your API's visibility, you'll attract more potential users.

e. Utilize social media and online forums: Share your API on social media platforms, developer forums, and other online communities relevant to your target audience. Engaging with potential users and answering their questions can help establish your API as a valuable resource in the market.

f. Create a dedicated website or landing page: Design a professional website or landing page that showcases the features and benefits of your API. Include detailed information about pricing plans, API documentation, and use cases to help potential users understand the value of your API.

g. Offer excellent customer support: Providing responsive and helpful customer support can significantly contribute to the success of your API. Ensure that users have a clear channel of communication to report issues, ask questions, or provide feedback. A positive user experience can lead to increased user satisfaction, word-of-mouth promotion, and long-term success.

By focusing on these aspects, you can create a well-documented, user-friendly API that caters to the needs of your target audience. With a solid implementation of rate-limiting and access control, you can monetize your API effectively and turn it into a profitable venture.

Conclusion

Building and monetizing a custom API with Python and Flask can be a rewarding and profitable experience. By following this guide, you can create a unique and valuable API that attracts developers and users alike. Remember to focus on identifying a unique API idea, collecting and organizing data, designing your API, implementing rate-limiting and access control, and promoting your API effectively. With dedication and persistence, you can successfully launch and monetize your custom API.

How to Use Decorators in Python?

Rahul Banerjee — Sun, 05 Mar 2023 02:33:23 +0000

Decorators in Python are a powerful tool that allows you to modify the behavior of functions and classes. In this tutorial, we'll take a deeper look at advanced decorators and how they can be used to solve common programming challenges.

This tutorial will talk about the different kinds of decorators, decorator chaining, and some common useful decorators

Decorator Functions

A decorator function takes a function as an argument and returns a modified version of it. It is used to modify the behavior of the original function.

Here's an example of a simple decorator function

# The decorator function adds two numbers to the result of the original function
def add_two(func):
    def wrapper(*args, **kwargs):
        # Original Function is invoked
        result = func(*args, **kwargs)
        # Result of original function is modified by adding 2
        return result + 2
    return wrapper

@add_two
def multiply(a, b):
    return a * b

@add_two
def divide(a,b):
  return int(a/b)

print(multiply(3, 4)) # 14
print(divide(10,2)) #7

In this example, the add_two decorator function takes the multiply function as an argument and returns the wrapper function, which adds two to the result of multiply. The @add_two syntax before the multiply function is a shorthand for the following:

def multiply(a, b):
    return a * b

multiply = add_two(multiply)

Since we do not know the number of parameters the original function might have we use args and kwargs

Decorator Classes

In addition to decorator functions, you can also use decorator classes to modify the behavior of functions and classes.

Here's an example of a simple decorator class that adds two numbers to the result of the original function:

class AddTwo:
    def __init__(self, func):
        self.func = func

    def __call__(self, *args, **kwargs):
        result = self.func(*args, **kwargs)
        return result + 2

@AddTwo
def multiply(a, b):
    return a * b

@AddTwo
def divide(a,b):
  return int(a/b)

print(multiply(3, 4)) # 14
print(divide(10,2)) #7

__init__ and __call__ are called dunder methods or magic methods

In this example, the AddTwo decorator class takes the multiply function as an argument and returns a callable object (an instance of AddTwo) that adds two to the result of multiply. The @AddTwo syntax before the multiply function is a shorthand for the following

def multiply(a, b):
    return a * b

multiply = AddTwo(multiply)

Parametrized Decorators

So far, we've looked at simple decorators that modify the behavior of a function in a fixed way. However, you can also create parametrized decorators that take parameters to control their behavior.

Here's an example of a parametrized decorator function that adds a specified number to the result of the original function:

def add_n(n):
    def decorator(func):
        def wrapper(*args, **kwargs):
            result = func(*args, **kwargs)
            return result + n
        return wrapper
    return decorator

@add_n(10)
def multiply(a, b):
    return a * b

@add_n(12)
def divide(a,b):
  return int(a/b)

print(multiply(3, 4)) # 22
print(divide(10,2)) # 17

In this example, the add_n decorator function takes a parameter n and returns a decorator function decorator that adds n to the result of the original function. The @add_n(10) syntax before the multiply function is a shorthand for the following:

def multiply(a, b):
    return a * b

multiply = add_n(10)(multiply)

Below is an image comparing our first decorator add_two and the parameterized decorator add_n

Before we were wrapping the original function and returning the wrapper
Now, we are wrapping the decorator (that wraps the original function) and returning the decorator

Decorating Classes

In our previous examples, we looked at how to decorate functions. But you can decorate classes as well

Here's an example of a simple class decorator that adds a class attribute n to the decorated class:

def add_attribute(n):
    def class_decorator(cls):
        cls.n = n
        return cls
    return class_decorator

@add_attribute(10)
class MyClass:
    pass

print(MyClass.n) # 10

In this example, the add_attribute decorator function takes a parameter n and returns a class decorator function class_decorator that adds the class attribute n to the decorated class. The @add_attribute(10) syntax before the MyClass class is a shorthand for the following:

class MyClass:
    pass

MyClass = add_attribute(10)(MyClass)

Below is another example fo decorating classes

def add_method(cls):
    def new_method(self):
        return "Hello, World!"
    setattr(cls, 'new_method', new_method)
    return cls

@add_method
class MyClass:
    pass

my_obj = MyClass()
print(my_obj.new_method()) # Hello, World

It simply adds new_method to every instance of MyClass

Decorator Chaining

One of the most powerful features of decorators is the ability to chain them together to modify the behavior of a function or class in multiple ways. Remember, a decorator simply returns a modified function. This modified function can be passed to another decorator to further modify it.

Here's an example of how to chain two decorators to modify the behavior of a function:

# Adds n to the result of original function
def add_n(n):
    def decorator(func):
        def wrapper(*args, **kwargs):
            result = func(*args, **kwargs)
            return result + n
        return wrapper
    return decorator

# Multiplies result of original function by n
def multiply_by_m(m):
    def decorator(func):
        def wrapper(*args, **kwargs):
            result = func(*args, **kwargs)
            return result * m
        return wrapper
    return decorator

@add_n(10)
@multiply_by_m(5)
def multiply(a, b):
    return a * b

@multiply_by_m(5)
@add_n(12)
def divide(a,b):
  return int(a/b)

print(multiply(3, 4)) # 70
print(divide(10, 2)) #85

We have two parameterized decorators to multiply and add. The order of applications of the decorators chained is from bottom to top. I.e the last decorator is applied first and the first decorator is applied last.

In this example, the multiply function is decorated by both the add_n(10) and multiply_by_m(5) decorators. The multiply_by_m(5) decorator is applied first, followed by the add_n(10) decorator.

In the second case, however, The add_n(12) decorator is applied first and multiply_by_m(5) is applied second.

Use Cases of Decorators

1. Timing Decorator

A timing decorator can be used to measure the time taken by a function to execute. Here's an example:

import time

def timing_decorator(func):
    def wrapper(*args, **kwargs):
        start = time.time()
        result = func(*args, **kwargs)
        end = time.time()
        print(f"Time taken: {end - start:.2f} seconds")
        return result
    return wrapper

@timing_decorator
def long_running_function():
    time.sleep(3)

long_running_function()

In this example, the timing_decorator decorator adds timing information to the long_running_function function. When the long_running_function is executed, the wrapper function provided by the decorator measures the time taken for the function to execute and prints the result.

2. Authorization Decorator

An authorization decorator can be used to enforce access control on a function. Here's an example:

def authorized(func):
    def wrapper(*args, **kwargs):
        user = get_current_user()
        if not user.is_authenticated():
            raise Exception("Unauthorized")
        return func(*args, **kwargs)
    return wrapper

@authorized
def restricted_function():
    print("Access granted")

In this example, the authorized decorator adds an authorization check to the restricted_function function. When the restricted_function is executed, the wrapper function provided by the decorator retrieves the current user and checks if the user is authenticated. If the user is not authenticated, the wrapper raises an exception.

3. Logging Decorator

A logging decorator can be used to log the execution of a function. Here's an example:

def logging_decorator(func):
    def wrapper(*args, **kwargs):
        print(f"Calling function: {func.__name__}")
        result = func(*args, **kwargs)
        print(f"Returning from function: {func.__name__}")
        return result
    return wrapper

@logging_decorator
def some_function(a, b):
    return a + b

print(some_function(3, 4)) # 7

In this example, the logging_decorator decorator adds logging information to the some_function function. When the some_function is executed, the wrapper function provided by the decorator logs the function name before and after the function is executed.

We can chain this with the timing decorator

@logging_decorator
@timing_decorator
def long_running_function():
    time.sleep(3)

'''
Calling function: wrapper
Time taken: 3.00 seconds
Returning from function: wrapper
'''
long_running_function()

4. Cache Decorator

A cache decorator can be used to cache the results of a function for faster access. Here's an example:

def cache_decorator(func):
    cache = {}
    def wrapper(*args, **kwargs):
        key = f"{args[0]}"
        print(cache)
        if key not in cache:
            print("Calling function")
            cache[key] = func(*args, **kwargs)
        else:
            print("Getting value from cache")
        return cache[key]
    return wrapper

@cache_decorator
def expensive_function(n):
    result = 0
    for i in range(n):
        result += i
    return result

'''
{}
Calling function
4950
{'100': 4950}
Getting value from cache
4950
'''
print(expensive_function(100)) # 4950
print(expensive_function(100)) # 4950

In this example, the cache_decorator decorator adds a cache to the expensive_function function. When the expensive_function is executed, the wrapper function provided by the decorator checks if the result is already present in the cache. If it is, the cached result is returned, otherwise the function is executed and the result is added to the cache.

Notice how the cache a dictionary is still able to persist between calls to the decorated function because the wrapper function created by the decorator retains a reference to the cache dictionary. This allows the wrapper function to access and modify the cache dictionary.

If a new function is decorated with the cache decorator, the wrapper function will have a reference to a new empty cache dictionary

'''
{}
Calling function
100
'''
print(outputResult(100)) 

'''
{'100': 4950}
Getting value from cache
4950
'''
print(expensive_function(100))

5. Singleton Class Decorator

A singleton class decorator can be used to ensure that only one instance of a class is created. Here's an example:

def singleton(cls):
    instances = {}
    def get_instance(*args, **kwargs):
        if cls not in instances:
            instances[cls] = cls(*args, **kwargs)
        return instances[cls]
    return get_instance

@singleton
class MyClass:
    pass

obj1 = MyClass()
obj2 = MyClass()
obj3 = MyClass()
print(obj1 is obj2) # True
print(obj2 is obj3) # True
print(obj1 is obj3) # True

In this example, the singleton decorator is used to decorate the MyClass class. The decorator works by creating a get_instance function that stores an instance of the class in a dictionary. When the get_instance function is called, it checks if an instance of the class already exists and returns it if it does, otherwise, it creates a new instance of the class and adds it to the dictionary.

Conclusion

In conclusion, decorators are a powerful feature in Python that allow you to add functionality to a function without modifying its original code. Decorators can be used to add caching, logging, validation, and other functionality to functions in a modular and reusable way.

In this tutorial, we learned how to define and use decorators in Python, including how to define decorator functions and decorator classes, and how to use decorators to add caching to a function.

If you'd like a more in-depth tutorial about decorators, you should watch this awesome Pycon Talk

Overall, decorators are a valuable tool in your Python toolbox, and mastering decorators can help you write more modular, reusable, and maintainable code

This article was originally published on RealPythonProject.com on March 4th,2023.

How to Create a Testable CLI using TypeScript?

Rahul Banerjee — Mon, 30 Jan 2023 05:18:41 +0000

A CLI is a tool that allows users to interact with a computer program through the command line. It can perform various tasks, such as running commands, executing scripts, and interacting with other programs. This tutorial will teach you how to build a simple CLI in TypeScript. We will create a CLI that has the following commands

command print a name
command to perform basic arithmetic operations with unit tests
command to make an API call
We will also format our output by adding some color to it!
Finally, we will use mocha and chai to write unit tests for our CLI functions

Setup

Create a new folder and set up a project.

mkdir my-cli
cd my-cli
npm init -y

This will create a new directory called my-cli and initialize it as a npm project with default settings

Now, let's install some of the required packages

npm install typescript @types/node --save-dev
npm install commander

Commander.js is an NPM package that makes it easier to build CLI tools. You can find its documentation over here

Configure Typescript for your project by creating a tsconfig.json with the following

{
  "compilerOptions": {
    "target": "es6",
    "module": "commonjs",
    "outDir": "dist",
    "strict": true
  },
  "include": ["src/**/*", "test/cli.spec.ts"]
}

Create two folders src and test.

Inside the src folder, create a file named index.ts and add a Hello World Statment

console.log("Hello World!");

You can either compile the above file and use node to execute the result javascript file

npx tsc && node ./dist/src/index.js

or alternatively you can install tsnode and directly execute the typescript file

npm install ts-node --save-dev

npx ts-node src/index.ts

I recommend installing ts-node since it'll be used when setting up mocha for unit testing.

Commands

Hello World Command

Create a file named utils.ts, this will contain our utility functions

export const printName = (name: string) => {
  console.log(`Hello, ${name || "World"}!`);
};

It accepts a string as an argument. If no string is provided, it simply outputs Hello,World!. If a string is provided, it will output the name Hello,[name]!

Instead of console.log, you can also use other methods such console.error and console.warn. In this tutorial, we will only work with console.log.

Now go back to index.ts. We will create an instance of Commander and register our first command.

import { printName } from "./utils";
import { Command } from "commander";

const program = new Command();

program
    .name("tcli")
    .description("Learn how to build a CLI using TypeScript")
    .version("1.0.0");

program
    .command("hello [name]")
    .action(printName);

Update the name, description and version number of your CLI as needed. When registering a command, the following syntax is used

${command name} ${[arguement1]} ${[arguement2]} ${<arguement3>} ......

If an arguement/flag is required, it should be specified inside <..> and if it is optional, it should be specified inside [...] In this case, we want the arguement to be optional since we have a default output Hello, World!. You can alternatively register a command in the following way as well

program
    .command("Hello")
    .argument("[name]")
    .action(printName);

program.parse(process.argv);

Next we will pass the printName function as a callback method.

Now run the following in your terminal to run the CLI

npx ts-node src/index.ts hello

npx ts-node src/index.ts hello Jake

Now, let's add some color to the output

For this, we will have to install a npm package called Chalk

npm install chalk

The various color functions provided by Chalk take a string as an argument and return a string, so we basically need to wrap our strings in one of Chalk's functions and can directly use it with console.log

Go back to utils.ts and update the printName function as follows

import * as chalk from "chalk";

export const printName = (name: string) => {
  console.log(chalk.yellow(`Hello, ${name || "World"}!`));
};

This will print the output in yellow.

Try running the below command again to see the output

npx ts-node src/index.ts hello

You can even add the above command as a script in package.json

  "scripts": {
    "local-cli": "ts-node src/index.ts"
  }

Now you could run index.ts with the following command

npm run local-cli

If you are using the script from package.json, you need to use -- before adding the command

npm run local-cli -- hello

npm run local-cli -- hello Jake

Command to Perform Operations on Numbers

First, let's write the a function to perform various operations

export const performOperation = (
  num1: number,
  num2: number,
  options: { operation: string }
) => {
  let result: number;
  switch (options.operation) {
    case "multiply":
      result = num1 * num2;
      break;
    case "divide":
      result = num1 / num2;
      break;
    default:
      console.error(chalk.red("Invalid operation"));
      process.exit(1);
  }
  console.log(chalk.green(result));
};

It is a pretty simple function, it accepts a couple of numbers and the operation to perform. It has a switch case block to perform either multiplication or division. If any other operation is provided, it prints an error message in red and exits.

For this command we need 3 values, the 2 numbers and the operation.

import {performOperation} from "./utils";

program
  .command("calculate <num1> <num2>")
  .option(
    "-o, --operation <operation>",
    "operation to perform: multiply, or divide"
  )
  .action(performOperation);

program.parse(process.argv);

In the code above, we add an option for the operation. We can also provide a help message if the user uses -h flag.

Test the above using the following command

npm run local-cli -- calculate 1 2 -o multiply

The following command should return an error message

npm run local-cli -- calculate 1 2 -o add

Command to make an API call

To make the API call, we will be using axios

npm install axios

We will be using the agify API, it tries to predict the age of a name. Below is the function to make a request to the API

Like before, write it in the utils.ts file

import axios from "axios";

export const getAge = async (name: string) => {
  const response = await axios.get(`https://api.agify.io/?name=${name}`);
  const data = await response.data;
  console.log(`${chalk.yellow(data["name"])} - ${chalk.gray(data["age"])} `);
};

Registering the command is pretty straight forward

import {getAge} from "./utils";

program
    .command('age [name]')
    .action(getAge)

program.parse(process.argv);

Test it using the below command

npm run local-cli -- age rahul

How to make the CLI Tool "Global"?

Right now, we have to manually run the script to run our CLI Tool but this is not optimal. We want users to be able to install it as an npm package and then use a command like tcli to run it. Below is how you can add that functionality

Add the following shebang line at the top of your index.ts or the file where you created an instance of Commander

#!/usr/bin/env node

Now, update your package.json file by adding the following

  "bin": {
    "tcli": "./dist/src/index.js"
  }

They key is what the user would have the call to run the CLI and the value is the path to the compiled javascript file. In this case, the user would have to type tcli to run our CLI tool.

You can also test it by installing your package globally

npm i -g

Now if you type tcli in your CLI, it should run your CLI tool

tcli hello

You can uninstall the package as well

npm uninstall typescriptcli -g

You can also update the scripts in package.json with some more helpful commands

 "scripts": {
    "install-cli": "npx tsc && npm i -g",
    "uninstall-cli": "npm uninstall typescriptcli -g",
    "test-cli": "tcli hello",
  }

How to Test a CLI Tool?

We will use mocha and chai for unit testing. Install the required dependencies

npm i mocha chai @types/mocha @types/chai --save-dev

Create a folder called test and a file called cli.spec.ts inside it

Before any of the unit tests are run, we need to install our package globally so that our unit tests can test the CLI

import { before, after, test } from "mocha";
import * as path from "path";
import * as childProcess from "child_process";

before(() => {
  childProcess.execSync(`npm install -g`, { cwd: path.join(__dirname, "..") });
  console.log("Installed CLI");
});

after(() => {
  childProcess.execSync(`npm uninstall -g`, { cwd: path.join(__dirname, "..") });
  console.log("Uninstalled CLI");
});

We will use childProcess.execSync to install and later uninstall the CLI. Now in the same file, let's add some unit tests to test the hello command and the calculate command

test("should print hello world", () => {
  // Run the hello command and capture the output
  const output = childProcess.execSync(`tcli hello`, {
    encoding: "utf8",
  });

  // Check that the output is correct
  expect(output).to.equal("Hello, World!\n");
});

test("should print a greeting", () => {
  // Run the hello command and capture the output
  const output = childProcess.execSync(`tcli Hello Rahul`, {
    encoding: "utf8",
  });

  // Check that the output is correct
  expect(output).to.equal("Hello, Rahul!\n");
});

test("should perform the specified operation", () => {
  // Run the calculate command and capture the output
  const output = childProcess.execSync(`tcli calculate 3 4 -o multiply`, {
    encoding: "utf8",
  });

  // Check that the output is correct
  expect(output).to.equal("12\n");
});

This will only check the string value output in the console, it doesn't check the color of the output. Similarly you add an unit test for the age command as well.

Conclusion

A CLI is a powerful tool and I hope you found this tutorial useful. Here a few other articles that you may find interesting

https://www.realpythonproject.com/what-is-axios-create-in-javascript/

My Software Development Blogging Journey: A 2022 Retrospective

Rahul Banerjee — Fri, 06 Jan 2023 09:16:45 +0000

This is going to be a retrospective piece going over my Blogging stats and earnings in 2022. I will also include some tips based on my personal experience if you would like to consider Blogging either for a side income or as a hobby.

My strategy is pretty simple, I post my article on my wordpress blog and then cross-post it along with a canonical link on Medium, Dev.to, and Hashnode. So basically, it is the same content but it's being shown to different audience groups.

If you would like to read more about COPE, this is a great article.

I have even written a tutorial on how to automate cross-posting to various different platforms.

Blog #1 RealPythonProject Stats

This was my first wordpress blog and I created it in the Winter of 2020. I integrated it with Ezoic on 5th January 2022 and have been using Ezoic to display ads on it ever since.

Ezoic Analytics for 2022

In 2022, my blog got 168,402 views.

Initially, I had some trouble integrating Ezoic with my website and hence there are a few missing data points. In the month of April, a lot of my website pages became unavailable and sometime in mid-April, my entire website went down. It was due to some issues with Bluehost, my website's hosting service. The website's speed was affected as well and most pages were taking 15-20 seconds to load. After a bit of back and forth with the Bluehost Team, I managed to resolve the issue. However, for some reason, some of my articles got taken out of Google's Search Index, and as a result, I had to apply to re-index multiple of my pieces.

In the last few months of 2022, my view started dropping significantly. There were a couple of reasons for it

I started posting less content since I was pretty busy
Google had a core Search Algorithm update in September and I believe this affected my blog. As a result, the views started tanking.

Blog #2 RealJavascriptProject Stats

I created this blog in the summer of 2022. It hasn't grown as much as I would have liked. That is mainly because I have not been consistent in posting articles.

I don't have Ezoic or Google Analytics setup so below is a screenshot from Jetpack Insights

Jetpack Stats for 2022

In 2023, I will focus more on this blog and try monetizing it.

CrossPosting Stats

Medium

Unfortunately, there is no easy way to get yearly stats from Medium. So I'll have to sum up the views for every month. Below is a screenshot of my best month on Medium in 2022

Medium Stats for March'22

In 2022, I got a total of 140132 views which is almost on par with my main wordpress Blog. I had started off initially on Medium and managed to gain a few hundred followers.

I have been pretty inconsistent over the past few months but still manage to get around 10k views a month. Medium does a pretty good job at promoting your articles and a lot of top search results are often Medium articles. For anyone getting started, I highly recommend Medium as your first platform.

Dev.to

Unfortunately, Dev.to doesn't support showing yearly stats as well. Below is a screenshot of my stats from last month

Dev.to stats

I used the Dev.to API to get my stats for 2022. My Dev.to posts got 11499 views in 2022.

Hashnode

Unfortunately, Hashnode has not worked out for me.

Hashnode Stats

IMO, Hashnode has the best UI compared to other blogging platforms and comes with a ton of features. However, I think like me, most people use it only to cross-post their articles and hence there aren't that many readers. It is still fairly new as compared to dev.to and Medium. Or maybe I am just missing out on something. If you have managed to crack the Hashnode Algorithm and are reading this, please send me some tips.

Earnings

Now the good stuff!

Money Spent

I spent 156 USD on both my blog's domains, hosting, etc via Bluehost.

As I talked about in the previous newsletter, I spent 175 CAD on some SEO services on Fiverr.

I used Ezoic to display ads and since June I have been paying 36 USD per month to gain access to Ezoic Premium.

Total Money Spent on Blog: ~540 USD

Ezoic Ads

Ezoic Earnings for 2022

The drop from August'22 was due to 3 main reasons

The biggest reason was I decided to add hard limits on the number of ads to show per page. Initially, I did not set any ad number limits and earned around 100-200 bucks per month. However, in the long term, you need to find the sweet spot between the number of ads to display and also having a good user experience. I still haven't managed to figure it out, Ezoic has some helpful tools for this as well.
Lower views due to SEO issues
Didn't publish much during the last few months.

Medium Partner Program

Medium Earnings

Total Earnings in 2022: 980 USD

As I mentioned earlier, I only cross-post my articles on Medium. So each article of mine earns me via ads and via medium partner programs. Medium also allows you to refer users and you get a portion of the membership payment. I earn around 25 USD a month (included in the snapshot above) from referrals.

Most of my time on Medium is spent reading other writers' work.

Guest Posts on Real Python Project

Surprisingly, a couple of people reached out to me to guest post on my blog and were willing to pay 50 USD for it. I had 4 guest posts and got 200 bucks for doing nothing other than reviewing and posting their article on my blog.

Net Income from my Blog

Total spent: 540 USD

Total earned: 2494 USD

Net Income: 1954 USD

Tips for Getting Started with Software Blogging and Making a Side Income

TLDR: Cross-posting and Posting on Sub-reddits

Initially start by creating content on Medium and cross-posting on Dev.to/Hashnode. You will not have to deal with any SEO/Website issues and can simply focus on writing content.
Once you have gained a few followers on any of the above platforms, consider buying a domain and publishing over there. Don't stop cross-posting on Medium and other platforms.
Be consistent. As you saw from my stats when you stop posting, your views and earnings drop. It's hard to stay motivated and be consistent so you'll have to manage your expectations for earnings. The more consistent you are, the more you will earn.
Shamelessly promote your articles on subreddits. Just post it and forget about it. Reddit will lead to a lot of traffic to your articles.
Write about stuff you learned at work/school or challenges you faced at work/school. Most of my articles are about challenges I faced during my day-to-day life as a developer.

About Me

I recently graduated with a BaSc in Computer Engineering from University of Toronto. I moved to Vancouver shortly after graduation and started working as a Software Developer at Amazon. I like to play around with new APIs/ Libraries and write about them. You can connect with me on LinkedIn or Twitter

Affiliate Links

Note: If you use any of my affiliate links, I will earn some money at no extra cost to you.

Ezoic

Medium

How to Send and Receive Data in Flask

Rahul Banerjee — Mon, 14 Nov 2022 08:49:36 +0000

This article was originally posted on realpythonproject.com

I’ll be using Repl.it for writing all the code. Repl.it lets you quickly deploy your API and is really convenient. You can find my Repl here

Here are the 4 types of data we will be looking at

Sending/Receiving Parameters
Sending/Receiving Form Data
Sending/Receiving JSON String Data
Sending/Receiving Files

Why would you want to learn how to Receive/Send Data in Flask?

In a lot of cases when you are developing a Flask API that interacts with 3p services such as other APIs, Webhooks, Serverless Functions, etc, you won’t have control over what kind of data you receive. However, knowing how to receive different kinds of data in Flask will help you handle the data appropriately. If the 3p service’s source code is in Python, you can go over this article to find the relevant section. Being able to send different formats of data is also helpful since it’ll allow you to write unit tests for your API.

Project Setup

Install Flask using the following command

pip3 install flask

Below is a simple Flask API

from flask import Flask,request
app = Flask('app')

@app.route('/')
def hello_world():
  return 'Hello, World!'


app.run(host='0.0.0.0', port=8080)

The endpoint simply returns a string.

Parameters

How to Access Parameters from a Request in Flask

Below is a code snippet that can access the parameters that were sent.

@app.route('/receiveParameters')
def receiveParameters():
  data = request.args
  return {'data': data}, 200

You can use the request.argsto access the parameters. It returns the parameters as a dictionary.

How to Make a Request with Parameters Using Python

Below is the code snippet which makes a request to a URL and sends a couple of parameters.

import requests

url = 'https://FlaskReceive.rahulbanerjee26.repl.co'

response = requests.get(url+'/receiveParameters?name=Rahul&age=22')
print(response.json())


response = requests.get(url+'/receiveParameters',
params = {
  'name': 'Rahul',
  'age': 22
  }
)
print(response.json())

'''
OUTPUT
{'data': {'age': '22', 'name': 'Rahul'}}
{'data': {'age': '22', 'name': 'Rahul'}}
'''

There are two ways you can pass parameters, the first is to directly add it to the URL and the second is to add it in a JSON object.

Form Data

How to Receive Form Data in Flask

Flask has a property called request.form that lets it access form data. Below is an endpoint to receive form data

@app.route('/receiveFormData',methods=['POST'])
def receiveFormData():
  name = request.form['name']
  age = request.form['age']
  print(name)
  print(age)
  return{
    'data': {
      'name': name,
      'age': age
    }
  },200

It will accept POST requests and simply return the data.

We will look at two ways to send form data to the above endpoint. In the first way, we will create a simple form in HTML, and in the second way, we will send the form data using Python

How to send For Data using HTML to our Flask API Endpoint

I created a new repl for this. You can find it over here. Below is the HTML form.

<!DOCTYPE html>
<html>
<body>

<h2>HTML Forms</h2>

<form action="https://FlaskReceive.rahulbanerjee26.repl.co/receiveFormData" method='post'>
  
  <label for="name">First name:</label><br>
  <input type="text" name='name'><br>
  
  <label for="age">Last name:</label><br>
  <input type="text" name='age'><br><br>
  
  <input type="submit" value="Submit">
</form> 

</body>
</html>

For the action attribute, I set the value to the Flask Endpoint we created in the above section. We also set method to POST since the endpoint only accepts Post Requests.

This will create a form with two inputs and submit button. Whenever the user clicks on the submit button, the form will make a Post request to the receiveFormData endpoint.

How to Send Form Data using Python

This could be useful when you are writing unit tests to test your Flask API.

Below is a code snippet to test request.form

data = {
  'name': 'Rahul',
  'age': 22
}
print("Request to /receiveFormData")
formResponse = requests.post(
  url+'/receiveFormData',
  data=data,
  headers= {
    'Content-Type': 'application/x-www-form-urlencoded'
  })
print(formResponse.json())

The Content-Type is set to application/x-www-form-urlencoded

JSON StringData

JSON is something that is common to most programming languages so this might be the most common case.

How to Receive JSON String Data in Flask

from flask import jsonify

@app.route('/receiveJson',methods=['POST'])
def receivePostData():
  data = jsonify(request.json)
  print(data)
  return  data,200

We can use request.json to retrieve the data. To convert the json string object to JSON , we will use a Flask method called jsonify . This will let you access keys similar to how you would do with a dictionary.

How to Send JSON String Data using Python

import json

print("Request to /receiveJson")
jsonData={
  'name': 'Rahul',
  'age': 22
}
jsonResponse = requests.post(
  url+'/receiveJson',
  json=json.dumps(jsonData)
)
print(jsonResponse.json())

We will use json.dumps to convert the dictionary to a JSON String before attaching it as a payload to the POST request.

You can directly send the dictionary without converting it into a JSON string as well. In this case, you wouldn’t need to use jsonify it before returning the response.

Files

I have created a simple text file. We will attach the text file as a payload to the API endpoint. The API will read the contents of the file and return the content.

How to Receive Files in Flask

@app.route('/receiveFile',methods=['POST'])
def receiveFile():
  file= request.files['textFile']
  data = file.read().decode('utf-8')
  print(data)
  return {
    "data": data
  },200

request.files returns an FileStorage object, as a result, you can directly use read to get the content of the file. We will need to decode it before sending it as response data.

Working with files can get somewhat confusing, specifically the following line

data = file.read().decode('utf-8')

Depending on the format of the file, you might not have to decode it. Instead, you might have to use something like json.loads(byteData) to convert a byte object to JSON.

How to Send Files Using Python

print("Request to /receiveFile ")
with open('testFile.txt','r') as f:
  fileResponse = requests.post(
    url+'/receiveFile',
    files={
      'textFile': f 
    }
  )
print(fileResponse.json())

Depending on if you open it using mode r or rb , the code you wrote to create the endpoint will change.

Conclusion

In this article, we created 4 endpoints for our API. Each endpoint expected a different format of data. I hope you found this article useful. If you liked this article, you might like another article that helps you get comfortable working with APIs

How to Store Credentials as Environment Variables and Read in JavaScript

Rahul Banerjee — Sat, 12 Nov 2022 01:07:21 +0000

Secrets are called secrets for a reason. We do not want others to know about them, be it in real life or in JavaScript. In this tutorial, you will learn how to store your credentials as environment variables and learn how to read them in JavaScript. We will discuss 3 different ways to store credentials and read them using JavaScript.

Storing them as Global Environment Variables
Providing them as a Command Line Argument
Storing them in a .env file
Which is the best way to Read Credentials Locally in Javascript?

If you are interested in a similar tutorial for python, you can check out this article

Storing Credentials as Global Environment Variables

If the credentials are stored as Global Environment Variables, they can be accessed by any script running on your PC.

To Create a Global Environment Variable, run this in your terminal

export varName=varValue

Ensure there are no spaces between the ‘=’. If you get an error

“zsh: Bad Assignment”

bash: export: `=': not a valid identifier

it is probably caused because of space between the ‘=’.

Let’s create a couple of global environment variables

export globalUser=secretUser
export globalKey=secretKey

In Windows, you might have to use “set” instead of “export”.

Below is how you would access the variables in Node

const USER= process.env.globalUser
const KEY = process.env.globalKey

console.log(USER) // secretUser
console.log(KEY) // secretKey

process.env returns an object with all your Global Environment Variable.

You do not need to install any npm library since process is a built-in library in Node. The biggest con of Global Environment Variables is that you risk overwritting previously existing variables. As time goes by, it will be hard to keep track of previously created variables and avoid overwriting.

If you want to remove the environment variable you created earlier

unset globalUser
unset globalKey

Providing Credentials as Command Line Arguments

If you want to pass your credentials in the command line

globalUser=secretUser globalKey=secretKey node app.js

You could access them in Node the same you way you accessed the global environment variables

const USER= process.env.globalUser
const KEY = process.env.globalKey

console.log(USER) // secretUser
console.log(KEY) // secretKey

A caveat to doing something like this is that you will have to remember to pass the credentials as command-line arguments each time you run your script.

Storing Credentials in a .env file

Code Snippet showing how to read environment variables from a .env file

You can create a file named .env and store your credentials over there. Although you can store your credentials in any file and read them from your file, conventionally credentials are stored in .env files.

globalUser=secretUser
globalKey=secretKey

.env files are usually used to store secrets and credentials. Make sure to add it to your .gitignore file so that you don’t commit it by mistake.

To read variables from an .env file you will have to install dotenv which is a npm package.

npm install dotenv --save

dotenv can be installed as a dev dependency since it will only be used during development to load variables from .env files.

Below is the code to read the variables from the env file

require('dotenv').config()

const USER= process.env.globalUser
const KEY = process.env.globalKey

console.log(USER) // globalUser
console.log(KEY) // globalKey

As you can see most of the code is same as before. The only addition being the require statement at top.

Which is the best way to Read Credentials Locally in Javascript?

In my opinion, the best way is to store them inside a .env file.

They are local to your project
You do not need to worry about ‘unsetting’ them.
If you add them to your .gitignore file, it is safe from the outside world

The only con is that if there is a Global Environment Variable with the same name as a variable in the .env file, process.env will return the Global Environment Variable.

Do you know these Software Engineering Terms?

Rahul Banerjee — Mon, 17 Oct 2022 04:12:21 +0000

I recently saw this reel by tiffintech and I went on a ‘Google Ride’ searching for some more funny terms in Software Engineering

Software engineering is a complex field with its own language. If you’re new to the field, or even if you’ve been around for a while, there are terms that you may not be familiar with. In this blog post, we will explore some of these terms and what they mean.

Will they help you impress an interviewer? Probably not!

Ostrich Algorithm

Photo by Simon Infanger on Unsplash

So, what exactly is the ostrich algorithm? It’s a strategy that can be used when dealing with errors or other problems in your code. It is a strategy of ignoring potential problems on the basis that they may be exceedingly rare. Essentially, it involves ignoring the problem since it might be more cost-effective to leave the problem as it is as compared to spending time/error trying to solve it.

Of course, this isn’t an ideal solution, but it can sometimes be helpful if you’re in a pinch. It can be used to solve deadlocks in concurrent programming.

Yoda Conditions

Photo by Riku Lu on Unsplash

Yoda Conditions are a style of coding where the programmer writes conditional statements in reverse or in a non-traditional way. For example, instead of writing:

if (x == 5) {
   // do something
}

They would write:

if (5 == x) {
  // do something
}

If you are interested, there is actually an English to Yoda Translator

The thinking behind this is that it’s less likely for a programmer to make a mistake and write x = 5 by accident, which would cause an error. While this may seem like a small thing, it can actually help to prevent a lot of bugs in your code.

So next time you’re reading some software engineering code and you see an if statement written in reverse, you’ll know that isn’t incorrect and the programmer may just be a star wars fan.

Status 418 I’m a teapot

Photo by Joni Ludlow on Unsplash

The term “Status 418 I’m a teapot” is derived from the HTTP status code 418, which is defined as “I’m a teapot”. The code was originally meant to indicate that the server was a teapot, but it has been repurposed to indicate that the client is attempting to brew coffee with a teapot.

In simpler words, it is used to describe a situation where the client is trying to perform an action that is not possible. For example, if you try to brew coffee in a teapot, it will not work because the pot is not designed for that purpose. Similarly, if you try to do something that is not possible with a piece of software, it will also fail.

The term “I’m a teapot” can be used in both positive and negative situations. If you are trying to do something and you know it’s not possible, you can use the term to describe your situation. On the other hand, if someone else tries to do something that’s not possible and they succeed, you can use the term to describe their accomplishment.

In either case, the term “I’m a teapot” is a way of describing a situation where someone is trying to do something that cannot be done.

Bugs

Photo by James Wainscoat on Unsplash

Heisenbug

Heisenbug is a term used to describe a software bug that disappears or changes behavior when an attempt is made to study or debug it. The name “heisenbug” is a pun on the Heisenberg Uncertainty Principle.

Hindenbug

A bug with catastrophic behavior. Think something along the scale of your entire database being wiped off.

If you manually execute a SQL Delete statement without a where clause, it is more of a mistake than a bug. However, if you write a function to do the above, now that sounds like a Hindenbug.

The Gangnam Style Bug

When Psy release Gangnam style, he was about to literally break Youtube. Youtube initially stored its view count as a 32-bit number, i.e 2,147,483,647 views but Psy had different plans. Although Youtube did update its code before the 2,147,483,648th viewer saw a weird number of views, this deserves to be on the list of funny terms.

Youtube decided to move to a 64-bit counter so they are probably safe for a few more years. In case you are wondering, that's Nine Quintillion Views Lol.

Type of Code

Photo by Valery Fedotov on Unsplash

Jenga Code

Jenga Code is a term used to describe a style of programming that results in the whole thing collapsing when you touch a single block of code. Named after the game where players try to remove wooden blocks from a tower without them falling down.

If you ever get a feeling that you don’t know what that line/piece of code does but removing it breaks everything down, well you might be working with some Jenga Code.

Spaghetti Code

This term is used to describe code that is messy and difficult to read. It’s often the result of bad coding practices or a rushed development process.

If you’re working on a project with spaghetti code, it can be tough to make sense of it all. But with a little patience and perseverance, you should be able to untangle the mess and get things back on track.

Hydra Code

A little backstory about Hydra.

Hydra was the offspring of Typhon and Echidna, the father and mother of all monsters respectively. Hydra’s lair was the lake of Lerna, which was claimed to be the entrance to the UnderWorld. Heracles’ second of the set of 12 labors was to slay the Lernaean Hydra. Hydra had a regeneration feature: for every head chopped off, Hydra would regrow two heads.

Heracles eventually managed to cut off all of Hydra’s heads with assistance from Iolaus.

Hydra Code is code that simply can not be fixed. For every new “fix”, two new bugs get introduced. Your best bet would be to rewrite the code.

Magic Code

Magic Code is Code that is usually taken verbatim from websites like StackOverflow. Although it might solve your problem, you don’t know how it works or why it works. It magically solves your problem.

Common Law Feature

A bug in the application that has existed so long that it is now part of the expected functionality, i.e it is a feature

Pokemon Exception Handling

Photo by Thimo Pedersen on Unsplash

If you have watched Pokemon, you must be familiar with it’s catch-phrase

try:
    // Some Code
except Exception as Pokemon:
    // print("Catching all Pokemons", Pokemon)

Ash Ketchum might have never managed to catch all the Pokemons, but you can!

Yak Shaving

In software engineering, yak shaving is the term for when you have to do a bunch of small tasks in order to complete a bigger task. It is a task, that leads you to perform another related task, that leads you to perform yet another related task, and so on, and so on — all distracting you from your original goal. This is also sometimes called “going down the rabbit hole.”

Yak shaving usually happens when you’re trying to do something that depends on something else that isn’t ready yet. For example, if you’re trying to deploy a new feature to your website but your database isn’t migrated yet, you might have to migrate the database first. Or if you’re trying to write a unit test but you can’t because one of the classes doesn’t have an interface yet, you might have to write the interface first.

Yak shaving can be frustrating because it feels like you’re not making any progress on the main task — you’re just doing a bunch of busy work. But it’s important to remember that every task you complete brings you closer to your goal, and eventually you’ll get there.

Monkey Patch

Monkey patching is a technique used in software engineering to make changes to a program or library without changing the original source code. Monkey patches are typically used to add features or fix bugs in a program without waiting for the official release of a new version.

Monkey patching can be dangerous because it can lead to code that is difficult to maintain and debug. When doing monkey patching, it is important to be aware of the potential side effects of your changes.

Faith-based programming

When you try running the same piece of code again with hopes that the error just gets fixed magically.

Hi-driven development

For most languages, there are a lot of debuggers at your disposal. You can add breakpoints and do a lot more to make it easier to debug a problem. However, a lot of programmers do not use Debuggers. What we do is, we add random print or console.log statements. If you see a statement like the below randomly in a function, there is a good chance the developer was debugging by practicing Hi-driven development

console.log(`HI.... ${varValue}`)

Most of us have our own version of Hi , personally, I use Yooo and a bunch of -x-x-x-x-x-x-x-

Conclusion

Let me know if you know of any other such terms in Software Engineering.

Want to Connect?

Find me on LinkedIn: https://www.linkedin.com/in/rahulbanerjee2699/

Originally Posted on https://www.realpythonproject.com/do-you-know-these-software-engineering-terms/

How to Use Python to Post on Popular Blogging Websites

Rahul Banerjee — Sun, 25 Sep 2022 23:15:36 +0000

Create once, publish multiple times. Automate the manual work with Python 💯

In this article, we will use Python to automate the following tasks

Creating gists for your code snippets
Embed gists into the article with different requirements for Wordpress, Medium, Dev.to, Hashnode
Post the Article on WordPress
Post the Article on Medium with WordPress Canonical Link
Post the Article on Dev.to with WordPress Canonical Link
Post the Article on Hashnode with WordPress Canonical Link

Essentially we will be using Python to automate cross-posting to multiple websites. We will be using various APIs for doing the above tasks.

You can find all the code in my GitHub repo

Setting Up the Project 💻

First, create a new folder for the project

mkdir crossPoster
cd crossPoster

Next, create a virtual environment and activate it

python3 -m venv venv

source venv/bin/activate

Install the required libraries

pip3 install requests, Markdown, python-dotenv

Create a file named '.env' in your root directory. This will be used to store all your credentials. You will also need a test markdown file to be able to cross-post. In this tutorial, I am going to assume that all the images have public URLs and are not loaded from local files.

1️⃣ How to use GitHub API to create Gists for Code Snippets

Before proceeding, we will need a GitHub token. Go to your developer settings token page to get your GitHub token. When creating the token, make sure the option "Create Gist" is checked. This allows you to create gists using your GitHub token. Copy the token and save it in your .env file

GITHUB_TOKEN = "YOUR_GITHUB_TOKEN"

First, we will need to write a function to get all the code snippets from a markdown file. We will use a Regex pattern to get the code snippet blocks. For now, ignore the function "getSnippetName", just assume it returns a tuple consisting of the file name and actual code. If the code snippet has a language "bash", we ignore it. You can extend the list of languages to ignore.

We are going to assume that the user will begin their code snippet with a comment and the comment will have the name of the gist (basically like the above code snippet). If no such comment is provided, the gist will have a default name, Eg: index.js for a javascript file, init.py for a python file. Now let's look at the "getSnippetName" function

It searches for a comment in the first line of the code snippet, if a comment is present it extracts the file name from the first line. If no comment is present it uses a dictionary to return the default file name. You can extend the code to support comments in other languages.

Now we have an array of tuples, the first element in the tuple is the name of the gist and the second element is the content of the gist. We can use this to create our gists.

We will make a post request to the following endpoint

https://api.github.com/gists

If you want the created gists to be private, you can set the parameter "public" to False. The function will return all the URLs of the created gists.

Now we need to replace the code snippets in the markdown file with the relevant gist URLs. Medium, Wordpress, Dev.to, and Hashnode have different formats to embed gists so let's create a dictionary for that

Each of the lambda function will accept the gist URL as an argument and return the corresponding embed element. The logic to replace to code snippet is similar to the logic to get all the code snippets

We keep a counter to keep track of which code snippet corresponds to which gist URL. Since we ignored "bash" when creating the gists, we increment the counter by 1 without actually replacing anything. In all other cases we replace the code snippet with the gist URL at "counter" index and increment counter by 1. This function is going to create new files for Wordpress, Medium, Dev.to, Hashnode.

Check the below code snippet that uses the above-created functions. It will create public gists for the code snippets present in "index".md" and create a markdown for wordpress, a markdown for medium, etc

2️⃣ How to use the Wordpress API to Create a Post

You can find the documentation here

You will have to install a plugin to generate an application password for your account. Search for the plugin Application Password and install it. After you install it, go to your Users>Profile. There should be an option called Application Passwords and a button to Add new application password. Give it a name and create a new application password. Copy the new password.

Update your .env file

WP_USERNAME = "YOUR ADMIN USERNAME"
WP_PASSWORD = "YOUR NEW APPLICATION PASSWORD"

Your wordpress username is the one that is shown in Users>Profile.

We will create a class to post to wordpress. The wordpress api endpoint is of the following format

{Your Website Domain}/wp-json/wp/v2/{route}

Check out the below code snippet that creates the class.

Now will have to add some methods to this class that will be used to post to WordPress.

The first method will be used to upload our cover image or header image to Wordpress.

It has 3 inputs - the path to the image, alternative text, and the caption of the image. A post request has to be made to the following endpoint

{Your Website Domain}/wp-json/wp/v2/media

The response is going to have a field called id. This will be used when we are posting our article. It will have another field called jetpack_featured_media_url that contains the URL to the image.

Next, we need a method to create tag IDs for our article

It has one input - the list of tags. For each tag, it will try to create a new tag. The response is either going to contain the id of the newly created tag. If the tag already exists, the response will contain the existing tag id. We will combine the ids and return them.

A similar function can be created for categories as well.

Finally, we will add a method to post to wordpress.

The function logic itself is pretty simple, let's look at the json data that we will be adding to our post request

title: A string that is the title of your article
content: Your article's content (in HTML, I'll show you how to convert your markdown file to HTML)
status: A string "publish" or "draft" depending on if you want to post your article as a draft or not
categories: The Category IDs returned by the get_category_ids method
tags: The Tag IDs returned by the get_category_ids method
featured_media: The ID of the cover/header image returned by the upload_media method

To convert your markdown file to HTML, we will use a library called markdown

pip3 install Markdown

We will need to write the following helper function to do the conversion

Below is a sample usage of the wordpress class

If you are posting a draft, you can create a new variable to store the post slug. The post slug is the URL to your article after it has been posted. The slug should be present in the response object.

3️⃣ How to use the Medium API to Create a Post

We will be using Medium's REST API for this part. You can find the documentation here.

To get your access token, go to your profile settings and go to Integration Tokens. Enter a description and click on the 'Get Integration Token' button.

Save the token in your .env file

MEDIUM_TOKEN = "Your Meidum Token"

Similar to the class we created for Wordpress, we will create a class for Medium as well.

Now let's define the getUserID method.

This simply makes a GET request to the following endpoint and returns your user ID. Note: this is different from your username.

https://api.medium.com/v1/me

Now, we can add the method to create a medium post

Let's take a look at the json data added to the post request.

title: The title of your article
content: The content of your article in HTML
publishStatus: A string 'draft' , 'published' or 'unlisted'
contentFormat: In our case this will be 'HTML'
tags: A list of strings that are the tags of the article.
notifyFollowers: A boolean value
canonicalUrl: If applicable, the canonical URL. In my case, I am going to set it to the WordPress Post's URL.

Now let's take a look at a sample usage of the class

4️⃣ How to use the Dev.to API to Create a Post

The Dev.to implementation is pretty similar and I am not going to go much in detail. You can find your token here. Scroll to the bottom to generate a token and update your .env file

DEVTO_TOKEN = "GCP1itdmDAmVzfTjEQRRnnFc"

You can find the documentation here

Below is the dev.to class

For dev.to, we do not need to convert our markdown file to HTML. We can directly use the markdown content.

5️⃣ How to use the Hashnode API to Create a Post

Hashnode has a GraphQL API and it took me quite some time to figure out the correct query and how to make the correct request. You should be able to directly use the code snippet shown in this section. You can generate your token here.

You can play around with their API over here

Below is the implementation of the Hashnode class

If you are not familiar with GraphQL, try playing with the API in their API Playground. Let's take a look at some of the variables that are passed to the query

title: The title of your article.
content: Your article's content in markdown. Similar to dev.to, you do not need to convert to HTML.
publicationID: This is a required parameter. If you do not add it to your query, your article won't be listed under your profile but it'll still be posted. It'll be under hashnode.com and not your personal hasnode blog or account. To find your publication ID, go to your dashboard and copy it from the URL

https://hashnode.com/{YOUR PUBLICATION ID}/dashboard

tags: A list of tags
canonicalUrl: The Canonical URL of the article
hideFromHashnodeFeed: The API doesn't support posting drafts (or at least I couldn't find a way to post draft). Setting hideFromHashnodeFeed to True posts the article but it won't show up on anyone else's feed. You can toggle this in the hashnode editor as well.

Future Improvements

You can find a sample usage of all the above classes to create an "automated pipeline" in this file in the repo

You can think of this as a POC to automate your process. It obviously doesn't cover all use cases. Feel free to clone the repo and edit it as you like. Here are some improvements that can be made

Support for local images - The logic should be similar to how code snippets are converted to gists. Either one of the existing APIs can be used or a new API like imgurr can be used
Extend this to automatically share your article to various subreddits or Twitter as well
Add support for other blog editors

Please let me know in the comments if you found this article useful or if you have any suggestions on features that can be automated, improvements, etc.

This article was originally posted on 
https://www.realpythonproject.com/how-to-use-python-to-post-on-popular-blogging-websites/

Connect with me on LinkedIn: 
https://www.linkedin.com/in/rahulbanerjee2699/

Advanced SQL: Going beyond the Basics

Rahul Banerjee — Tue, 19 Jul 2022 04:15:38 +0000

Structured Query Language (SQL) makes a great addition to your toolbox. It may not be as popular as Python or JavaScript, but knowing SQL, especially advanced SQL, offers a number of benefits.

For one thing, knowing advanced SQL techniques can help you get better insights from your data, so that you can communicate your ideas more effectively. And since SQL is subject to the same performance bottlenecks and performance issues as other languages, a deeper knowledge will help you optimize your queries.

Finally, additional knowledge will help you stand out from other developers, who might know CRUD operations but not advanced SQL concepts. This gives you a career advantage.

This article will introduce you to some advanced SQL concepts, including subqueries, running totals, and common table expressions (CTEs). To follow along, you can use any database that supports SQL and an editor of your choice. You can also use an online SQL editor.

The following examples use an online SQL editor along with an SQLite database. If you are working with a database such as PostgreSQL, you might have to change the syntax; however, the underlying concepts are the same.

UNION, INTERSECT, and EXCEPT Operators

You can use sample data from this resource. In this example, all the commands have been combined into a single script in this repository. Run the script and it will create three tables with sample data. Below is the schema of the tables:

Image courtesy of w3resource

The first five rows of each table are shown below:

UNION, INTERSECT, and EXCEPT operators help you combine results from two queries. Operators in SQL are similar to math operators while working with sets. You must remember the following rule:

Below is the general syntax for using operators in SQL:

SELECT * FROM TABLE1
OPERATOR
SELECT * FROM TABLE2

UNION Operator

The UNION operator returns a combination of the tables from the two queries. By default, the result will not have duplicate values. However, you can use the UNION ALL operator if you want to keep the duplicates.

The following command will return a list of all distinct cities:

SELECT working_area FROM AGENTS
UNION
SELECT cust_city FROM CUSTOMER

The following command will return a list of all cities, including duplicates:

SELECT 
  working_area 
FROM 
  AGENTS 
UNION ALL
SELECT 
  cust_city 
FROM 
  CUSTOMER

Below is an image comparing the outputs of UNION and UNION ALL:

As you can see, in the right table, London and Bangalore are repeated.

INTERSECT Operator

The INTERSECT operator returns the common elements in the tables returned by the first and second query.

-- Returns a list of common cities in both columns
SELECT 
  working_area 
FROM 
  AGENTS 
INTERSECT 
SELECT 
  cust_city 
FROM 
  CUSTOMER

The result is shown below:

EXCEPT Operator

The EXCEPT operator returns all the elements in the table returned by the first query that are not also in the table returned by the second query.

SELECT 
  working_area 
FROM 
  AGENTS 
EXCEPT 
SELECT 
  cust_city 
FROM 
  CUSTOMER

The above command will return a table with a single row containing Vancouver. You’ll note that Vancouver is the only city that is in CUSTOMER.cust_city but not in AGENTS.working_area.

Subqueries

In SQL, you can have a query inside a query, also known as a subquery. A subquery can also have a query inside it. Subqueries take three main forms:

As part of a SELECT statement
After FROM
After WHERE

Subqueries help you break down complex joins and unions into smaller parts to make your query more readable and maintainable, especially since you can use results from existing queries. Subqueries also make debugging easier, because you can debug each query independently. Understanding subqueries also helps you write recursive SQL.

Subquery with SELECT

Look at the tables named AGENTS and ORDERS. You are tasked with returning a table with two columns. The first column should contain the agent_code from the AGENTS table, and the second should contain the sum of the ORDERS.ord_amount column. The second column should have the same value, or the sum repeated multiple times.

First, return the sum of all order amounts:

SELECT 
  SUM(ord_amount) 
FROM 
  ORDERS

The above command will be your subquery. You could use it as shown below to obtain the required result:

SELECT 
  agent_code, 
  (
    SELECT 
      SUM(ord_amount) 
    FROM 
      ORDERS
  ) total 
FROM 
  AGENTS

The command in the second line is the subquery. It returns the sum of the ORDERS.ord_amount column. You can also name the second column, either by using AS or by including your preferred name after the subquery. Below is a screenshot of the partial result:

Subquery with FROM

The following SQL command returns a table with the total number of order amounts, an average of order amounts, and the number of orders sold by an agent to a customer:

SELECT 
  agent_code, 
  cust_code, 
  sum(ord_amount) total_amnt, 
  avg(ord_amount) avg_amount, 
  count(*) AS num 
FROM 
  ORDERS 
GROUP BY 
  agent_code, 
  cust_code

You must use the result to find the total number of orders sold by each agent. You can use the above command as a subquery along with FROM:

SELECT 
  agent_code, 
  count(agent_code) AS count 
FROM 
  (
    SELECT 
      agent_code, 
      cust_code, 
      sum(ord_amount) total_amnt, 
      avg(ord_amount) avg_amount, 
      count(*) AS num 
    FROM 
      ORDERS 
    GROUP BY 
      agent_code, 
      cust_code
  ) 
GROUP BY 
  agent_code

The above command returns the total number of customers that each agent sold to. Instead of selecting from an existing table, you select from the table returned by your subquery. Below is the partial result:

Subquery with WHEN

Now use the AGENTS and CUSTOMER tables to return a table that only contains common names shared by agents and customers.

First, return a table with the names of every customer, using this command:

SELECT 
  cust_name 
FROM 
  CUSTOMER

Now use the IN operator and the table returned by the above query to find the common names. Below is the command, using the previous command as a subquery:

SELECT 
  agent_name 
FROM 
  AGENTS 
WHERE 
  agent_name IN (
    SELECT 
      cust_name 
    FROM 
      CUSTOMER
  )

The WHERE clause checks if the AGENTS.agent_name is present within the table returned by your subquery. It will return a table with two rows: Benjamin and Charles.

Common Table Expressions (CTEs)

In SQL, you create temporary variables to store the results from your subqueries. These variables are only accessible within the outer query. For the earlier example, in which you returned a table with the number of orders sold by each agent, you can store that result as a CTE to make your queries more readable.

WITH CTE_AGENT_CUSTOMER AS (
  SELECT 
    agent_code, 
    cust_code, 
    sum(ord_amount) total_amnt, 
    avg(ord_amount) avg_amount, 
    count(*) AS num 
  FROM 
    ORDERS 
  GROUP BY 
    agent_code, 
    cust_code
) 
SELECT 
  agent_code, 
  count(agent_code) AS count, 
  AVG(total_amnt) AS AVG, 
  SUM(total_amnt) AS SUM 
FROM 
  CTE_AGENT_CUSTOMER 
GROUP BY 
  agent_code

CTEs are required when working with recursive SQL.

Recursive CTEs in SQL

Recursion in SQL is like recursion in any other language. You divide a bigger problem into a smaller problem and keep on solving the smaller problem till you reach the terminating condition. If you don’t provide a terminating condition, it will send your SQL command into an infinite loop.

The following are two common problems that can be solved using recursion.

First N Numbers

Your task is to use recursion to print a table with numbers from one to twenty. Below is the recursive code:

WITH NUMBER_CTE AS (
  SELECT 
    1 AS num -- Starting Table
  UNION 
    -- Combine Starting Table with Table from Below Query
  SELECT 
    num + 1 as num -- Update Rule
  FROM 
    NUMBER_CTE -- CTE calling itself, ie recursion
  WHERE 
    num < 20 -- Terminating Condition
    ) 
SELECT 
  * 
from 
  NUMBER_CTE -- Displaying the recursive CTE

Consider the first and the second query of the UNION operator separately.

SELECT 
  1 AS num -- Starting Table

The above is a straightforward query that returns a column called num with the value of 1 in it.

SELECT 
  num + 1 as num -- Update Rule
FROM 
  NUMBER_CTE -- CTE calling itself, ie recursion
WHERE 
  num < 20 -- Terminating Condition

The above query increments the value of column num by one as long as the original value is less than 20.

In the first run, the result from the first subquery will be:

num
------
1
------

And the result from the second query will be:

num
------
2
------

The UNION operator will combine both tables and update your CTE to the following table:

num
------
1
------
2
------

In the second query, since you select from your CTE, the two queries along with the UNION operator will be executed again. The first query will again return:

num
------
1
------

The second query will use the CTE from the previous run and increment each row by one, returning the following:

num
------
2
------
3
------

The UNION operator again combines the tables from both queries. The updated CTE will be:

num
------
1
------
2
------
3
------

This process will keep going until the column contains a row with the value 20. In this case, the WHEN condition won’t be satisfied and you will stop the recursion process.

Factorial

Now try a SQL command that creates a table with factorial values. Try to find the factorial of 5.

WITH FACTORIAL AS (
  SELECT 
    5 AS NUM, 
    1 AS PRODUCT -- Starting Table
  UNION 
    -- Combine Starting Table with Table from Below Query
  SELECT 
    NUM - 1 AS NUM, 
    PRODUCT * NUM AS PRODUCT -- Update Rule
  FROM 
    FACTORIAL -- CTE calling itself, i.e recursion
  WHERE 
    NUM > 1 -- Terminating Condition
    ) 
SELECT 
  * 
FROM 
  FACTORIAL

Your initial table should have two columns—one to store the current multiplier and one to store the current product. The product will be 1 while the first multiplier will be 5.

The result from the first query will be:

num    product

------ ---------
5        1
------ ---------

The result from the second query will be:

num        product
--------   --------------
5-1 = 4     5*1 = 5
--------   --------------

The new CTE will be:

num    product
------ ---------
5         1
------ ---------
4         5
------ ---------

In the next run, the result from the first query will be the same:

num    product
------ ---------
5        1
------ ---------

The result from the second query will be calculated based on the CTE from the previous run:

num         product
---------   ---------
5 - 1 = 4   1 * 5 = 5
---------   ---------
4 - 1 = 3   5 * 4 = 20
---------   ---------

The UNION operator will combine both tables. The updated CTE will be:

num     product
------  ---------
5         1
------  ---------
4         5
------  ---------
3        20
------  ---------

This process will keep going until you reach the terminating condition.

General Syntax for Recursive CTEs

Based on the above problems, you can see the general syntax for recursive CTEs in SQL:

WITH CTE AS
(
    -- Starting Table
    UNION -- Combine Starting Table with Table from Below Query
    -- Updated Table with Update conditions if any
    FROM CTE-- CTE calling itself, i.e recursion
    -- Terminating Condition
)

SELECT * FROM CTE

The terminating condition doesn’t necessarily have to be a WHERE condition; it can also be a JOIN condition. You’ll look at terminating conditions with JOIN in the following cases.

Add Sample Data

Recursion is commonly used to find hierarchical structures in a company or when working with family trees. You’ll need to create a new table in your database. The first column will be the agent’s code, and the second will be their manager’s agent code. If an agent has no manager, the value in the second column will be NULL. Below is the hierarchical structure you will be adding to your database:

You can find the script to add the data in this repo.

Task One

Say you’re given an agent’s ID and are asked to return a table with the agent’s manager, skip-manager, skip-skip-manager, etc., all the way to the top. Below is the command to complete this task:

with HIERARCHY_CTE AS (
  SELECT 
    h1.agent_code, 
    h1.manager_code 
  from 
    HIERARCHY h1 
  WHERE 
    h1.agent_code == 'A006' 
  UNION 
  SELECT 
    h2.agent_code, 
    h2.manager_code 
  FROM 
    HIERARCHY h2 
    JOIN HIERARCHY_CTE ON h2.AGENT_CODE = HIERARCHY_CTE.manager_code
) 
SELECT 
  * 
FROM 
  HIERARCHY_CTE

Again, consider each query individually.

SELECT 
  h1.agent_code, 
  h1.manager_code 
FROM 
  HIERARCHY h1 
WHERE 
  h1.agent_code == 'A006'

The above query returns the agent’s code and their manager’s agent code. Below is the output:

agent_code   manager_code
---------   ------------
A006         A005

Now look at the second query:

SELECT 
  h2.agent_code, 
  h2.manager_code 
FROM 
  HIERARCHY h2 
  JOIN HIERARCHY_CTE ON h2.AGENT_CODE = HIERARCHY_CTE.manager_code

This performs a JOIN between the HIERARCHY table and the CTE you created. It tries to find the rows where the agent_code from HIERARCHY matches the manager_code for the CTE. It then returns the agent’s code and their manager’s code, or A006’s manager and A006’s skip-manager.

Below is the output:

agent_code  manager_code
---------   ------------
A005        A003

The union returns the following table:

agent_code  manager_code
---------   ------------
A006        A005
---------   ------------
A005        A003

In the second run, the first query will again return:

agent_code   manager_code
---------    ------------
A006           A005

The second query, however, will include A003’s manager as well. Below is the output:

agent_code manager_code
--------- ------------
A005        A003
--------- ------------
A003        A001

The UNION operator combines the tables. Your updated CTE will be:

agent_code    manager_code
---------    ------------
A006           A005
---------    ------------
A005           A003
---------    ------------
A003           A001

A001 has no manager. In the next run, you will hit the terminating condition.

Task Two

In this task, you’ll assign a level to each agent. The agents with no boss are level 1, the agents who report to them are level 2, and so on. Below is the command that returns this table:

with HIERARCHY_CTE AS (
  SELECT 
    h1.agent_code, 
    h1.manager_code, 
    1 AS level 
  from 
    HIERARCHY h1 
  WHERE 
    h1.manager_code IS NULL 
  UNION 
  SELECT 
    h2.agent_code, 
    h2.manager_code, 
    level + 1 AS level 
  from 
    HIERARCHY h2 
    JOIN HIERARCHY_CTE on h2.manager_code == HIERARCHY_CTE.agent_code
) 
SELECT 
  * 
FROM 
  HIERARCHY_CTE

This is similar to the command from your previous task. The main difference is that you start from the top agent, rather than the bottom.

Running Totals

Running totals are also known as cumulative sums. Consider the ORDERS table in your database. It has columns called ord_amount and ord_date. Try to calculate the cumulative sum on each date record in the ord_date column. Below is the command:

SELECT 
  ord_date, 
  SUM(ord_amount) OVER (
    ORDER BY 
      ord_date
  ) AS Running_Total 
FROM 
  ORDERS

The OVER command tells SQL that you want to find the cumulative sum of the ord_amount column over the whole data set. The column in the ORDER BY clause is used to sort the results by date.

CASE WHEN Statements

CASE WHEN statements in SQL are similar to if-else or switch-case statements in other languages. Below is the general syntax for CASE WHEN statements:

CASE
WHEN CONDITION THEN RESULT
WHEN CONDITION THEN RESULT
WHEN CONDITION THEN RESULT
ELSE RESULT
END Column_Name

The result in ELSE is returned if none of the conditions return true.

The CUSTOMER table has a column called grade with values from 0-4. You are tasked with creating a new column called grade_letter that maps 1 to A, 2 to B, and everything else to F. You can use CASE WHEN statements to complete this task. Below is the command:

SELECT 
  *, 
  CASE WHEN grade == 1 THEN 'A' WHEN grade == 2 THEN 'B' ELSE 'F' END grade_letter 
FROM 
  CUSTOMER

Conclusion

This guide offered you some hands-on experience in working with more advanced SQL topics. Implementing these tools will help you gain more insights from your data so that you can write better queries in SQL.

If you are working with SQL, you might be interested in CloudQuery. The SQL-based open source cloud asset inventory integrates with major cloud providers including Azure, AWS, and GCP. You can use it to audit and evaluate your cloud assets from a current and historical perspective.

For more on what CloudQuery can do for you, check out its documentation.

This article was originally posted on 
https://www.realpythonproject.com/advanced-sql-going-beyond-the-basics/

Connect with me on LinkedIn: 
https://www.linkedin.com/in/rahulbanerj

The concept of Triple CCC: Currying, Closure, Callback in JavaScript

Rahul Banerjee — Wed, 06 Jul 2022 04:05:38 +0000

JavaScript is a programming language that is used in both FrontEnd and BackEnd to build interactive web applications. It is the only scripted language that the browser can understand. As far today, almost 98% of websites use this language on the client-side for web page behavior. It is the most popular programming language in the world.

But the journey of this language was not so smooth. This language was designed by Brendan Eich and first appeared on December 4, 1995. At the initial moment, it was only used to bring interactivity to the web pages while the HTML and CSS are used to structure and design the web page.

The Game-changing moment happens when Ryan Dahl introduces NodeJS for the first time in 2010. It is nothing but an open-source, cross-platform, back-end JavaScript runtime environment that runs on the V8 engine and executes JavaScript code outside a web browser.

Since that moment this language has been used widely. In 2022, it is worth learning this amazing language to build a strong career in the Web Development sector. To make your learning journey smooth and also make your base solid, today, I will discuss the concepts of JavaScript those are Currying, Closure, and Callback (CCC).

No matter which position you are in now! If you want to make yourself a good JavaScript developer then you need to have a solid understanding of these topics. Moreover, it will help you to crack interview questions because most of the time there’s a common question that interviewers ask about these topics. At the end of this tutorial, you will get a clear understanding of Currying, Closure, and Callback in JavaScript.

Things You will learn from this tutorial

We will cover the following topics in this tutorial:

Function as First-Class Citizen
Currying
Closure
Callback

Function as First-Class Citizen

Before learning about the CCC’s concepts, you need to understand the concept of a First-Class Citizen. In JavaScript, a function is said to be a First-Class Citizen for the following three reasons

A function can be assigned as a value to a variable
A function can be passed as an argument to other functions
A function can be returned by another function

Let's take a look at some code snippets that show how a function in JavaScript can satisfy the above three reasons

A function can be assigned as a value to a variable

const myFunc = name => `Hello, ${name}! `;

const result = myFunc("Rahul");
console.log(result);

// Output: Hello, Rahul!

Here, you can see that we have declared an arrow function and later on, we have stored it into a variable named myFunc.

A function can be passed as an argument to other functions

const myFunc = (greetings,name) => `${greetings()}, ${name}! `;

const hello = () => "Hello"
const bonjour = () => "Bonjour"
const salut = () => "Salut"

const result_1 = myFunc(hello,"Rahul");
const result_2 = myFunc(bonjour,"Rahul");
const result_3 = myFunc(salut,"Rahul");

console.log(result_1);
console.log(result_2);
console.log(result_3);

// Output: 
// Hello, Rahul! 
// Bonjour, Rahul! 
// Salut, Rahul!

Instead of hard-coding "Hello" in our function myFunc, we now accept a parameter which is a function. As you can see, there are three different functions hello,bonjour and salut and each of them are passed as an argument to myFunc

A function can be returned by another function

const outerFunc = () => {
  console.log("Outer Function")
  const innerFunc = () =>{
    console.log("Inner Function")
  }
  return innerFunc
}


outerFunc()()
// OUTPUT
// Outer Function
// Inner Function


const myFunc = outerFunc()
myFunc()
myFunc()
// OUTPUT
// Outer Function
// Inner Function
// Inner Function

We have a function named outerFunc. It doesn't do much, it simply prints a statement to the console. Another function called innerFunc is created inside. The innerFunc also doesn't do much, it simply prints a statement to the console. The outerFunc returns the innerFunc.

There are a couple of ways this can be used. The first way is to use two braces together. The first pair of brackets invokes outerFunc and the second pair of brackets invokes the object returned by outerFunc, i.e the function innerFunc.

In the second way, we store the returned function in a variable and invoke it it like a normal function

Currying

In simple terms, Currying is a technique of chaining multiple functions. It’s a functional programming feature. Let’s understand this concept by solving a common interview question :

The Question

Look at the below code snippet.

function sum(x, y, z){
    return x + y + z
}

console.log(sum(10, 20, 30))

// Output: 60

The task you need to perform is to call the sum function like sum(10)(20)(30) instead of sum(10, 20, 30) this.

// Current way of Invoking function
sum(10, 20, 30)

// Desired way of Invoking function
sum(10)(20)(30)

How can you perform this action? The answer is simple, the term you need to use is Currying. See the below code example

function sum(x){
    return function(y){
        return function(z){
            return x + y + z
        }
    }
}

console.log(sum(10)(20)(30))

// Output: 60

Here, you can see that we have declared a function inside another and these have created a function chaining. You can also see we are getting exactly the same output and this process is known as Currying in JavaScript.

Currying is used in popular javascript libraries such as Lodash and Rambda

Closure

A closure in JavaScript is nothing but a function that references variables in the outer scope from its inner scope. Let’s see the problem that closure solves in the below section and by doing so, you will be able to know why we need

function sayHello(){
    let name = 'Rahul'

    function inner(){
        console.log(`Hello ${name}`)
    }

    return inner
}

let abc = sayHello()
console.log(abc)

// Output: 
// [Function: inner]
// Hello Rahul

Here, you can see that we are getting the output as expected. But the main question is how we are getting this? Let’s break it down in the below section:

At first, we declared a function named sayHello() and stored a string inside the name variable. Later on, we declared another function and print the name with a Hello message, and returned the inner() function. Finally, we have stored the returned function into the abc variable.

Now, when JavaScript will execute the code it will not find the name variable as the sayHello() function’s memory has been destroyed. But how we are getting the output as expected?

The main reason behind this is that Closure. Here, it is true that the sayHello() function’s memory has been destroyed but it creates a reference of that function and as a result, while executing the inner() it can access the variable from the reference of the sayHello() function. To perform this action, you don’t need to write one line of extra code. Because the JavaScript engine is smart enough to perform this action. All you need to do is to understand the concept of closure and it is created by default every time when a function has been created in JavaScript.

Callback

A callback is a function passed as an argument to another function. Let's look at a use case below.

Suppose, we have an array of names saved in our database. We want to add a new name and later on, we want to print the names with the newly added name. To perform this action we have written the below code:

let names = ['Rahul','Rohit','Rohan']

function addNewName(name){
    setTimeout(function(){
        names.push(name)
    },5000)
}

function printNames(){
    setTimeout(function(){
        console.log(names)
    },3000)
}

addNewName('Alex')
printNames()

// Output:
// [ 'Rahul', 'Rohit', 'Rohan' ]

Here, you can see that, we have added the functionality for adding new names but in the output, the new name has not been added. The main reason behind this is the asynchronous behavior of JavaScript. We know that for tasks that take much time, JavaScript does not wait for them, rather they work on other lighter tasks and keep the heavy task in the queue. The same things happen here.

We need to use a callback function so that we can break the asynchronous behavior of JavaScript and it can wait for us based on a specific task. See the below code example to understand it more clearly:

let names = ['Rahul','Rohit','Rohan']

function addNewName(name, cb){
    setTimeout(function(){
        names.push(name)
        cb()
    },5000)
}

function printNames(){
    setTimeout(function(){
        console.log(names)
    },3000)
}

addNewName('Alex',printNames)

// Output:
// [ 'Rahul', 'Rohit', 'Rohan', 'Alex' ]

We have passed a function (printNames) as an argument to another function (addNewName). This time printNames will execute only after the previous functions have been executed. This concept is known as the Callback and you can see that we have called this function after the previous function finished its tasks. That’s the reason we have mentioned Callback as Call later.

There are many advanced concepts that have been built on top of the Callback. You may also have heard about the concept of Callback hell. I'll be discussing this in one of the future articles.

Conclusion

In this tutorial, we have covered three essential concepts of JavaScript (Currying, Closure, and Callback). I hope you found this article helpful and informative.

This article was originally posted on 
https://realjavascriptproject.com/javascript-currying-closure-callback/

Connect with me on LinkedIn: 
https://www.linkedin.com/in/rahulbanerjee2699/

How to Share API Changes with Your Team

Rahul Banerjee — Tue, 05 Jul 2022 02:21:20 +0000

If your team is developing or updating an API as part of your software project, you need to be careful about making changes to that API. Those changes could cause unexpected bugs or failures and leave an unpleasant surprise for team members unaware of the changes. A lack of communication or coordination can turn even small issues into bigger ones for your organization.

It’s important to document and share API changes with your team as well as other stakeholders. In this article, you’ll learn some best practices you can use to share API changes and avoid bugs due to API updates. You’ll also learn about tools like Optic that you can use to track and share API changes.

What API Changes Cause Problems?

The following are some examples of API changes that could cause bugs in your software.

Changing the Response Object of an Endpoint

Assume you have a simple Flask API. On the left is your original API with a single endpoint, which returns information like ID, first name, last name, and full name. You decide that first name and last name are redundant since they can be derived from the full name using a simple split operation; on the right is the same endpoint with the updated response object.

If one of your team members consumes your API as shown in the code snippet below, the change to your API would break their code:

import requests

responseObject = requests.get(apiUrl).json()

print(f"Your id is {responseObject['id']}")

print(f"Your name is {responseObject['fname']} {responseObject['lname']}")

This sort of bug could also occur if you update the names or data types of some of the fields in your response object.

Changing the Name or Structure of an Endpoint

The two code snippets above differ by only a single letter. On the left, the endpoint uses user, while on the right it uses users.

If a team member consumes the API as shown below, the update would break their code, because the request would be made to a nonexistent URL:

import requests

responseObject = requests.get('www.api.com/user/1').json()

print(f"Your id is {responseObject['id']}")

print(f"Your name is {responseObject['name']}")

Updating the Response Code of an Endpoint

If you update the error response code for one of your API endpoints from 500 to 404, it could cause a bug. Consider the code snippet below:

import requests

responseObject = requests.get(apiUrl)

if responseObject.status_code == 500:
    # Handle Error
    pass
else:
    # Use data from API
    pass

If someone implements error handling but checks for a status code of 500, the new response code of 404 would not be handled. As a result, the code in the else block would be executed and lead to an error.

Making Optional Parameters Required

If you make optional parameters required, you could potentially run into another issue. Assume you have an endpoint that returns books from a book database. The endpoint accepts an optional parameter for the number of books to be returned, set to ten by default. If you later decide to make the number of books a required parameter, anybody who had previously used your endpoint without passing the optional parameter would face errors.

The above are just some of the possible problematic changes. Even updating things such as authentication or rate-limiting values can cause bugs in your team members’ code.

Why Share API Changes with Your Team?

In addition to the possibility of breaking changes, the following are other reasons to share changes with your team:

Team members could provide valuable feedback that might otherwise get lost. It's important to consider the potential implications of any API change before taking action, and having your team on board will help ensure that you're making the right choice.
They need to prepare for API changes. This might include testing the changes or making sure that their code is compatible with the new API. They need to be able to account for the changes in their code; otherwise, they could be caught off guard by unexpected issues or errors, which could damage relationships between team members.
Not only could changes break your API, but they could cause damage to other applications that use the API. If your team didn’t know about the changes, they wouldn’t be prepared to handle the consequences.

How Should You Share API Changes?

There are multiple ways that you can keep your team informed of changes. Here are some options.

Code Reviews

You can set up a code review system to ensure any updates to your API are reviewed by your team members before they’re finalized. This gives your team a chance to identify issues and suggest enhancements.

Tools like Optic allow you to review these types of changes. Below is a sample API changelog generated by Optic:

Image courtesy of Optic

Optic also offers a GitBot that can add API changelogs to every pull request you make:

Image courtesy of Optic

Communication Channels

One way to make sure that your team is aware of any API changes is to use a separate channel for sharing these changes—for instance, a Slack channel or an email list. This will ensure that everyone knows what to expect. Below are a few tips to manage such a channel:

Make it a read-only channel to avoid any discussions. Your team should be able to check notifications without having to scroll through other messages.
Ensure that potential breaking changes are clearly noted. You could add [IMPORTANT] at the beginning of your message or add a red background.
Communicate about such changes well in advance. This will give your team time to prepare for them.

API Documentation Updates

When it comes to API changes, having comprehensive and up-to-date documentation is essential. This will help ensure that everyone on your team understands the changes and knows how to implement them.

The documentation can either be updated manually or through automated services such as Swagger or Postman. Optic, for instance, analyzes requests, responses, and traffic to your API to more accurately document it and present changes. Read more about how Optic handles API changes in documentation.

API Contract Testing

If you are updating your API based on user feedback, you can work with your team or stakeholders to build an API contract describing how your API should function. You can use JSON schemas to document your API’s endpoint response codes and set up the contract. The contract must be accepted by all parties. Then, any changes made to the API must pass the contract test before being approved. Contract testing can be automated and incorporated into your CI/CD pipeline. Contract testing tools include Pact and Spring Cloud Contract.

Documentation-Driven Development

This is similar to test-driven development, but instead of writing tests first, you will write the documentation before developing your API. This means any changes made to your API will have to be documented first. You can follow OpenAPI standards to write the documentation, or use a service like Optic to compile the documentation instead of doing it manually.

Versioned APIs

Using versioned APIs allows team members and end users to continue working with an older version of your API while they update their code to work with the newer version. This significantly reduces the impact of any breaking API changes. But it comes at a cost, since you might still have to provide support for older versions. You will also need to maintain additional documentation to ensure users can still use older versions.

Ensure your API is backward compatible; if a user is able to use a previous version of your API, they should either be able to use future versions of your API without any changes or still be able to use the previous version. This can be done by providing support for both versions of the API and adding a deprecation warning to the old API to let the user know they will have to migrate soon.

You could also define your API URLs so that the version number is included. For example:

www.myapi.com/v1/getBooks

This would ensure that any updates or new releases of your API don’t break your team member’s or end user’s code if they use the previous version.

Conclusion

Even small changes to your API’s response object or URL can have a big impact on other team members or stakeholders. It’s important to keep your team informed. Having a good workflow to share API changes, document them, and test them will prevent errors or other problems down the line.

One way to improve your API development workflow is to use Optic to automate testing, documentation, and transparency of changes. The tool tracks and reviews API changes by generating a changelog, then uses requests, responses, and traffic to your API to generate documentation.

Want to know more? Optic is an open source project, so you can join the community working hard to make writing documentation easy

How to Make an AI-Assisted Phone Service with Twilio

Rahul Banerjee — Fri, 24 Jun 2022 02:51:07 +0000

This article was originally posted on

https://www.realpythonproject.com/how-to-make-an-ai-assisted-phone-service-with-twilio/

If you’ve ever called a business to book an appointment or tried to chat on a website to get your questions answered, there’s a good chance you have already interacted with an AI-assisted phone service. AI-assisted phone services have existed for a long time, and with the recent advancements in natural language processing (NLP), they have become more sophisticated.

AI-assisted phone services can be used in a number of different ways. For example, chatbots can be used to answer customer questions, while interactive voice responses (IVRs) can be used to take orders or book appointments. Additionally, AI-assisted phone services can be used to gather customer feedback and track customer satisfaction. This feedback can then be used to improve customer experience and ensure that customers are happy with the services they receive.

In this tutorial, you will build a bot that will be used to save a user’s information in a database. This bot will be built using Twilio and developed in Replit. Replit is a powerful browser-based integrated development environment (IDE) that lets you develop your projects online. You will be using various features provided by Replit, such as their unit testing and their live servers and database. Here is a high-level overview of how the bot will work:

What Are AI-Assisted Phone Services

AI-assisted phone services are powered by voice recognition software and NLP, meaning that they can understand the user’s voice and respond in a way that is natural for humans. This makes the user experience more like talking to a human than using a traditional automated phone service.

There are two general classes of AI-assisted phone services:

Personal assistant phone services: They can do things like provide information about local businesses, book appointments, and connect people with customer service representatives. They are helpful in cases where the user cannot look up the information themselves, such as when driving.
Customer service phone services: They provide customer support for businesses. They can take orders, book appointments, cancel reservations, and transfer people to the right department. They are helpful in cases where these tasks need to be done without human intervention or when it would be more practical to do so.

Here are some of the benefits of AI-assisted phone services:

They are typically cheaper than hiring multiple people to work in shifts and around the clock.
They have the ability to simultaneously handle multiple customers.
They can be programmed to automate tasks, like creating tickets and saving users’ feedback.
They enable users to experience a more efficient and less frustrating customer service experience.

As mentioned earlier, the most common AI-assisted phone services are chatbots and IVRs. Chatbots are computer programs that can mimic human conversation via text. They are commonly used to simulate a human’s conversation in online customer service. They are also used to communicate with customers on behalf of brands and businesses, such as in a phone app that lets people order food or ask for a local business’ operating hours. In Facebook Messenger, they can be added as a friend and sent messages to make exchanges feel more humanlike. Chatbots can also be used in Discord and Slack.

IVRs are computer programs that interact with people by voice. They are also commonly used to provide customer service. IVRs can be used to ask people questions about their accounts or to direct them to the right department, to provide information about a product or service, or even to take orders over the phone. A customer can call an IVR and press the appropriate buttons corresponding to the information they have or need, and the IVRs then automatically direct them to the right department or provide the details the customer has requested.

Many companies use these technologies in their online support structures. Although it does not completely eliminate the requirement of an actual person for customer support, it makes the process more efficient.

Building Your Own AI-Assisted Phone Service

This tutorial is divided into two sections. First, you will build your own AI-assisted phone service using Twilio. Second, you will build a Flask API to receive data from the phone service and store it in a database. All the development will be done using Replit. You can sign up for free here.

Here are some useful links for easy access:

Working with Twilio

You will need a Twilio account to follow along. After signing up, you should be redirected to your dashboard.

To start, go to Account > API keys & tokens.

You should see a button to create an API key, which you will need in order to consume Twilio’s API.

After you create your API key, you will need to save a couple of strings: the Account SID and the Auth Token.

Now, create a new Python project on Replit.

Normally, you would have to create an .env file in which to store your credentials. However, Replit has an alternative for this. Click on the lock icon on your sidebar.

Save your Twilio Account SID (account_sid) and Auth Token (account_key), and then scroll down. You should see an option to insert the code to read them using Python.

Twilio has a Python wrapper for their API, and Replit provides a simple way to install packages. On the sidebar, select the cube-like icon and search for Twilio. Click on the plus icon to install the package.

You will have to create an Assistant resource:

You can use the code to create the Assistant inside a try-except block to avoid creating a new resource every time you run the script. The code in the except block is run when an Assistant resource with the same unique name is created. In this case, you can simply fetch the existing Assistant resource. You can run the repl to create your own. By default, Replit will run the file named main.py. However, if you wish to, you can click on shell and run your file using the python3 command.

Replit has a really cool feature that can explain what a code snippet does. Simply highlight the code snippet and click on Explain Code. Note that you will need to upgrade to the Hacker Plan in order to use this feature.

Replit will analyze your code comment and generate an explanation for you. You can also create a thread to add comments (similar to how you would do in code reviews). This was the explanation generated for the highlighted code snippet:

1. The code creates a variable called the assistant and sets it to None.

2. The code then tries to create an assistant with the unique name "twilio-bot-repl-assistant".

3. If that fails, then the code fetches the assistant with the unique name "twilio-bot-repl-assistant".

As you can see, it is pretty accurate.

Next, you can add a task to the above Assistant. A task is basically an action your Assistant resource can perform. You can read more about it in the Twilio Docs.

Like the Assistant resource, you can use a try-except block again to avoid creating multiple tasks. Right now, an empty object is being passed to the actions parameter. This will be updated with the action you want your Assistant resource to perform. You can read more about the actions Twilio supports in their docs.

Now, we want the Assistant resource to ask the user for their name and phone number. For our use case, we will use the collect action. It is equivalent to an HTML form element in many ways.

Below is the syntax for a collect action:

You can add multiple actions, but for now, we just need a single collect action. You can add multiple questions to a single collect action. Here’s what mine looks like:

I used the built-in types provided by Twilio. Optionally, you can define your types by following these instructions. In the value for the key uri in redirect, you will notice a variable api_url. If you have a functioning API endpoint, you can create a variable named api_url and set it to your API’s URL. If you do not have a functioning API, you do not need to create a variable and can leave the value for uri as it is. You can follow the subsequent sections to build an API and update the value for uri later.

Now that the task is created, you will need to add sample phrases to trigger this task:

You can add multiple samples. Twilio recommends adding ten samples per task, but for this tutorial, one will suffice. Whenever the user says “register,” your Assistant resource will perform the collection action you declared earlier.

Now you can build your model:

This process will take some time. Head over to your dashboard and click on Explore Products > Autopilot.

Click on Your bots and check the Last model build time for your Assistant resource.

After the build process is complete, select your model and click on Simulator.

This will allow you to test your model. Since you do not have an API right now, the Assistant will simply collect the information and will not send it to an API. Turn on the Debug view to check which task is being performed by the Assistant. You can also see the values being stored in the variables you created using the collect action earlier.

On the right-hand menu, you should see an option called Channels. This will let you add your bot to a Twilio phone number so the bot can work as a chatbot or an IVR.

Building the API

To start, create a new Python project on Replit using one of their existing templates. In the main.py file, you should see text asking you if you would like to choose an example. Click on the examples link and select Server (Flask).

Click on the Run button to run the Flask server. Replit will automatically install all the required dependencies.

Normally, you would have to use a service like ngrok to expose your Flask server via a public URL, but Replit handles that for you. As long your server is running, it can be publically accessible by anyone via the URL shown on the right:

You can set your repl to Always On, which basically means that your Flask server will be running even if your close repl, so in a way, it deploys your Flask server. Note that this feature is only available on the Hacker Plan, so you will have to upgrade if you want to use it.

Now create a new endpoint that will accept both GET requests and POST requests. A GET request will return the users who have previously registered, while the POST request will save the information of the user. The Twilio bot you created in the previous section will make a POST request.

Replit has a built-in database that can be used to store the user information:

Some helpful code snippets to perform create, read, update, and delete (CRUD) operations will be shown when you click on the database icon, almost like a cheat sheet. Below is the code snippet to handle GET requests:

This code simply reads all the rows in the database and returns them as an array. Replit also allows you to add unit tests. You can add a unit test to verify that the endpoint handles GET requests as expected. The code in the image below creates a simple unit test that checks the response status code of the GET requests and compares the length of the data in the database to the length of the data returned by the GET request:

You will have to add a couple of import statements for your unit tests to work. Click on Setup and add the following imports:

Click on Run tests, and you should see a success message saying that all your unit test cases passed.

Now add the code to handle the POST requests:

Here, the Twilio bot sends a deeply nested object. You can print the data in request.form to check the format of the data. The above code gets the username and phone number, and stores them in the Replit database. Then it returns a Twilio action object so the bot can respond to the user once their information is saved.

Adding the API URL to Your Twilio Bot

Before testing the bot, you will have to add your API’s endpoint to the Twilio task you created earlier. This can be done using either the Twilio Console or the Twilio API.

Using Twilio Console

Go to the Twilio Console and navigate to your Twilio bot, as shown in the previous sections. Then go to Tasks and select the task you already created.

Initially, you had the following in your action:

Update the value for uri with your API’s URL. In my case, it looks like this:

Save the new task and build the model.

Using the Twilio API

Alternatively, if you’d prefer, you can update the API URL value using code. The task resource you created earlier has an update method that can be used to pass a new action with your API URL. You can read more about it in the Twilio Docs.

Go to the file where you wrote the code for your Twilio bot. In the section where you create a Task resource, replace assistant.tasks.create() with assistant.tasks.update(). This will ensure that your script will update the action instead of creating a new one. Update the JSON object to include your API URL. Once you are done, run the script again to update your action and build the model.

After you have added your API’s URL to your action resource—either via the Twilio Console or the Twilio API—your bot should work as shown below:

Conclusion

In this tutorial, you built an AI-assisted phone service using Twilio to collect a user’s name and phone number. You also built an API to save this information inside a database.

As you’ve likely noticed by now, Replit has a lot of useful features that allow you to code online. Although this article only showed its support for Python, it equally supports other programming languages, such as C++, JavaScript, and TypeScript; and frameworks, such as React or Vue.js. It comes with a built-in browser to preview your frontend or API. Additionally, it also has a built-in database and support for unit testing and storing environment variables.

Furthermore, Replit’s Explain Code feature is able to generate explanations for code snippets. Replit also has support for Git, allowing you to add commits or revert to previous ones as needed. Be sure to check out this powerful online IDE that can be used to make phone services, among many other things.