DEV Community: The CTO General 🦅

How to ship good code to production

The CTO General 🦅 — Sat, 26 Oct 2024 18:14:17 +0000

In my experience as a system developer, I’ve realized just how important security is—and it’s not something you can afford to overlook.

Recently, I witnessed a situation on X, when devs began discovering loopholes and vulnerabilities in a system that have been overlooked, and gone unnoticed for years. These weren’t just minor bugs; they were potential security risks, and they revealed just how vulnerable a system can become when tech debt piles up unchecked.

What started as a few issues quickly turned into a full-blown saga, with more and more security gaps being uncovered. Seeing this unfold firsthand made me reflect on how crucial it is to stay vigilant about security, especially with the rise of AI-generated code.

Developers are under more pressure than ever to ship code quickly, but that rush often leads to oversights. Rushed, untested code can open the door to attacks compromising a system’s integrity.

Bad codes are being shipped to production without thorough scrutiny, exposing the systems to various attacks.

From my own experience, I know how easy it is to fall into the trap of pushing code into production without enough scrutiny. But security issues, if not caught early, can escalate into serious problems. That’s why I’ve come to appreciate the importance of a structured, thorough code review process. It’s an essential part of ensuring that the systems I build are not only functional but also secure and resilient against potential attacks.

After watching the chaos unfold on X from the security vulnerabilities exposed, I felt the need to share a process that has helped me in my projects. It’s a straightforward approach that reduces the chances of bugs slipping into production by making sure every change to the codebase is carefully reviewed and aligns with best practices—especially when integrating new code into an existing system.

Effective Code Review Practices I Rely On:

• Keep Pull Requests Small: Small, focused pull requests are much easier to review and far less likely to introduce hidden issues. Large changes can be overwhelming and are more likely to cause problems that go unnoticed.

• Provide Clear Feedback: It’s important to give feedback that’s actionable and respectful. Instead of simply pointing out what’s wrong, suggest improvements. For example, instead of saying, “This is incorrect,” you might say, “Consider this approach for better scalability.” This keeps the discussion focused on improving the code rather than placing blame.

• Enforce Coding Standards: Using automated tools like ESLint and Prettier has helped me maintain consistency across my codebases. By handling style and formatting automatically, code reviews can focus on more significant issues, like functionality and security.

3 Key Reasons to Review and Test Code Before Shipping to Production:

Catch Bugs Early: Code reviews help identify bugs and performance issues before they reach production, saving you from bigger problems down the line.
Knowledge Sharing: When team members review each other’s code, it encourages collaboration and shared ownership of the codebase, which leads to stronger, more reliable systems.
Improve Code Quality: Code reviews enforce coding standards and best practices, ensuring that new code maintains the quality of the overall project.

In the systems I’ve worked on, following these practices has not only helped catch bugs early but also significantly improved security and overall code quality.

It’s about more than just preventing bugs—it’s about building systems that are reliable, scalable, and secure from the ground up. I hope these insights help you build stronger, more resilient systems in your work.

If you would like to collaborate with me on your project, reach out to me on X

Practical Tips for Developing Systems That Are Both Fast and Safe

The CTO General 🦅 — Thu, 17 Oct 2024 15:30:07 +0000

Have you ever wondered how to build a system that’s both secure and super fast? It’s something that a lot of people struggle with, and honestly, it’s easy to see why—there’s so much to consider and prioritize.

Lately, I’ve been getting a lot of questions about how to optimize performance, design secure systems, and which tools or strategies I would use. So, I thought, why not share my thoughts with you? After all, if you’re here, you’re probably dealing with some of these challenges too, right?

In this article, I’ll break down my approach and walk you through some practical strategies that can help you build systems that are both speedy and safe.

Let’s get into it!

Have you subscribed to my newsletter yet? Do so if you haven’t

Click here to subscribe

Performance: Keeping Things Smooth and Fast

We’ve all been there—waiting forever for an app to load, getting frustrated when pages take forever, or giving up on a site because it’s just too slow. Performance matters, and if your system isn’t quick, your users will feel it. So, how do we keep things fast and responsive? Let’s take a look at some strategies you can put to work right away:

Caching

Think of caching like giving your system a cheat sheet. Instead of asking the database for the same information over and over again, you store frequently accessed data in a “shortcut” memory, making it easier to grab next time. This is where tools like Redis and Memcached come in—they’re perfect for handling caching and speeding things up.

Database Indexing

Imagine trying to find a needle in a haystack. Now imagine the haystack has an index that tells you exactly where the needle is—pretty useful, right? That’s what database indexing does. By setting up indexes, you’re giving your system a way to quickly locate the data it needs, which can drastically improve performance.

Load Balancing

If too many users are hitting your app at once, things can slow down. But with load balancing, you can spread the work across multiple servers. It’s like having extra hands to help when things get busy. Tools like NGINX or AWS Load Balancer can make sure no single server gets overwhelmed, keeping everything running smoothly.

Content Delivery Networks (CDNs)

CDNs are a bit like placing “copies” of your website’s content in different locations around the world. When a user tries to access your site, they’ll download those files from the closest location, making everything load faster. If you’re serving images, videos, or other static files, a CDN can make a huge difference.

Asynchronous Processing

Not everything needs to happen right away. Some tasks, like sending emails or processing reports, can happen in the background without affecting the user’s experience. This keeps the app snappy while the heavy lifting happens behind the scenes. It’s all about keeping things smooth for your users.

Code Optimization

We all know the value of clean, efficient code, but sometimes we let things slip. Taking time to review and streamline your code—whether that’s reducing unnecessary complexity or fixing bottlenecks—can do wonders for performance. It’s like decluttering your workspace so everything runs smoother.

Horizontal Scaling

When your app starts growing and attracting more users, your servers need to grow with it. Horizontal scaling allows you to add more machines to share the workload, instead of beefing up a single server. It’s like opening up more checkout lanes at a busy store—it keeps everything moving quickly, no matter how crowded it gets.

Security: Keeping Your Data and Users Safe

We know that speed is important, but it’s equally crucial to make sure your system is secure. After all, what’s the point of a fast app if it’s not safe to use? Here are some key strategies to protect your system:

Encryption

Encryption is like putting a lock on your data. Whether it’s stored (at rest) or being transferred between systems (in transit), encryption ensures that only those with the right “key” can access sensitive information. Always use HTTPS for secure communication and encrypt things like passwords with strong algorithms like AES or RSA.

Authentication and Authorization

Ever worry about who’s accessing your system? You’re not alone. That’s why robust authentication and authorization are key. Make sure users are who they say they are with OAuth, MFA, or JWT. And don’t stop there—authorization ensures that users can only access what they’re supposed to.

Regular Security Audits

Think of security audits like health check-ups for your system. They help catch vulnerabilities before they become major issues. Regularly review your code, run penetration tests, and use tools like OWASP ZAP or Burp Suite to identify potential weak spots. It’s always better to be safe than sorry.

Data Validation and Sanitization

If you’ve ever built a system, you know that you should never trust user input. Always validate and sanitize the data coming from users to prevent attacks like SQL injection or cross-site scripting (XSS). Tools like Joi can help make sure inputs are safe, and query builders protect your database from harmful queries.

Rate Limiting and Throttling

No one wants their system to be overwhelmed by a flood of requests, especially from malicious sources. Rate limiting sets boundaries on how often users can make requests, protecting your system from DoS attacks and keeping things running smoothly for legitimate users.

Secure APIs

APIs are the backbone of many modern systems, so they need to be secure. Always use API keys, OAuth tokens, and enforce HTTPS. Limit the data exposed by your APIs and protect them from attacks like CSRF and XSS. A little extra effort here goes a long way in keeping your system safe.

Patch Management

Keeping your software up-to-date might seem like a small task, but it’s a big deal for security. Outdated software is a hacker’s best friend. Regularly applying security patches for your OS, libraries, frameworks, and dependencies is a must to avoid unnecessary risks.

Building a secure and fast system is all about balance. You can’t focus on performance at the expense of security, or vice versa. But with the right strategies in place, you don’t have to compromise on either.

I hope these tips help you create a system that’s both robust and responsive, giving your users the best possible experience.

Have questions or want to share your thoughts? I’d love to hear from you!

And before you go—if you haven’t subscribed to the newsletter yet, now’s the perfect time.

Click here to subscribe

Until next time!

Library vs Framework vs Package vs Module

The CTO General 🦅 — Fri, 19 Jul 2024 18:01:48 +0000

What are they? Are these terminologies different? What is the difference in this terminology?

It’s not a crime if you don’t have answers. These terminologies are often confusing, especially for newbies and even some senior engineers find it difficult to explain their differences.

This article was originally published in Tech Wrap Weekly newsletter

In software engineering, ‘library’, ‘framework’, ‘package’, and ‘module’ are constantly used to describe reusable code in the software. However, these terms can be used differently in different scenarios depending on the context in which they are being used.

In this article, I will demystify the subtle difference between a library, a framework, a package, and a module.

If you haven't subscribed to this newsletter, you should. Don’t miss out on interesting article topics like this, every week.

Subscribe now!!!

Now, that you have subscribed, let’s get to business.

Module

A single file or collection of files that encapsulate related code.

A module Is the smallest piece of software. It is a collection of methods and software that are ready to be reused somewhere else.

Modules are generally used to organize code into manageable, reusable pieces, into files, or a collection of files, promoting modularity and separation of concerns.

Example:

JavaScript: A single file that exports functions or classes using export and import statements.
Python: A .py file that defines functions, classes, and variables.

Package

Packages are simply a collection of related modules (it could also include multiple related files and resources)

Example:

JavaScript: An npm package like Axios, which is a promise-based HTTP client.
Python: A pip package like Requests, which is used for making HTTP requests.

Library

A library is a collection of related functionalities that you can use to perform specific tasks by calling its functions or methods. When you want to enhance your code with additional capabilities, you utilize a library.

Example:

JavaScript: Lodash is a popular utility library that provides various helper functions.
Python: NumPy is a library for numerical computations.

Framework

A framework is a collection of libraries, for developing a specific type of applications

They define the overall structure and flow of your application, making it easier to develop complex systems.

Example:

JavaScript: React (for building user interfaces) and Express (for building web servers).
Python: Django (for web development).

In Conclusion

Commonly, these terms are used interchangeably to refer to each other.

TLDR

Modules are the basic building blocks.
Libraries are collections of modules.
Packages are distribution units that can include libraries and modules.
Frameworks are comprehensive systems that include packages, libraries, and modules, along with additional tools and guidelines for application development.

Each concept builds upon the previous one, with frameworks being the most comprehensive and modules being the most granular.

The definitions of these terms can overlap or change depending on the programming language and context being used.

What change would you like to see in the next newsletter? Make your comment by replying to this article.

Subscribe to Tech Wrap Weekly newsletter here

Choose the right database for your needs: BASE vs ACID model

The CTO General 🦅 — Wed, 15 May 2024 12:20:11 +0000

Hi there, it's been a while

Today, I am excited to simplify the ACID and BASE models for anybody to understand how to make the right database decision. Continue reading and tell me if you enjoyed your reading.

In databases, transactions ensure data integrity by grouping multiple operations into one unit. How database transactions are handled is greatly dependent on the choice of database model that is being used.

ACID and BASE are transaction models that determine how a database manipulates and organizes transactional data.

The difference between a BASE and ACID database model lies in their unique approaches to handling data availability, consistency, and scalability.

In this article, we'll delve into the fundamental differences between these two models, examining their strengths, weaknesses, and suitable use cases.

Let's get to it, shall we?

Understanding ACID model: The high consistency model

ACID stands for Atomicity, Consistency, Isolation, and Durability. These properties guarantee that your data remains reliable and predictable across transactions. Let's break them down:

Atomicity: Transactions in ACID databases are atomic, meaning they either succeed entirely or fail entirely. There's no partial execution. This ensures data integrity even in the presence of failures.
Consistency: ACID databases maintain a consistent state, adhering to predefined constraints, rules, and relationships. This consistency is immediate and enforced at all times, across all distributed nodes, preventing anomalies or contradictions in data.
Isolation: Isolation ensures that concurrent transactions do not interfere with each other, maintaining data integrity and preventing issues like dirty reads or lost updates. All transactions are independent of other transactions.
Durability: Durability guarantees that committed transactions persist even in case of system failures. Once a transaction is complete, its effects are permanent and recoverable.

Examples of ACID databases: MySQL, PostgreSQL, Oracle Database, Microsoft SQL Server, etc.

Understanding BASE model: The high availability model

BASE is an acronym that represents a set of principles often associated with NoSQL databases. Unlike ACID, which prioritizes strict consistency, BASE offers a more relaxed approach, emphasizing availability and partition tolerance over immediate consistency.

BASE stands for Basically Available, Soft State, and Eventually Consistent. Here's what that means:

Basically Available: The focus is on keeping the database operational even during partial failures. Imagine a social media platform. BASE allows continued user activity even if updates take a moment to fully propagate across all servers. BASE systems are always available.
Soft State: Data can be temporarily inconsistent during updates. This might not be ideal for financial transactions, but for social media feeds that update constantly, a slight delay can be acceptable.
Eventual Consistency: Over time, all replicas/nodes of the database will eventually reflect the same data. This ensures that while temporary inconsistencies might occur, they are resolved eventually.

Examples of BASE databases: MongoDB, Cassandra, DynamoDB, Redis, etc.

Conclusion

In summary, BASE database systems offer availability and partition tolerance, while ACID-based system offers reliability and partition tolerances.

The choice of database depends on the specific requirements of your application. Once the requirement has been laid out and understood, the choice of database can then be made accordingly.

Hope this article, helped you to understand and weigh the metrics used to determine the right choice of database for your application.

Be sure to ask your questions in the comments. I will reply to all.

Connect with me on where I share regular software engineering updates:
Twitter- @thectogeneral
LinkedIn - Salem Olorundare

Build a search application with React and Typesense

The CTO General 🦅 — Mon, 04 Apr 2022 00:43:01 +0000

One common feature most applications have is the search feature. This feature allows the user to search through different collections of products or databases. Some applications take more time than they should, to search through the database for simple products or any other resource, to get the results to the user in due time.

E-commerce platforms like Amazon, eBay, and Flipkart make use of the search feature to enhance the user experience and get the right product. displayed to its customer.

Search-based applications are very common as they help to query the database faster with little or no technical experience through a search box.

What is Typesense?

Building a fully functional search feature from scratch takes a lot of time, energy, and resources to create. Managing the user input and search application against cyber attacks can be daunting.

Typesense is a free and open-source search engine for instant search-as-you-type experiences for application users. Typesense reduces the need for developers to build a search engine for each web application from scratch. This contributes vastly to a developer’s productivity when developing applications.

Sometimes, typo errors from the user can lead to errors in search results. Typesense has a typo detection feature that detects errors and makes sure the right and expected result is returned it the user. Typesense is also incredibly fast when it is being queried.

With Typesense, developers can easily add a search feature to their application with less and reduced effort.

Setting up a Typesense account

Let’s begin by setting up a Typesense account to connect with our React application

Step 1: Create an account

Typsense makes it easy to create an account with Github. Let’s head over to Typesense to create an account and get our API keys.

Step 2: Create a new Cluster.

After creating an account, we would need to create a cluster to manage our different collections of data.

Leave the defaults and click the launch button to create the cluster.

Clicking on the launch button will get Typesense to start creating the cluster. This would take a couple of minutes to get the cluster created as it is shown in the image below.

After the cluster has been created, we can then create our API key and get our host name. We would need this information to connect our app with Typesense.

Note: We would be using Typesense cloud to connect our app with Typesense, as it is an easier way to connect with Typesense.

Below is the host name and port number we will be using. Store this information somewhere you could easily have access to.

Step 3: Get API keys

Click on the Generate API keys to get the API key for our app. This will download a .txt file where you will find all information to connect your app with Typesense. This includes the API keys, host name, and also port number.

Note: We will be making use of the Search only API key for this tutorial.

Now, we are all set to connect our app with Typesense. Yipee!!!

Connect Typesense to React application

We would be creating a simple search application with Typesense and React. Let’s have our React app created first.

Create a React app with the following commands and install these dependencies.

Create a React app

npx create-react-app book-search-app
Install Typesense and react-instant-search

npm install typesense

`npm install react-instantsearch-dom`

`npm install typesense-instantsearch-adapter`

These are the libraries we would use to build our search application. The react-instantsearch used in this tutorial is a react open-source UI library that lets you quickly build and integrate a search interface(UI) into your application. We would see how to use it in the tutorial.

Now that we have our React application setup, we can start to connect with Typesense.

To connect and make changes to our cluster we will need to set up a Typesense client. The Typesense client would allow us to run commands to create clusters, create collections and also build schema, which holds the structure of how data are to be stored in the collection.

Creating and building Typesense schemas in React

To create a Typesense client and build our book schema(book structure), create a file in your React root directory and give it a name of your choice, in my case I gave it a name with loadData.js. This file will hold all the configurations needed to connect and create our Typesense client.

Before we continue, let’s get our books.json file ready. This file is a collection of different book titles and their descriptions. This is the file we want to query and search through to get information faster. Click here to download the book.json file.

Now, let us create a self-executing function so that when our app begins to run, our function also runs. Copy and paste the code below into your app.

// loadData.js
const Typesense = require('typesense');

module.exports = (async () => {
    //Configure Typesense
  const TYPESENSE_CONFIG = {
    nodes: [
      {
        host: process.env.TYPESENSE_HOST, // For Typesense Cloud use xxx.a1.typesense.net
        port: '443', // For Typesense Cloud use 443
        protocol: 'https', // For Typesense Cloud use https
      },
    ],
    apiKey: process.env.TYPESENSE_APIKEY,
  };

//Create and instance of Typesense client
  const typesense = new Typesense.Client(TYPESENSE_CONFIG);

// Build Schema
  const schema = {
    name: 'books',
    "fields": [
      {
        "facet": false,
        "index": true,
        "name": "title",
        "optional": false,
        "type": "string"
      },
      {
        "facet": true,
        "index": true,
        "name": "authors",
        "optional": false,
        "type": "string[]"
      },
      {
        "facet": true,
        "index": true,
        "name": "publication_year",
        "optional": false,
        "type": "int32"
      },
      {
        "facet": false,
        "index": true,
        "name": "ratings_count",
        "optional": false,
        "type": "int32"
      },
      {
        "facet": false,
        "index": true,
        "name": "average_rating",
        "optional": false,
        "type": "float"
      }
    ],
  };

// Import book.json
  const books = require('./dataset/books.json');

//Checks if the collection exists
  try {
    await typesense.collections('books').retrieve();
    console.log('Found existing collection of books');
  } catch (err) {
    console.error(err);
  }

// Create Booke schema
  await typesense.collections().create(schema);
  console.log('Creating schema...');

//Upload book.json to Typesense Database
  try {
    const returnData = await typesense
      .collections('books')
      .documents()
      .import(books);
  } catch (err) {
    console.error(err);
  }
})();

Let's break this down

To make the Typesense library accessible and usable we imported the typesense library
We then created a const to store our Typesense configuration. This configuration includes the setting up of the host, port, protocol, and apikey. All these parameters are in the file we downloaded from Typesense earlier when we created our cluster.
Then we made a new instance of Typesense client, using the Typense we imported earlier in the code.
To create the schema for our book collection we created a const and put in the structure of each field that each of the books in our data provides us with. Then we created this schema with the typesense.collections().create(schema) command.
Finally, we uploaded our books.json file to the Typesense database so that this file can be indexed when we make searches.

To make this function self-executing, navigate to your package.json file and add "indexer": "node loadData.js" to the script option. Now, this function will run each time the application runs.

With the code above, we have successfully created and configured our Typesense client. Now we can move on to building a search UI to query this data and get different results from each search that we make.

That is all for the first part on how to set up and configure Typesense with your React application.

Now, let’s dive into the section, where we would start querying our Typesense and React application.

Building Search UI with React-instantsearch and Typesense

In this section, we would create a simple UI to display our search input and our results. Copy and paste this code into your App.js file.

import React from 'react';
import './style.css';
import {
  InstantSearch,
  SearchBox,
  Configure,
  Hits,
  Pagination,
} from 'react-instantsearch-dom';

import TypesenseInstantSearchAdapter from 'typesense-instantsearch-adapter';

const typesenseInstantsearchAdapter = new TypesenseInstantSearchAdapter({
  server: {
    nodes: [
      {
        host: procsess.env.TYPESENSE_HOST,
        port: '443',
        protocol: 'https',
      },
    ],
    apiKey: process.env.APIKEY,
  },
  additionalSearchParameters: {
    query_by: 'title',
  },
});

export default function App() {
  const Hit = ({ hit }) => {
    return (
      <div className="hit">
        <div className="hit-image">
          <img alt={hit.authors} src={hit.image_url} />
        </div>
        <div className="hit-content">
          <div className="hit-price">{hit.title}</div>
        </div>
      </div>
    );
  };
  return (
    <div>
      <h1>Search Books</h1>
      <InstantSearch
        indexName="books"
        searchClient={typesenseInstantsearchAdapter.searchClient}
      >

        <SearchBox autoFocus />
        <Configure hitsPerPage={5} />
        <Hits hitComponent={Hit} />
        <Pagination />
      </InstantSearch>
    </div>
  );
}

Let’s explain the code above

We made the necessary imports to set up Typesense and use instantsearch
TypesenseInstantSearchAdapter configuration. The TypesenseInstantSearchAdapter helps us to to configure Typesense for any search query.

We have configure the host, port, protocol and the apikey parameter.

The additionalSearchParameters is used to configure search parameters. i.e where to search how to do a search when it is queried.
The <InstantSearch /> component allows us to wrap all our react-instantsearch components to give us the UI display: the search box and the results. Add the search components to your app and configure it with the right parameters.
To display and make a simple search bar we used the <SearchBox /> that we imported earlier. This should display for us a search bar to start making our queries.
The <Hits /> components is used to to display the search results.

If you have made it to this point, congratulations, you have successfully created a search application with Typesense and React. Now, let’s test our application to see if everything works fine. Run your React app and see if you have the following displayed on your screen.

As we can see, a list of book images and their titles are being displayed when we make a search through the search box that is provided to use by the react-instantsearch. These data (images and titles) are gotten from the books.json file that we uploaded to the Typesense database earlier.

Conclusion

With this, we have successfully integrated Typesense to our React application and built a search feature or search application.

Typesense is a very useful tool for developing application features within the best possible time. There are lots that we can do with Typesense as a search engine. This is a tutorial to get you familiar with Typesense and how to use it with your Javascript(React) application.

Check out the documentation here to learn more about Typesense.

Node Js Authentication with Auth0

The CTO General 🦅 — Mon, 07 Feb 2022 05:45:43 +0000

Hello Guys, You must have heard of Auth0 a quite number of times but you are not sure what it is. Or probably you might have known what it is but you have no idea how to use it. In this tutorial, I will show you what Auth0 is, what it is used for, and how you can also use it to speed up your development process. Without wasting much of your time, let's get to it.

What is Auth0?

Auth0 is easy to implement, adaptable authentication and authorization platform that creates login boxes to authenticate web application users.

Authentication is a system that is used to identify a particular application user through signing in with either an email, a username, or anything that can be used to identify the user.
So, imagine you wanted to create an application, as a developer, a lot of things are required of you to build for this application to be secured.

Users need to prove their identities through email and passwords to be able to access the resource from the application. Without these authentications passed the user won't be allowed to log on to the site.

As a developer, this requires you to create an interface on the frontend to collect data and send it to the server to save it on a database.

Creating these functionalities and integrating this into your application can be very tasking and time-consuming.
From logging in and logging out, to social medial signing in and signing out to even sign-ups. It takes a lot of time to build these functionalities.

This is the reason why auth0 was built.
To allow developers to authenticate their users with a ready-made authentication system, with beautiful Sign in, Sign out, and Signup functionality all embedded in one single piece.
Auth0 provides all these functions to make your application secure in one piece.

How does Auth0 work?

Auth0 allows developers to authenticate and authorize services to their web application through its API.
Auth0 works in a simple way.

When Auth0 is integrated into an application, the application then redirects the user to auth0 customizable login page, where it required that the user logins in before further usage.
Auth0 then takes them back to the app after successful login.
Auth0 sends user data to the application server which the developer can then use to know more about its user and perform some computations with this data.

How to integrate Auth0 into your web app

These simple steps will guide you on how to integrate Auth0 into your Node application to authentication your user and use the data that is being sent to the server by Auth0:

Step 1: Create an account on Auth0 by clicking here

Step 2: After successful registration, you will be redirected to your Auth0 account dashboard. Click on "Application" as shown below.

Step 3: To create a new application click on the "Create Application" Button.

Step 4: Name your application, Select the "Single page web application" and click on create.

Step 5: Go to your settings tab and Copy all necessary information we will be using for our application out i.e Domain and the ClientID.

Now that we have set up our Auth0 application, we build our Node application.

Step 1: Create your application folder and let's run npm init to start a Node app.

Step 2: Run the following commands to install the dependencies that we need to build the Auth0 authentication system.

npm install express express-openid-connect dotenv

express-openid-connect allows us to be able to use Auth0 services on our application
Step 3: Create and app.js file and put in the following code:

const express = require('express');
const { auth } = require('express-openid-connect');
require('dotenv').config();

const config = {
  authRequired: false,
  auth0Logout: true,
  secret: process.env.SECRET,
  baseURL: process.env.BASEURL,
  clientID: process.env.CLIENTID,
  issuerBaseURL: process.env.ISSUER
};

const app = express();

app.set('view engine', 'ejs');

app.use(express.urlencoded({ extended: true }));

// auth router attaches /login, /logout, and /callback routes to the baseURL
app.use(auth(config));

// req.isAuthenticated is provided from the auth router
app.get('/', (req, res) => {
  console.log(req.oidc.isAuthenticated());
  res.render("index", {isAuthenticated: req.oidc.isAuthenticated()});
});

app.listen(3000)

We first made some imports in this file so that we will be able to use Auth0.
Then we configure Auth0 with the const config. This is all that is needed to configure your Node application to work with Auth0.
Secret: Just a randomly generated string
baseURL: This is the URL that we will use to test out our application
clientID: This is the ID that Auth0 gave to us on the application dashboard
issuerBaseURL: This is the URL that was given to us by Auth0 earlier.

Step 4: For how Front end, Create a folder called view and open and new file in this folder called index.js with the following code:

<!doctype html>
<html lang="en">
  <head>
    <!-- Required meta tags -->
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">

    <!-- Bootstrap CSS -->
    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0/css/bootstrap.min.css" integrity="sha384-Gn5384xqQ1aoWXA+058RXPxPg6fy4IWvTNh0E263XmFcJlSAwiGgFAW/dAiS6JXm" crossorigin="anonymous">

    <title>Authentication with Auth0</title>
  </head>
  <body>

  <nav class="navbar navbar-light bg-light justify-content-between">
      <a class="navbar-brand">Auth0</a>
      <form class="form-inline">
      <% if(!isAuthenticated) { %>
        <a href="/login" class="btn btn-outline-success my-2 my-sm-0">Login</a>
        <% } else { %>
        <a href="/logout" class="btn btn-outline-danger my-2 my-sm-0">Logout</a>
        <% } %>
      </form>
  </nav>



    <% if(!isAuthenticated) { %>
        <div class="row d-flex justify-content-center align-items-center flex-nowrap" style="min-height: 100vh;">
            <h2>You are not Logged In</h2>
        </div>
        <% } else { %>
        <div class="row d-flex justify-content-center align-items-center flex-nowrap" style="min-height: 100vh;">
            <h2>You are Logged In</h2>
        </div>
    <% } %>


    <!-- Optional JavaScript -->
    <!-- jQuery first, then Popper.js, then Bootstrap JS -->
    <script src="https://code.jquery.com/jquery-3.2.1.slim.min.js" integrity="sha384-KJ3o2DKtIkvYIK3UENzmM7KCkRr/rE9/Qpg6aAZGJwFDMVNA/GpGFF93hXpG5KkN" crossorigin="anonymous"></script>
    <script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.12.9/umd/popper.min.js" integrity="sha384-ApNbgh9B+Y1QKtv3Rn7W3mgPxhU9K/ScQsAP7hUibX39j7fakFPskvXusvfa0b4Q" crossorigin="anonymous"></script>
    <script src="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0/js/bootstrap.min.js" integrity="sha384-JZR6Spejh4U02d8jOt6vLEHfe/JQGiRRSQQxSfFWpi1MquVdAyjUar5+76PVCmYl" crossorigin="anonymous"></script>

  </body>
</html>

Now our application is all set and ready to go.
Note that the URL link for Login and Logout are '/login' and 'logout' respectively. This is the convention Auth0 has made us use.

Testing your application

To test our application, run the following command in your terminal and open http://localhost:3000/ on your web browser.


node app

If you have done everything right, your page should look like this:

Now try the login and logout funtionality and see how Auth0 has helped us to achieve Authentication functionality within minutes.

Congratulations!!!

Conclusion

Auth0 has made building and adding authentication funtionality to our app very much easy. In this article, we were able to add this feature to our app with minutes. Although, Auth0 offers more functionalities for your applicatio. Authentication is a good way to start with Auth0.

This codes used in this tutorial can be found on my Github repo. Click here to access it.

Also follow me on {Twitter](https://twitter.com/itz_salemm) and ask you questions if you have any as I take web devevoplopemt tutorial series daily.
@itz_salem

Cheers.

What are APIs? Simplified

The CTO General 🦅 — Thu, 03 Feb 2022 08:51:48 +0000

What are APIs

API - Application Programming Interface is a software that allows two different applications which might be written in two languages to communicate and retrieve data from each other without a language barrier. It is as simple as that.

It is software that serves as an intermediary or interface that allows another application to use some of its data which is allowed by the Host application creating the API.

It is a medium that allows two applications to talk to each other.

API analogy

Let's look at an example to see how an API works and what it does for better understanding.
The common analogy used to describe API is the Food Restaurant analog. But let's take it a step further and make things simpler.

`
Consider that there is a Bank. In this bank, you get to keep your money and other valuable items.

In other to get money or have access to your values each time you need it, there is a procedure you need to follow, else you won't be given the access. right?

Now there are just two parties involved, you and the bank. But it is usually not an easy option for you to go to the bank directly to retrieve your valuables or withdraw your money. Trust me, it is tedious.

So, in other to ease its customer's pain, the bank decided to provide you with an ATM or a POS which allows you to withdraw your money through your card from the comfort of your home.
This is a lot easier, right? yeah, it is.

How APIs work

There are now three things involved in the process:
The Bank(application): In which you store your money (Information/Data)
The ATM(API): This is what gives you access to your money or valuables
The Customer(application): This is the owner of the money or valuables(Information/Data)

So, this is how the process goes, you have the customer, whenever you need to get or withdraw money, you do not need to go to the bank to have access to this service that your bank offers you.
All you have to do is to use your card to withdraw through the ATM or POS to get your money.

Analogy explanation:

The Bank which is the application that information is needed from creates and gives you an ATM which is the API with which you can withdraw and get your money or information or data from.

API gives you access to information that cannot be accessed directly from an application.
With APIs applications of different types can interact with each other easily. It means users of your application do not have to leave your application to use features of other applications.

A good example of this is the use of the Facebook or Google API to log in or sign up on other applications without having to move out of the application to log in to the app or website.
Google and Facebook API gives the application access to this login information or details through its APIs.

APIs reduce the complexity of having to deal with complex steps involved for two apps written in different languages to send and receive information from each other.

APIs are hugely important in today's economy. These are some of the practices companies use to stay relevant as users or developers use their products directly or indirectly.

Integrating Stripe payments with Flask

The CTO General 🦅 — Wed, 15 Dec 2021 23:37:28 +0000

Introduction

This article shows you how to integrate Stripe payment into your Flask web application.

Stripe is an online payment processor that allows payments with credit cards for your business. It allows you to easily accept payment on your web application using its Checkout option. Stripe Checkout has one of the best User-friendly payment providers.

To make things easy while creating our app, we will be making use of the Stripe Checkout. The Stripe Checkout allows for a better payment integration with our flask application. It also handles the whole process of payment for you, so you do not need to create your custom checkout form. The Stripe Checkout allows you to start accepting payments immediately.

Prerequisites

To successfully follow along with this tutorial, a basic understanding of Python and the Flask framework is needed.
This tutorial assumes you have this basic understanding and you also have Python and Flask installed on your local machine.

So, having gotten all this setup, let’s start building…

Project setup and installation

We need some basic setup to get our flask application running and integrate Stripe. So, let's get our app set up.

Make a folder to put in your flask project files and create a file called “app.py” in the root folder, then add the following code.

    from flask import Flask, render_template

    app = Flask(__name__)

    @app.route('/')
    def checkout():
        return render_template('checkout.html')

    if __name__ == '__main__':
        app.run()

This simple code above is a flask server that renders the checkout.html file we created in the templates directory of the root folder.

Create another folder in your project folder and call the folder “templates". Create, the “checkout.html” file in this folder and add the following code.

    <form action="/charge" method="post">
      <article>
        <label>
          <span>Amount is $10.00</span>
        </label>
      </article>

      <script src="https://checkout.stripe.com/checkout.js" class="stripe-button"
              data-key="{{ key }}"
              data-description="A Flask Charge"
              data-amount="1000"
              data-locale="auto"></script>
    </form>

Here is a description of what the code above does:

class: this attribute takes in the Stripe class name.
data-key: the data key takes in the secret key we will store in our environment variable, which Stripe provides to us while creating our Stripe account.
data-description: A description of the charges is being passed here.
data-amount: this is the integer value of the amount being charged (in cents).

Note that a POST request is sent to the server-side, to a route called “/charge” on submit.

Setting up Stripe

Now, let’s set up our Stripe account. Let’s start by installing Stripe on our computer. Open your terminal and run the following code

`(env)$ pip install stripe`

This would take a fragment of your time but will install.

Next, let’s create our Stripe account. To register Click here!, this would take you to the Stripe sign-up page, which will look much more like this.

After signing up, you would be taken to your dashboard. Click on the developer link at the top right corner of the page as shown below.

.
.
.

Click on the API Keys tab on the left to get your view like this.
Every Stripe account has two API keys, one for testing applications and the other for live applications. Each of these has two pairs of keys, the Publishable key, and the Secret key.

For this tutorial, we will be making use of the testing key. Copy the Publishable key and the Secret key on your dashboard and let’s save in our environment variable.

Save your keys to an environment variable like this, in a “.env” file in your project folder.

`(env)$ export STRIPE_PUBLISHABLE_KEY = <YOUR_STRIPE_PUBLISHABLE_KEY>`
`(env)$ export STRIPE_SECRET_KEY = <YOUR_STRIPE_SECRET_KEY>`

Now, let’s set up Stripe in “app.py”.

    from flask import Flask, render_template, request
    import stripe

    app = Flask(__name__)

    stripe_keys = {
        "secret_key": os.environ["STRIPE_SECRET_KEY"],
        "publishable_key": os.environ["STRIPE_PUBLISHABLE_KEY"],
    }

    stripe.api_key = stripe_keys["secret_key"]

    @app.route('/')
    def checkout():
        return render_template('checkout.html')

    if __name__ == '__main__':
        app.run()

This is all we have to do to set up our Stripe account with our flask application.

Template Setup

Now, let’s create some HTML templates for our page. Create an charge.html file on the same directory (root folder). Put in the following code.


<!DOCTYPE html>
<html>
<head>
        <title>Stripe Success</title>
</head>

<style type="text/css">
        body {
                display: flex;
                  align-items: center;
                  justify-content: center;
                  min-height: 100vh;
        }
</style>

<body>

  <h2>Thanks, you paid <strong>$10.00</strong>!</h2>

</body>
</html>

Creating Stripe checkout form

Let’s update our checkout form to display properly with the following code.

    <!DOCTYPE html>
    <html>
    <head>
            <title>Stripe Payment</title>
    </head>

    <style type="text/css">
            body {
                    display: flex;
                      align-items: center;
                      justify-content: center;
                      min-height: 100vh;
            }
    </style>

    <body>
                    <form action="/charge" method="post">
      <article>
        <label>
          <span>Amount is $10.00</span>
        </label>
      </article>

      <script src="https://checkout.stripe.com/checkout.js" class="stripe-button"
              data-key="{{ key }}"
              data-description="A Flask Charge"
              data-amount="1000"
              data-locale="auto"></script>
    </form>
    </body>
    </html>

Now that we are done with setting up our template, let’s handle our route.

Update your “app.py” with the following code.

    from flask import Flask, render_template, request
    import stripe

    app = Flask(__name__)

    stripe_keys = {
        "secret_key": os.environ["STRIPE_SECRET_KEY"],
        "publishable_key": os.environ["STRIPE_PUBLISHABLE_KEY"],
    }

    stripe.api_key = stripe_keys["secret_key"]

    @app.route('/')
    def checkout():
        return render_template('checkout.html',key=stripe_keys['publishable_key'])

    @app.route('/charge', methods=['POST'])
    def charge():
        # Amount in cents
        amount = 1000

        customer = stripe.Customer.create(
            email='customer@example.com',
            source=request.form['stripeToken']
        )

        charge = stripe.Charge.create(
            customer=customer.id,
            amount=amount,
            currency='usd',
            description='Flask Charge'
        )

        return render_template('charge.html', amount=amount)


    if __name__ == '__main__':
        app.run()

Our application is done!!!, let’s test it to make sure it works.

Testing our application

Go to your terminal and run this code to start the server up.

`(env)$ flask run`

You should get the following port number displayed on your terminal if your server is successfully set up.

* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)

Now run this port on your browser. This should be your result

You should get a success message like this after submitting the form.

Yay!!!!! You have successfully Integrated Stripe with your flask application.

Note that you can track every transaction on your app right from your Stripe account, as it is in the image below.

Conclusion

In this tutorial, we have successfully integrated the Stripe, we have successfully integrated the Stripe checkout with our Flask application and we were also able to visualize every payment on the Stripe platform.

Although, Stripe offers a couple of more payment options asides, the Stripe checkout. The option is used in most cases it is very easy to set up and start receiving payment for your business.

With this tutorial, I hope you have learned how to create your payment system with stripe. And you can now easily integrate Stripe all by yourself.

About Vybe

The CTO General 🦅 — Mon, 13 Dec 2021 08:49:40 +0000

We created Vibe with one main purpose, which is to manage your financial account.

We believe that a lot or people have actually ended up broke and lack financial discipline as a result of their mis-management of funds.

We Vybe, we can achieve finance management and perform up to expected with our easy to use platform and analytics.

With the help of our Machine learning engineer, we have been able to build and train model that helps to achieve this goal of financial litracy. We have built and train this model on data to be able to compare and give a feed back on each users financal performance.

We Used the Mono Connect built by Mono Technologies to link several bank accounts to the app(Vybe). Not to worry, your details are well secured and encrypted.

How to use

Vybe is very simple to use. Following the steps and the links below.

Create an account or Login to an existing account.
Link your bank account with Vybe with the little arrow displayed at the top right coner of your dashboard

Done!!!!!!!!!!

Now, you can start using Vybe and getting tips on how to achieve your budgets and financial goals.

Note: Vybe's feature are way more than the listed here.

Chech Vybe out here

Vybe

You path to financial freedom

Build a web crawler from scratch

The CTO General 🦅 — Mon, 22 Nov 2021 20:27:02 +0000

Introduction

Have you ever wondered how search engines like Google, fetch data from different parts of the web and serve it to you as a user, based on your query? The method used by applications like this is termed crawling.

Search engines work and by crawling and indexing billions of web pages using different web crawlers called web spiders or search engine bots. What these web spiders do is follow links from each web page that have been indexed to discover new ages.

In this short article, you will be building out your own web crawler using Python and its popular web framework for extracting the data you need from websites.

Also, in this tutorial, developers tend to be confused with web crawling and web scraping. We would learn the difference between these two concepts and examine some examples

Prerequisites and Requirements

Before you begin with this tutorial, make sure you have a very basic understanding of python programming, as the article assumes you have gotten familiar and some of the concepts we would be discussing here would be new to you.

To move and understand things quickly, you must also know how to use your chrome developer tools to find tags and classes, in other for you to scrap the website well.

You also need to have the python package manager PIP, to enable you to install Scrapy.

What are web crawlers?

Web crawlers also known as spider bots are internet bots that search the entire Internet for content and prepare them for indexing. This is how crawlers work in a search engine.

In how case the data were gotten from our crawler won't be indexed as we are not building a search engine (at least not yet).

Crawlers consume and extract data from different pages it crawls from and gets them ready for use. Depending on the use case of the user. Crawlers can also be used for web scraping and data-driven programming. The Goal of each web crawler is to know what each web page is all about and to serve it when it is needed or requested for. It automatically accesses a website, consumes and downloads its data through software programs.

Since the internet is constantly changing and it is impossible to know the total number of web pages on the internet, Web crawlers start crawling from a list of known URLs. They crawl the pages of those URLs first and then start looking for a hyperlink to other related web pages from those URLs.

Though crawlers also crawl and find web pages based on the number of traffic on a website, the basic and default of how web crawlers find pages are from the list of known URLs or seeds.

By applying proven algorithms to building crawlers and finding web pages, search engines can provide relevant links and respond almost accurately to user queries.

Difference between Scrapy and BeautifulSoup

As we said earlier, we are going to be using scrapy for this tutorial. But there seems to be a couple of web crawling and scraping framework used in python. The most common of them all is Scrapy and BeautifulSoup.

Although both of these tools are used for web scraping and crawling, there’s quite a large difference between them both. Unlike Scrapy which is used to be specifically created for downloading, cleaning, and saving data from the web and will help you end-to-end, BeautifulSoup is a smaller package that will only help you get information out of webpages with the request module.

Scrapy Installation

To be able to make use of scrapy in our application we need to install scrapy to our local machine, and to do this, run the following command on your terminal:

`pip install scrapy`

The code above downloads scrapy for us. Once it finishes downloading we will then move to create our project.

Creating the Scrapy app

Create a scrapy-python application with a name with the command below. In our case, we named our python application webcrawler.py.

`scrapy startproject webscrawler`

The code above starts a scrapy project in a folder with a scrapy boilerplate. The file structure created should look like this:

Open your application folder (in my case webcrawler) in a code editor, and let's continue building our crawler.

Creating Web Crawler

Creating your first spider. Spider are classes we define, and scrapy uses to gather information from the web.
To create your first spider in scrapy, navigate to the spider file and create and a new folder called spider1.py. This will be our first spider.

All spiders or crawlers are created and stored in the spider folder.

Open your spider1.py and paste the following codes, to create your first crawler, as we would be crawling through the http://quotes.toscrape.com/page/1/ web page.

    #imports
    import scrapy

    #Spider class
    class QuotesSpider(scrapy.Spider):
            #name of your spider
            name = "quotes"

            def start_requests(self):
                    #Website links to crawl
                    urls = [
                            'http://quotes.toscrape.com/page/1/',
                            'http://quotes.toscrape.com/page/2/',
                    ]

                    #loop through the urls
                    for url in urls:
                            yield scrapy.Request(url=url, callback=self.parse)

            def parse(self, response):
                    page = response.url.split("/")[-2]
                    filename = f'quotes-{page}.html'
                    with open(filename, 'wb') as f:
                            f.write(response.body)

The code above crawls through the web links given and downloads the page and gets it ready to be scraped.

To run the following codes, run the following code in your terminal in the directory of your scrapy project.

scrapy crawl webscrawler

When this code is run successfully, two files (quotes-1.html & quotes-2.html) will be created in your project directory. These are the two pages that are downloaded by scrapy and ready for scraping.

Congratulations, you have built your first Web crawler. Tadaa.

Scraping the web pages

Now that you have crawled the web pages and it has been downloaded. We then need to scrape or extract the data needed from the web page. And we would be using the css selector to get this data.

Update your scraper1.py file with the following codes:

    import scrapy

    class QuotesSpider(scrapy.Spider):
            name = "quotes"

            def start_requests(self):
                    urls = [
                            'http://quotes.toscrape.com/page/1/',
                            'http://quotes.toscrape.com/page/2/',
                    ]

                    for url in urls:
                            yield scrapy.Request(url=url, callback=self.parse)

            def parse(self, response):
                    for quote in response.css('div.quote'):
                            yield {
                                    'text': quote.css('span::text').get(),
                                    'author': quote.css('small.author::text').get(),
                                    'tags': quote.css('div.tags a.tag::text').getall()
                            }

In above code extracts the text, author, and tags for each of the web pages listed in the URL.
To see our results from the scraped webpages run the following command in your terminal.

`scrapy crawl quotes -o results.json`

This command runs the scraper, scraps the data, and stores it in a folder called results.json.

If you have followed the tutorial till this point, your code should run successfully and you shouldhave the following data in your results.json file, which is created in your project directory.

Conclusion

Congratulations, on building your first web scraper. This is how simpler it is to create a web crawler and scrape data from it. By now you should have a basic idea of how web crawlers and web scrapers work and how it's being implemented with search engines to extract data from the web.

There are a lot of things you can do with scrapy and web scrapy. Data scientists and AI engineers used web scraping to get a substantial amount of data to perform various data analyses and train a model with this data.

I hope you enjoyed the tutorial and it was helpful. Be sure to build projects get a vast knowledge in performing web scraping and web crawling.

How to use celery with flask

The CTO General 🦅 — Sun, 21 Nov 2021 12:30:08 +0000

Introduction

Have you ever come across programs or task that takes a lot of time to process or give an actual output? Tasks such as sending emails and uploading data over the Internet takes time to process which can slow down your application’s workflow. These task should be run separately from other tasks.

Your application should be able to process with these task in the background and continue with other tasks. After these task have processed and the result is ready, then it can be served to the user.

I will be introducing you to setting up and configuration of celery and Redis in a flask project which handles async function or tasks like this. We would also take a look at the application of celery in real time with build and email sender app. Remember that these task takes time.

We would be making our email sender run as a background task.

Prerequisites

This tutorial assumes that you already know about the basics of Python and Flask and also assumes that you have python and Flask framework set up on your system or machine. You must also have the very least and basic understanding of HTML to build out email forms.

What is a Task Queue?

A task queue is a system that distributes task that needs to be complete as a background without interfering with the applications request and response cycle.

Task Queues makes Assigning work that slows down application processes while running easier. Intensive application can be handle by software applications like this in the background while users still interacts with the website and carry on with other activities.

This ensures that the user's engagement is consistent, timely, and unaffected by the workload.

What is Celery?

Celery is a Python task queue that allows task to run asynchronously with web applications without disturbing the application’s request response cycle.

Celery is highly scalable, which is one of the several reasons why it is being used for background work and also allowing new workers to be dispatched on-demand to handle increasing workload or traffic.

With Celery being a well supported project and also well documented, Celery still has a thriving user community, though it is still under development.

Celery is also easy to integrate into various web frameworks, with most of them providing libraries to assist with this. Celery can also interface with several other message brokers hence, its thriving community.

What is an Asynchronous Task?

An asynchronous task is simply a function that runs behind every other process running on your app. This kind of function, when called, does not affect the normal flow of the application.

With asynchronous operations, you can switch to a new task before the previous one is complete. By using asynchronous programming, you can handle many requests at once, accomplishing more in a shorter amount of time.

Installation and Configuration for Celery on Flask

Running Celery requires the use of a broker. Redis is the most well-known of the brokers.

For sending and receiving messages, Celery requires the use of message broker, such as
⦁ RabbitMQ
⦁ Redis
⦁ Amazon SQS

Note: Message brokers are programs built to aids the communication between services when sending information from both ends.

We would be making use of the Redis server in the tutorial.

Creating a Flask server

Creating a Flask server is easy. Navigate to the folder where you want your server created. Create a new python file and give it a name, in our case celeryapp.py
And add this simple code to your python script:

    from flask import Flask
    app = Flask(name__)
    @app.route("/")
    def home():
    return ""Hello, World!""
    if name == "__main":
    app.run(debug=True)

Now, let's test our server to make sure it's working.
To start our server, run the following commands in your terminal:
python celeryapp.py

If you have followed the article correctly, your result should look simple to the image below.

Communicating between Celery and Flask

Now that we have created our server, we need to connect Celery with our flask application. To do this, update your celeryapp.py file to look like this below.

    #imports
    from flask import Flask
    from celery import Celery

    #creates a Flask object
    app = Flask(name)

    #Configure the redis server
    app.config['CELERY_BROKER_URL'] = 'redis://localhost:6379/0'
    app.config['CELERY_RESULT_BACKEND'] = 'redis://localhost:6379/0'

    #creates a Celery object
    celery = Celery(app.name, broker=app.config['CELERY_BROKER_URL'])
    celery.conf.update(app.config)

Celery is initialized by creating an object of class Celery with the application name and the connection to the message broker URL which is set to CELERY_BROKER_URL as key in the app.config. If you run something other than Redis or have the broker on a different machine, you will need to change the URL accordingly.

it’s always best to add additional configuration through celery.conf.update() for Celery. Though it is not a requirement, the CELERY_RESULT_BACKEND is only necessary to store status and results from tasks in Celery.

The function that would run as a background task is just a normal function with has the celery.task decorator. With just this decorator, the function would always run in the back ground. For example:

    @celery.task
    def async_function(arg1, arg2):
    #Async task
    return result

Just like any other function, to execute our celery task, it needs to be invoke, To invoke the function, add the following code to your celeryapp.py file.

 `async_function(10, 30)`

Sending an Asynchronous Email with Celery.

Now, let's see how Celery works in the real world. Let's apply Celery in sending an email with our flask application. First, we need to build out our email form to let users send emails. Here is the HTML template to build the form.

    <html>
      <head>
        <title>Flask and Celery</title>
      </head>
      <body>
        <h2>Sending Asynchronous Email</h2>
        {% for message in get_flashed_messages() %}
        <p style="color: red;">{{ message }}</p>
        {% endfor %}
        <form method="POST">
          <p>Send email to: <input type="text" name="email" value="{{ email }}"></p>
          <input type="submit" name="submit" value="Send">
        </form>
      </body>
    </html>

This is just a regular HTML syntax with the ability to show flask messages from Flask. Hopefully, you should be able to get around with it.

Note: The code above is the index.html file.

To send Emails, we would make use of the Flask-Mail extension.
Flask-Mail requires some configuration, including information about the email server that it will use to send emails.

Add the following code to your celeryapp.py app to configure your Email sender:

    Flask-Mail configuration
    app.config['MAIL_SERVER'] = 'smtp.googlemail.com'
    app.config['MAIL_PORT'] = 587
    app.config['MAIL_USE_TLS'] = True
    app.config['MAIL_USERNAME'] = os.environ.get('MAIL_USERNAME')
    app.config['MAIL_PASSWORD'] = os.environ.get('MAIL_PASSWORD')
    app.config['MAIL_DEFAULT_SENDER'] = 'flask@example.com'

Note: For this to work, make sure you have saved your email and password to an environment variables. For security reasons and easy accessibility my password and email are stored in an environment variables.

Since we have a single route in this app, we created a route index alone to cater for it. Update your celeryapp.py file with the following code.

    @app.route('/', methods=['GET', 'POST'])
    def index():
    if request.method == 'GET':
    return render_template('index.html', email=session.get('email', ''))
    email = request.form['email']
    session['email'] = email
    # sends this content
    email_msg = {
        'subject': 'Testing Celery with Flask',
        'to': email,
        'body': 'Testing background task with Celery'
    }
    if request.form['submit'] == 'Send':
        # sends the email content to the backgraound function
        send_email.delay(email_msg)
        flash('Sending email to {0}'.format(email))
    else:
        flash('No Email sent')

    return redirect(url_for('index'))

The code above shows a function, which gets the input from our html form and saves it in a session, for easier accessibility.

This function checks for events on the submit button and send_email after the submit button is clicked. The email_msg contains the subject, recipient’s email address, and the body of the message being sent.

To allow users know what is going in the background, a flash message is displayed when the email is being submitted.

Note: We saved the user's value in the text field in the session to remember it after the page reloads.

The last piece of this application is the asynchronous task that gets the job done when a user submits the email form.

    @celery.task
    def send_email(email_msg):
    #Async function to send an email with Flask-Mail
    msg_sub = Message(email_msg['subject'],
    email_sender = app.config['MAIL_DEFAULT_SENDER'],
    recipient = [email_msg['to']])
    msg_sub.body = email_msg['body']
    with app.app_context():
    mail.send(msg_sub)

As said earlier, this task is decorated with celery.task to make it run in the background.

The function creates a Message object from Flask-Mail using the email data dictionary. For Flask-Mail to run, it must build an application context before calling the send() method.

Conclusion

Celery more than a few extra steps beyond simply sending a job to a background thread, but the benefits in terms of flexibility and scalability are hard to ignore.

Sending a scheduled task is easier done using celery than any other means of running scheduled task. Imagine you want to perform a task daily, In this case, Celery can be used to run programs in the background with any necessarily human triggered.

Although Celery is used in most cases for long running task, it can also be used to connect to third party APIs. As soon as data is gotten back from the API in your celery task, it is then served to the user.

How to build your first API in Django

The CTO General 🦅 — Mon, 23 Aug 2021 08:53:56 +0000

We often hear about how APIs are all the time. They can be very confusing when trying to understand how APIs work. In this article, We will create our API using Python/Django. Every program or piece of software uses REST APIs to connect the backend and frontend components. Let's get started with it.

What are APIs

First, we need to understand what APIs are and what they do.
API stands for Application Programming Interface. Just as seen in the name, They are interfaces. An API is a software interface that enables two apps to communicate with one another. Wikipedia gives a broad definition:

“In computer programming, an application programming interface (API) is a set of subroutine definitions, protocols, and tools for building application software. In general terms, it is a set of clearly defined methods of communication between various software components” - Wikipedia

We would be making use of the Django REST Framework for this tutorial to create our API.

What is Django REST Framework

Django REST framework (DRF) is a powerful and flexible toolkit for building Web APIs. Django is a web framework that makes it easier to construct REST interfaces by reducing the amount of code required to perform a task.

Setting up Django REST framework

This article assumes that you already have Django virtual environment installed on your computer. With that, we can get started by installing the Django REST framework.
To install Django rest framework in our virtual environment run:

$ pip install Django rest framework

Your result should look like this:

   $ pip3 install djangorestframework
   Collecting djangorestframework
     Downloading djangorestframework-3.12.2-py3-none-any.whl (957 kB)
        |████████████████████████████████| 957 kB 595 kB/s
   Requirement already satisfied: django>=2.2 in ./venv/lib/python3.8/site-packages (from djangorestframework) (3.1.3)
   Requirement already satisfied: asgiref<4,>=3.2.10 in ./venv/lib/python3.8/site-packages (from django>=2.2->djangorestframework) (3.3.1)
   Requirement already satisfied: sqlparse>=0.2.2 in ./venv/lib/python3.8/site-packages (from django>=2.2->djangorestframework) (0.4.1)
   Requirement already satisfied: pytz in ./venv/lib/python3.8/site-packages (from django>=2.2->djangorestframework) (2020.4)
   Installing collected packages: djangorestframework
   Successfully installed djangorestframework-3.12.2

To create our django_todo project now by running the command django-admin startproject todo

After successfully creating your Django project, change your working directory to your project folder and then run the Django Development server using this code: python manage.py runserver

$ cd todo
$ python manage manage.py runserver
Watching for file changes with StatReloader
Performing system checks...
System check identified no issues (0 silenced).
You have 18 unapplied migration(s). Your project may not work properly until you apply the migrations for app(s): admin, auth, contenttypes, sessions.
Run `python manage.py migrate` to apply them.
November 17, 2020 - 10:14:01
Django version 3.1.3, using settings 'todo.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

For our endpoints, we need to create a todoapp
run the following code to create your app

$ python3 manage.py startapp todoapp

To add our todo app to the INSTALLED_APPS list. Open settings.py in the project directory django_todo and then We'll also include the rest framework app so it can be used in our project.

# ./django_todo/settings.py
...
INSTALLED_APPS = [
    ...
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'rest_framework',
    'todoapp',
]

Creating the Todo model

Models in Django are Python classes that represent a database table. When we run the command python manage.py migrate, Django creates a Todo table in the database, which we can see in the Todo Model.
We'll create our Todo model in the models.py file of the todo app using the following code:

from django.db import models
# Create your models here.
class Todo(models.Model):
    title = models.CharField(max_length = 100)
    body = models.CharField(max_length = 100)
    is_completed = models.BooleanField(default=False)
    date_created = models.DateField(auto_created=True)
    last_modified = models.DateField(auto_now=True)
    def __str___(self):
        return self.title

To create the todoapp table in the database will run the command python manage.py migrate.
To make migrations run:

python manage.py makemigrations

To apply the migrations run

python manage.py migrate

Serializing the Todo model

Let's subclass the Django rest framework's Serializer.ModelSerializer class to make it easier to transform Python objects retrieved from the database to the JSON format required by our APIs and back to Python objects that can be mapped to our database tables.
In the todo app directory, create a file serializers.py, where we will write our TodoSerializer code.

from rest_framework import serializers
from todo.models import Todo
class TodoSerializer(serializers.ModelSerializer):
    class Meta:
        model = Todo
        fields = "__all__"

Creating the TodoAPIView

We'll build the code for our app's CRUD capabilities in the views.py file in our todo app directory. The Django Rest Framework has built-in classes that make creating CRUD functionality a breeze.

We'll begin by importing the Django rest framework classes we'll require to build our CRUD API; then we'll create a class for each of our crud endpoints and add the queryset and serializer class for our Todo model.

from django.shortcuts import render
from rest_framework.generics import ListAPIView
from rest_framework.generics import CreateAPIView
from rest_framework.generics import DestroyAPIView
from rest_framework.generics import UpdateAPIView
from todo.serializers import TodoSerializer
from todo.models import Todo
# Create your views here.
class ListTodoAPIView(ListAPIView):
    """This endpoint list all of the available todos from the database"""
    queryset = Todo.objects.all()
    serializer_class = TodoSerializer
class CreateTodoAPIView(CreateAPIView):
    """This endpoint allows for creation of a todo"""
    queryset = Todo.objects.all()
    serializer_class = TodoSerializer
class UpdateTodoAPIView(UpdateAPIView):
    """This endpoint allows for updating a specific todo by passing in the id of the todo to update"""
    queryset = Todo.objects.all()
    serializer_class = TodoSerializer
class DeleteTodoAPIView(DestroyAPIView):
    """This endpoint allows for deletion of a specific Todo from the database"""
    queryset = Todo.objects.all()
    serializer_class = TodoSerializer

Creating URL paths for our Todo endpoints

We may connect with the various Todo crud views using the URLs. Create a file urls.py in the todo app directory where we will write URLs for various endpoints.

from django.urls import path
from todo import views
urlpatterns = [
    path("",views.ListTodoAPIView.as_view(),name="todo_list"),
    path("create/", views.CreateTodoAPIView.as_view(),name="todo_create"),
    path("update/<int:pk>/",views.UpdateTodoAPIView.as_view(),name="update_todo"),
    path("delete/<int:pk>/",views.DeleteTodoAPIView.as_view(),name="delete_todo")
]

Testing the endpoints

Let's make sure our development server is up and running before we test our API endpoints. Run the command python manage.py runserver in the root folder of our project, where the manage.py file exists, to start the development server.

$ python manage.py runserver
To create a new Todo we make a POST request to
http://localhost:8000/api/v1/todo/create/ with the new Todo object.
{
    "date_created": "2020-11-19",
    "title": "Take a bath,
    "body": "Don't smell and take a break",
    "is_completed": true
}

Conclusion

In Django, we now know how to define and document restful endpoint APIs.

We'll protect our endpoints and add social authentication to our app in the following article.

Thanks for reading.