DEV Community: Tijani Ayomide

From Postman to Postman Extension

Tijani Ayomide — Tue, 15 Aug 2023 15:40:10 +0000

Introduction

Once upon a time, during my internship at a startup, I began my software exploration journey. As a newcomer, my initial tasks revolved around simple manual testing and pushing the limits of the application to uncover its vulnerabilities. When I discovered these errors I document them and uncover them during the standup meetings.

To cut a long story short, my role evolved and I was entrusted with more responsibilities. I was given the task to handle functionality for the feedback service. After familiarizing myself with the framework, one might think it would be an easy task ( looking back it was an easy task) but at the time I was met with a reality check.

I was queried by the senior developer on my team and it dawned on me that software development was not all about writing code. I also had to test, validate and ensure my implementation meets the business logic. This is where my Postman origin began as this tool became my trusted companion. And so my journey took a turn.

What is Postman?

Aside from being my trusted companion, Postman is an API development and testing tool that simplifies designing, testing, and documenting APIs. while offering a clean and easy-to-use interface to collaborate, create, manage, and inspect requests and responses respectively.

What is the Postman extension?

If you’re a fan of vscode (Visual Studio Code), you should know how important extensions are in amplifying your development process. Many extensions can be found at the Microsoft marketplace ranging from auto-formatter, Dark Dracula themes to chilled lo-fi cat extensions just chilling at your IDE’s corner. It’s a world where extension possibilities can seem endless.

But let’s focus on something more exciting — Postman's new extension that allows you to test your API right from your code editor, without the need to juggle screens or switch windows.

Features and Capabilities

“Basically” everything you can perform on Postman Web and Desktop platform can be done with this extension. If you’re one of those developers who prefer not leaving their development environment think of this extension as a tailored and condensed version.

Let’s look into some of its features:

Sending of HTTP and Multiprotocol Requests

Easily send HTTP, WebSocket, gRPC, and other multiprotocol requests without leaving your work environment. You can fire them off from within your IDE, so there's no need to interrupt your workflow. Everything you need is right at your fingertips.

Collection and Workspace Creation

Organize requests easily by grouping them into various collections and workspaces to maintain a tidy and structured workflow environment.

Managing Environments and more

The extension automatically synchronizes collections, requests, and environments, saving you the hassle of creating them from scratch or having the need to import them. Just ensure that the credentials you use to sign in match the workspace you want to access.

I began this paragraph with basically quoted, this is because updates are still being made to enhance the functionality of the application, similar to what is available on the Web/Desktop version. Keep an eye out for more updates and newly added features.

Install Postman Extension

Installation of this extension can be done super easily in three simple steps:

Open your vscode extension marketplace by navigating to it directly or using the keyboard shortcut: ctrl+shift+x (Windows/Linux) or cmd+shift+x (Mac). In the extensions tab, search for "Postman".

Wait for the installation process to be completed successfully, depending on your internet speed the duration may vary.

Complete the process by signing into your Postman account.

Conclusion

While the surface is merely scratched, the Postman extension offers a glimpse into a future of enhanced productivity and a simplified testing environment. Keep an eye out for new features and improvements that the team at Postman diligently unveils.

Is this an extension you would use? Leave your thought in the comment section below.

Let's Connect

Reach out to me on Linkedin
Reach out to me on the X app (Formerly Twitter)
And we can also connect in the comment section below ⤵️

Understanding dependencies and dev-dependencies: Beginner’s Guide

Tijani Ayomide — Mon, 10 Jul 2023 14:10:30 +0000

When I first delved into the world of JavaScript, I frequently encountered a file called package.json. Initially, I understood it to be a file where one simply names their current project. However, as I continued my journey, I discovered that this file holds much more significance than meets the eye.

The package.json file serves as more than just a repository for descriptions, versions, and scripts for your project. It also functions as a home for the packages utilized within your project, giving rise to the term "dependencies." This is because your project relies on these packages to function effectively.

Before we delve into learning more about dependencies, it is important to have a brief understanding of the term "Packages" for quick reference. Packages are pre-written libraries or code modules that can be used to enhance the functionality of your project. These packages serve as a ready-made solution to common programming challenges without the need to reinvent the wheel.

Some basic examples of popular packages in the Javascript community include Express (a framework for building server applications) and Multer (a middleware for handling file uploads). These packages, when integrated into your project, can significantly streamline your development process.

However, let's shift our focus away from packages themselves, as today's topic revolves around understanding dependencies and developers who depend on them. (pun intended!).

What is a dependency❓

As mentioned above a dependency refers to an external module or library that your project relies on to function correctly. These dependencies are typically third-party packages maintained by other developers or organizations. Let's take the example of installing the Express package. To install Express, you can execute the following command:

npm install express
// or
yarn add express
// Depending on the package manager of your choosing

This command will fetch the Express package from the npm registry and add it to your project's dependencies. Additionally, it will automatically update your package.json file to include the specific version of Express that was installed.

For example, your package.json file may now contain an entry like this:

"dependencies": {
  "express": "^4.17.1"
}

In this case, Express is listed as a dependency, and the version specifier "^4.17.1" indicates that your project requires Express version 4.17.1 or any compatible version within the specified major version.

What is a dev-dependency❓

A dev-dependency, short for development dependency, is another type of dependency that is specific to the development environment. Dev dependencies include tools, libraries, or utilities that are helpful during the development process but are not necessary for the production version of your application.

Let’s take the example of installing nodemon package which is a popular tool that automatically restarts your Node.js server whenever a file is modified. To install nodemon as a dev-dependency, you can run the following command:

npm install --save-dev nodemon
// or
yarn add -D nodemon
// Depending on your package manager

The --save-dev and -D flag tells npm and yarn respectively to add nodemon as a dev-dependency in your project. Similarly, the package.json file will be updated to reflect this addition.

"devDependencies": {
  "nodemon": "^2.0.12"
}

Updating Dependencies 🔃

Keeping your packages up to date is of utmost importance, as these updates often include fixes, security patches, and new features that can facilitate performance improvements for your project. Let's explore two approaches to updating packages in your project.

Updating All Dependency

To update all packages in your project, you can use the package manager of your choice:

npm update
// or
yarn upgrade

// Note: Make sure you have a good understanding of your chosen package manager

However, it is important to note that updating all packages at once may not always be advisable. There may be instances where certain libraries and modules have dependencies on specific versions of packages, and updating all dependencies in your project can lead to compatibility issues and potential breakages in your codebase. Therefore, exercise caution and perform thorough testing after the update.

Updating a Specific dependency

In some cases, you may need to update a specific package to get the new feature or to resolve an issue in your codebase. To update specific packages you can use the package manager of your choosing followed by the package name and the desired version specifier. For example:

// Node Package Manager (NPM)
npm update <package-name>

// Yarn Package Manager
yarn up [package]
or
yarn up [package]@[version]

By specifying the package name, you ensure that only the selected package is updated, reducing the chances of unintended side effects. When updating a specific package, it is also important to consider any potential impacts on your project. Some updates may introduce breaking changes or require adjustments in your codebase. Therefore, it is recommended to review the package's release notes or changelog to understand the changes introduced in the new version. it is a good practice to test your application thoroughly after updating to ensure its continued functionality.

Conclusion

Understanding and managing your dependencies is crucial for efficient project development and deployment. It allows you to gain a clear understanding of which packages are required for your project's functionality and which ones are only necessary during development. It is quite common for developers to mistakenly install packages meant to be dev-dependencies as regular dependencies. However, this can lead to bloated production builds and slower deployment times. By understanding the difference between the two, you can optimize your workflow and avoid unnecessary package inclusion.

Let's Connect 🔗

Reach out to me on Linkedin
Reach out to me on the Bird's App 🐦 or the Thread's App 🧵 ( Kindly follow I'll follow back immediately )
And we can also connect in the comment section below ⤵️

Which Case is Best for Your Coding Style

Tijani Ayomide — Wed, 07 Jun 2023 05:29:41 +0000

Naming variables, functions, and other elements in programming can be a daunting task. It becomes even more difficult when you have to decide which naming convention to follow. One common debate revolves around the choice between snake case and camel case.

However, this article aims to introduce you to a wider range of casing styles used in programming. While snake case and camel case are well-known, there are other casing conventions worth exploring.

What is Casing?

Before we delve into the topic at hand, let’s understand the importance of casing in programming. Casing refers to the practice of using standard conventions for naming variables, functions, classes, and other elements in code. It plays a significant role in enhancing code readability, maintaining consistency, and improving collaboration among developers.

What is Snake Case?

snake case involves separating words with underscores. When using snake case each character needs to be lowercase.

user_id = 12;
welcome_message = "Hello Human";

You’ll encounter this case-naming convention in programming languages like Python and Ruby. You will also come across it when working with databases, as it is used for creating tables and column names.

model User {
  firstName       String            @map("first_name")
  lastName        String            @map("last_name")
  phoneNumber     String            @map("phone_number")
}

There is also the capitalized version of snake case, where all the characters are in upper case, known as screaming snake case.

FIRST_NAME = "John";
LAST_NAME = "Doe";

What is Camel Case?

Camel case involves capitalizing the first letter of each word except for the first word, which starts with a lowercase letter. Basically, a capital letter appears at the start of the second word and at each subsequent word that follows it.

String firstName = "Jane";
int userId = 2;

This naming convention is widely used in programming languages like Java and JavaScript.

What is Pascal Case?

This is also known as the "Upper Camel Case," as it exhibits similarities with the camel case writing style, but in Pascal's naming convention, it capitalizes the first letter of each word, including the first word.

int FirstName = "John";
string LastName = "Doe";

You’ll see this naming convention in most programming languages, but it is more pronounced in languages like C# (C hashtag) and C++.

What is Kebab Case?

This is also known as the spinal case, it separates words using hyphens. it is often used in file and URL naming… Although it may not be used as the go-to naming convention, it is worth mentioning for its relevance.

helper-files.js
first-name = "John";

When choosing a casing style, it is important to consider the programming language or framework you are using. Adhering to these conventions promotes code consistency and enhances collaboration within a development team.

Conclusion

Selecting the appropriate casing style for your coding style is crucial for readability and maintaining a consistent naming convention. Understanding the difference between each case styling will allow you to write clean and readable code that is easier to understand and maintain.

Let’s Connect

Reach out to me on Linkedin
Reach out to me on the Bird app ( Kindly follow I'll follow back immediately )
We can also connect in the comment section below

API Limiting: Best Practices and Implementation

Tijani Ayomide — Mon, 01 May 2023 13:17:04 +0000

As developers, it is crucial to ensure the optimal performance of our APIs. Slow database performance and lags can have severe consequences on our applications. However, performance is not the only reason for implementing API limiting. It can also act as a crucial safety measure against Denial of Service (DDoS) attacks that can overload a server with unlimited API requests.

However, the unrestrained use of APIs can lead to grave risks, such as overwhelming servers, compromising sensitive data, and even causing service downtime. To counter such issues, API limiting has emerged as a critical measure that controls and manages API usage, promoting the reliability and security of APIs.

In this article, we will explore the meaning, importance, types, use-case and application of API limiting in a simple API. By the end of this article, you will understand how to rate limit your APIs, thereby enhancing scalability while mitigating risks.

What is API Limiting?

API limiting is the process of controlling the rate and frequency at which users can access and use APIs. It is achieved by setting specific thresholds, such as the number of requests per minute or hour, and limiting users' access to the API when they exceed those thresholds. By enforcing API limits, organizations can prevent API abuse and overload, improve service reliability, and enhance security.

Think of it this way, offering full access to your API is the same as granting unrestricted access to your platform. This can lead to negative consequences, such as users overusing the API and consuming excessive resources, which can diminish its value. While it is great people want to use your API and find it useful, an open-door policy can negatively impact your business's success and hinder scalability. It is essential to recognize the need for rate limiting as a critical component in managing API usage, promoting value, and achieving business objectives.

Types of API Limiting

Given your newfound understanding of API limit and its significance, let's look at the various types kinds of API limiting techniques, they include:

IP-based limiting: This method limits API access based on the user's IP address. It restricts API access from a specific IP address, regardless of how many requests come from that address. The pros of IP-based limiting include its simplicity, and it is ease of implementation. However, it has a significant drawback in that it can be bypassed by using proxies or VPNs, which can make it ineffective in preventing malicious attacks. Services such as CloudFlare make use of IP-based limiting to mitigate DDoS attacks. It blocks traffic coming from blacklisted IP addresses to ensure the network's availability.
Token-based limiting: This method involves assigning a unique token to each user, which is used to track and limit API usage. Tokens are issued for a specific period, and users need to renew them after expiry. The advantage of token-based limiting is that it is more secure than IP-based limiting, and users can only access the API if they have a valid token. However, token management can be challenging, especially for large-scale applications. Services such as Twilio, make use of token-based limiting to secure its API. It issues unique API keys to its users, which are used to authenticate API requests and limit usage based on the user's plan.
Quota-based limiting: This method limits API usage based on the number of requests allowed within a specific time frame, such as per hour or day. It is a flexible method, and it enables developers to customize their API usage limits based on individual user requirements. The downside is that it can be challenging to determine the correct quota limit for each user, and it may require trial and error to find the correct balance. Services such as Google Maps API, make use of quota-based limiting to limit API usage. It allows developers to set a daily limit on API usage, which can be customized based on the specific application's requirements.
Hybrid-based limiting: This method combines two or more limiting methods, such as IP, token-based and quota-based limiting. The benefit of hybrid-based limiting is that it leverages the strengths of each limiting method, creating a more comprehensive and robust API limiting strategy. Service such as Stripe makes use of a combination of token and quota-based limiting to secure their API. It assigns unique API keys to its users and limits API usage based on the user's plan and API request type.

When to use API-Limiting

Implementing rate limiting is one of the most common ways to ensure efficient API performance. When an API is used excessively, it can slow down the database and cause lags, ultimately resulting in a poor user experience. By implementing rate limiting, developers can ensure that API requests are processed efficiently and prevent the overloading of the server.

Another important use case for API limiting is to protect the API from malicious attacks, such as DDoS attacks, which can flood the server with unlimited requests. By implementing rate limiting, developers can limit the number of requests that a user can make within a specific timeframe, thereby preventing the server from being overwhelmed.

API limiting can also be used to protect sensitive data and prevent unauthorized access. By controlling and monitoring API usage, developers can ensure that only authorized users have access to sensitive data.

Furthermore, API limiting can be used to manage and control costs associated with API usage. By limiting the number of requests a user can make, developers can prevent excessive usage, which can result in increased costs.

Best Practice for API Limiting

Determine appropriate limits

One of the most crucial aspects of implementing API limiting is determining appropriate limits for your API usage. The goal is to find a balance between providing enough access for users while still maintaining control and preventing abuse.

To determine appropriate limits, you should consider the following factors:

Business Needs: Your API usage limits should align with your business needs and goals. It is essential to evaluate how much traffic your API can handle, what level of service you want to provide, and what kind of users you are targeting.
User Behavior: Your API usage limits should be based on user behavior, such as the number of requests, the rate of requests, and the size of the response. You should also consider how frequently users are accessing the API and the impact on the server resources.
Scalability: As your API usage grows, you should ensure that the limits are scalable to accommodate the increasing traffic.

Set clear and informative error messages

When API requests exceed the set limits, it is important to provide clear and informative error messages to users. These messages should inform users of the limit breach and guide how to resolve the issue.

{
    "statusCode": 429,
    "message": "Too many request, kindly wait 10 minute before sending another request."
}

Allow for flexibility and exceptions

While it is essential to have appropriate limits in place, you should also allow for flexibility and exceptions. For example, you may need to provide higher limits for premium users or allow temporary increases in usage during peak times. You should also have a process in place for handling exceptions and appeals from users who have exceeded the limits.

Implementing rate limiting in a node js application

Have you ever been to a buffet and seen that one person who just keeps piling up their plate with food?

While it might seem tempting to follow in their footsteps, it's not fair to the other people waiting in line. Similarly, in API development, it's important to ensure that one client doesn't hog all the resources and leave others waiting.

That's where rate limiting comes in. By setting a limit on the number of requests a client can make within a certain time frame, we can ensure that everyone gets a fair share of the resources. In our simple CRUD project, we have implemented rate limiting using the express-rate-limit package.

So, just like at a buffet, we encourage everyone to take only what they need and leave some for the other clients. With our simple rate-limiting implementation, we can ensure that everyone gets a fair share of the resources and can enjoy a smooth and efficient experience with our API.

Let’s create a simple CRUD project called “manga-reader” to demonstrate the implementation of rate limiting.

Prerequisites

To follow along with this guide, you will need:

Node.js installed on your machine
A basic understanding of JavaScript and Express.js

Creating the Project

Start by creating a new directory for your project and initializing it with yarn:

mkdir manga-reader
cd manga-reader
yarn init

Install the necessary dependencies:

yarn add express morgan dotenv prisma @prisma/client express-rate-limit

Schema of the Project

The schema for the project defines two models - Author and Manga - with a one-to-many relationship between them. The Manga model contains various attributes such as title, description, and language.

// schema.prisma
generator client {
  provider = "prisma-client-js"
}

datasource db {
  provider = "mysql"
  url      = env("DATABASE_URL")
}

model Author {
  id     String  @id @unique @default(uuid())
  name   String
  bio    String
  mangas Manga[]
}

model Manga {
  id          String   @id @unique @default(uuid())
  title       String
  description String
  content     String
  published   Boolean  @default(false)
  pages       Int
  language    Language @default(English)
  author      Author   @relation(fields: [authorId], references: [id])
  authorId    String
  created_on  DateTime @default(now())
  updated_on  DateTime @updatedAt()
}

enum Language {
  English
  French
  Japanese
}

Implementing Rate Limiting

With the express-rate-limit package one can implement rate limiting in our project.

// index.js
import morgan from "morgan";
import express from "express";
import * as dotenv from "dotenv";
import rateLimit from 'express-rate-limit';

import index from "./routes/index.route.js";

dotenv.config()
const app = express();

const apiLimiter = rateLimit({
  windowMs: 900000, // 15 minutes
  max: 100, // limit each client/ IP to 100 requests per 15 minutes
  message: 'Too many requests, please try again later.'
});

app.use(express.json());
app.use(morgan("dev"));
app.use(express.urlencoded({ extended: false}));
app.use(apiLimiter);

// routes
app.use("/manga", apiLimiter, index);

// Home
app.get("/", (_req, res) => {
  return res.status(200).json({statusCode: 200, message: `Manga Reader Example API`});
});

// 404 route
app.get("*", (_req, res) => {
  return res.status(404).json({statusCode: 404, message: `Route not found`});
});

export default app;

In the code above, a rateLimit middleware instance was created together with the set of windowMs and max options. The windowMs option sets the time interval for which the requests will be counted (in this case, 15 minutes), while max sets the maximum number of requests that can be made within that time interval (in this case, 100).

In this simple application, If a client tries to make more than 100 requests within a 15-minute time frame, they will receive an error message stating that they have reached the limit. This helps to prevent one client from overwhelming the system and ensures that all clients have equal access to the resources. To keep things simple and user-friendly, we can include a clear and concise error message along with a status code to provide more information on the cause of the error. This way, clients can easily understand what went wrong and take the necessary steps to rectify the issue.

Conclusion

Implementing API rate limiting in your Node.js project is an important step toward maintaining the stability and reliability of your application. With the use of packages like express-rate-limit or rate-limiter-flexible, you can easily set limits on requests and prevent abuse of your API by malicious users.

It is important to remember that rate limiting is just one aspect of securing your API. There are other security measures you can implement, such as authentication and authorization, to ensure that only authorized users have access to sensitive data.

So, have fun implementing rate limiting in your Node.js projects, and keep your API secure and stable.

Let's Connect

Reach out to me on Linkedin
Reach out to me on the Bird app ( Kindly follow I'll follow back immediately )
We can also connect in the comment section below

Understanding the Basics of ACID Principles in Database Management

Tijani Ayomide — Mon, 20 Mar 2023 16:26:49 +0000

During a recent interview, I was asked if I was familiar with the term "ACID" in the context of database administration or software development. While I had heard of the term before, I was uncertain of its meaning. I provided the interviewer with a brief explanation of my understanding, to which he replied “Hmm…alright”. In this article, we will delve into the ACID principle, its importance, real-world use cases, and why it is important.

Let's discuss transaction and its meaning before delving into the ACID principles and their significance.

Transaction

In the context of a database, a "transaction” is a group of database read-and-write operations that only succeed if all the operations within them succeed. Let’s walk through a classic example to explain this concept. Imagine you made a transfer from a personal account to a work account. Two actions are to take place

Your account is debited (if you have enough funds to make that transfer) and your account balance is updated accordingly.
Your work account is credited with the transferred amount, and the account balance is updated accordingly.

These two operations have to happen, or neither will happen at all. To keep the system consistent, there cannot be an in-between state during this operation. In a basic sense, that is what a transaction is all about.

A.C.I.D Properties of a transaction

ACID is an acronym that stands for Atomicity, Consistency, Isolation, and Durability. which refers to properties that define a transaction. These ACID properties ensure that a set of database operations leaves the database in a valid state even in the event of unexpected errors. ACID transactions guarantee that each read, write, or modification of a table has the following properties:

Atomicity

Atomicity ensures that a transaction's operations are handled as an independent unit and will either succeed or fail together. This is important because we can be certain of the status of the database in the event of an unpredictable scenario, such as a crash or power outage. If any aspect of the transaction failed, it would have either been completed successfully or rolled back.

If we continue with the above example, if money is deducted from the account and the transaction fails, the changes are discarded and the transaction doesn't go through.

Consistency

Consistency ensures that transactions only make predetermined, predictable modifications to tables. Transactional consistency makes sure that data corruption or mistakes don't have unintended effects on your table's integrity.

Let's go back to the earlier scenario and say you want to transfer money from your personal account to a work account once more, but your balance is negative. Even though you are aware that your account does not meet the requirements, you still attempt to make the transfer. You will be breaking the transaction's integrity, and the transaction will fail as a result.

Isolation

Isolation is a critical property of a transactional system that manages single or multiple transactions simultaneously. To prevent interference with each other, an isolated environment is created for each transaction. A good example of this is when two transactions for a $150 withdrawal begin at the same time, with each operation being carried out independently. This ensures that when both operations are completed, the balance will be $0 rather than $150.

Durability

Durability ensures that changes are maintained after a transaction completes and is written to the database. By doing this, it is guaranteed that data in the system will be saved even in the event of system breakdowns, power outages, etc.

Why are ACID Principles important?

ACID principles are important because they ensure that the data remains consistent and trustworthy, even when unexpected errors occur. They provide a reliable way to manage database transactions and ensure that the data is always available and accessible to users. Without the ACID principles, the integrity and consistency of the data would be compromised, leading to a loss of trust in the system and potentially significant consequences for the organization. Therefore, it is essential to understand and implement the ACID principles in any database management or software development project.

Conclusion

ACID principles are crucial for ensuring the integrity and consistency of data in any database management or software development project. By providing a reliable way to manage transactions, ACID principles ensure that data remains available and accessible to users, even in the event of unexpected errors or system failures. As such, organizations need to understand and implement ACID principles to maintain trustworthy and consistent data and avoid the potential consequences of compromised data integrity.

Let’s Connect

Reach out to me on Linkedin
Reach out to me on the Bird app ( Kindly follow I'll follow back immediately )
We can also connect in the comment section below

Access Token and Refresh Token: A Comprehensive Guide

Tijani Ayomide — Sun, 19 Feb 2023 06:02:10 +0000

Tokens are essential components of modern web application security. They are used to authenticate and authorize access to protected resources, such as APIs or databases, and ensure that only authorized users can access them. In this article, we will provide a thorough description of what tokens are and the many sorts of tokens, such as access tokens, refresh tokens, and more.

What are Tokens?

Tokens are digital credentials that allow users and applications to be authenticated and authorized. A token is a string of characters that indicates a specific permission grant and often comprises information about the person or client who receives the token. Tokens can be used to gain access to restricted resources such as APIs or web services, as well as to validate the user's or client's identity.

Types of Tokens

1. Access Token

An access token is a bearer token that is used to grant access to a protected resource. It is typically issued by an authentication server or identity provider and contains information about the user and their permissions.

An authentication server can issue JWTs as access tokens that contain information about the user and the permissions granted to the client in order to authenticate users and grant access to protected resources.

The access token is then presented to the resource server whenever the user tries to access a protected resource and is used to verify that the user is authorized to access the resource.

Access tokens are short-lived and expire after a certain amount of time, typically an hour or less. This ensures that even if an access token is compromised, it can only be used for a limited amount of time before it becomes invalid. Additionally, access tokens can be revoked at any time by the authentication server or the user, which makes them more secure than long-lived tokens.

2. Refresh Token

Refresh tokens are long-lived tokens that are used to obtain a new access token. They are typically issued along with an access token and can be used to request a new access token when the current one expires. Refresh tokens are more secure than storing credentials on a device or browser, as they can be revoked by the authentication server at any time.

Refresh tokens are usually kept separate from access tokens and are only used to obtain new access tokens. They are not passed along with API requests or used to authenticate users directly. This ensures that even if a refresh token is compromised, the attacker cannot use it to access protected resources directly.

3. ID Token

ID tokens are tokens that are used to authenticate the user and provide information about the user's identity. ID tokens are typically used in scenarios where a user logs in with an external identity provider (e.g., Google or Facebook). The ID token contains information about the user, such as their name and email address, and can be used by the client to authenticate the user.

4. Session Token

Session tokens are tokens that are used to maintain a user's session with a web application. When a user logs in to a web application, the server issues a session token that is used to identify the user's session. The session token is typically stored in a cookie on the user's computer and is used to authenticate the user on subsequent requests.

Note: There are various sorts of tokens used in modern authentication servers; the ones listed above are some of the most common.

Using Access Tokens and Refresh Tokens

A simple illustration of a Node.js application using JavaScript to show how access tokens and refresh tokens work.

Prep work

// Create a directory
mkdir desktop/access_token_refresh_token
// Change the directory into the folder
cd access_token_refresh_token
// Initialize npm
npm init -y

Install the required dependency for this project with the command below

npm install jsonwebtoken express

Because this is a simple app, we'll make two files: index.js will handle all of the app's logic, and auth.middleware.js will handle token authorization and checking.

touch index.js
touch auth.middleware.js

Once that is created let us create the logic for the app

// Index.js
import express from 'express';
import jwt from 'jsonwebtoken';
import { authenticateToken } from './auth.middleware.js';

const app = express();

app.use(express.json());

// Generate a secret key for signing JWTs (normally this would be a long and random string)
const secret = 'very_Secure_secret_key';
const refresh_token_1 = "enter user1 refresh token here";

// In a real application, this data would be stored in a database
const users = [
  { id: 1, username: 'user1', password: 'password1' },
  { id: 2, username: 'user2', password: 'password2' },
];

// Index route
app.get('/', (_req, res) => {
    return res.status(200).send({ message: "access_token_refresh_token"});
});

// Route for logging in and generating tokens
app.post('/login', (req, res) => {
  // In a real application, this would be validated against a database
  const user = users.find(u => u.username === req.body.username && u.password === req.body.password);

  if (user) {
    // Generate an access token with a short expiry time (e.g. 10 minutes)
    const accessToken = jwt.sign({ userId: user.id }, secret, { expiresIn: '1m' });

    // Generate a refresh token with a long expiry time (e.g. 30 days)
    const refreshToken = jwt.sign({ userId: user.id }, secret, { expiresIn: '30d' });

    res.json({ accessToken, refreshToken });
  } else {
    res.status(401).json({ error: 'Invalid username or password' });
  }
});

// In a real application, this would retrieve data from a database based on the user id
app.get('/protected', authenticateToken, (_req, res) => {
    return res.status(200).send({ message: "This is the protected message "});
})

app.listen(3000, () => {
  console.log('Server listening on port 3000');
});

he code above consists of two main parts: a login route and a protected route. When the server starts, it generates a secret key that will be used to sign and verify JWTs. In a real-world application, this key would be kept secret and not hardcoded in the source code.

The users array contains some hardcoded user data that will be used to validate the login credentials. Again, in a real-world application, this data would typically be stored in a database.

The root route just returns a simple JSON message that says "access_token_refresh_token" and is used for testing purposes.

The /login route is where the user logs in and receives both an access token and a refresh token. The user's credentials are validated against the users array, and if they are valid, an access token and a refresh token are generated. The access token has a short expiry time of 1 minute, while the refresh token has a longer expiry time of 30 days. The tokens are signed using the secret key and returned to the client in a JSON response.

The /protected route is where the user can access a protected resource. This route is protected by the authenticateToken middleware function, which checks the validity of the user's access token. If the access token is valid, the user is allowed to access the protected resource.

When the client sends a request to the /protected route, it includes the access token in the Authorization header. The authenticateToken middleware function extracts the token from the header, verifies its validity using the secret key, and if the token is valid, allows the request to proceed to the protected route. If the token is not valid, the middleware function returns a 401 Unauthorized status code.

// auth.middleware.js
import jwt from 'jsonwebtoken';
const secret = 'very_Secure_secret_key';

export function authenticateToken(req, res, next) {
  const authHeader = req.headers.authorization;
  const token = authHeader && authHeader.split(' ')[1];
  if (!token) {
    return res.status(401).json({ error: 'Access token not provided' });
  }
  jwt.verify(token, secret, (err, payload) => {
    if (err) {
      return res.status(403).json({ error: 'Invalid or expired access token' });
    }
    // Attach the user ID to the request object for use in subsequent handlers
    req.id = payload.id;
    next();
  });
}

If the access token expires, the client can use the refresh token to obtain a new access token without having to log in again. In a real-world application, this would typically involve sending the refresh token to the server in a separate request, which would then generate a new access token if the refresh token is still valid. However, this functionality is not implemented in the code above.

Let us now use the refresh token that was provided. You will see that the accessToken is configured to expire in 1 minute (intentionally) even if the optimal length in a real-world application exceeds that. When the accessToken expires, the user will no longer be able to request that route, which is where the /refresh the route comes in.

// Route for refreshing tokens
app.post('/refresh', (req, res) => {
  const refreshToken = req.body.refreshToken;

  // Verify that the refresh token is valid and retrieve the user ID from it
  jwt.verify(refreshToken, secret, (err, decoded) => {
    if (err) {
      return res.status(401).json({ error: 'Invalid refresh token' });
    }
    const userId = decoded.id;
    // Check if the refresh token is in the database and has not expired
    // This would typically be done using a database query
    const refreshTokens = [
      { userId: 1, refreshToken: 'refresh_token_1', expiry: Date.now() + 30 * 24 * 60 * 60 * 1000 },
      { userId: 2, refreshToken: 'refresh_token_2', expiry: Date.now() + 30 * 24 * 60 * 60 * 1000 },
    ];

    const storedToken = refreshTokens.find((token) => token.refreshToken === refreshToken);

    if (!storedToken || storedToken.userId !== userId || storedToken.expiry < Date.now()) {
      return res.status(401).json({ error: 'Invalid refresh token' });
    }

    // Generate a new access token with a short expiry time (e.g. 10 minutes)
    const accessToken = jwt.sign({ userId }, secret, { expiresIn: '10m' });

    // Generate a new refresh token with a long expiry time (e.g. 30 days)
    const newRefreshToken = jwt.sign({ userId }, secret, { expiresIn: '30d' });

    // Update the refresh token in the database with the new value and expiry time
    storedToken.refreshToken = newRefreshToken;
    storedToken.expiry = Date.now() + 30 * 24 * 60 * 60 * 1000;

    // Return the new access token and refresh token to the client
    res.json({ accessToken, refreshToken: newRefreshToken });
  });
});

When a client sends a request to this route with a valid refresh token, the server generates a new access token and a new refresh token and returns them to the client.

The code starts by extracting the refresh token from the request body

const refreshToken = req.body.refreshToken;

It then verifies the refresh token by calling jwt.verify() , this function checks if the token is valid and has not expired. If the token is invalid or has expired, the server returns a 401 response with an error message

jwt.verify(refreshToken, secret, (err, decoded) => {
  if (err) {
    return res.status(401).json({ error: 'Invalid refresh token passed.' });
  }
    // If the refresh token is valid, the server extracts the user ID from the decoded token
  const userId = decoded.id;
});

The server then checks if the refresh token is stored in the database and has not expired. In this example, the refresh tokens are stored in an array of objects

const refreshTokens = [
  { userId: 1, refreshToken: refresh_token_1, expiry: Date.now() + 30 * 24 * 60 * 60 * 1000 }, // expires in 30 days
  { userId: 2, refreshToken: 'refresh_token_2', expiry: Date.now() + 30 * 24 * 60 * 60 * 1000 }, // expires in 30 days
];
const storedToken = refreshTokens.find((token) => token.refreshToken === refreshToken);
if (!storedToken || storedToken.userId !== userId || storedToken.expiry < Date.now()) {
  return res.status(401).json({ error: 'Invalid refresh token' });
}

The server searches the refreshTokens array for a token that matches the refresh token provided by the client. If a matching token is found, the server checks if the user ID in the token matches the user ID extracted from the decoded token and if the token has not expired. If any of these checks fail, the server returns a 401 response with an error message. If the refresh token is valid and has not expired, the server generates a new access token and a new refresh token and returns them to the client

const accessToken = jwt.sign({ userId }, secret, { expiresIn: '10m' });
const newRefreshToken = jwt.sign({ userId }, secret, { expiresIn: '30d' });

storedToken.refreshToken = newRefreshToken;
storedToken.expiry = Date.now() + 30 * 24 * 60 * 60 * 1000;

res.json({ "accessToken": accessToken, "refreshToken": newRefreshToken });

The server calls jwt.sign() to generate a new access token and a new refresh token with short and long expiry times, respectively. It then updates the refresh token in the database with the new value and expiry time, and returns the new access token and refresh token to the client in a JSON response.

Wrapping Up

Access tokens and refresh tokens are essential components of modern web applications that require user authentication. A well-designed token-based authentication system can help prevent unwanted access to restricted resources while also providing an efficient approach to managing user authentication across numerous devices and applications. By effectively implementing token-based authentication, developers can protect the security and privacy of user data while providing a seamless and streamlined user experience.

Let’s Connect

Reach out to me on Linkedin
Reach out to me on the Bird app ( Kindly follow I'll follow back immediately )
We can also connect in the comment section below (Also link to the GitHub repo can be found here)

Prisma in 500 Seconds

Tijani Ayomide — Thu, 09 Feb 2023 08:04:39 +0000

With so many ORMs available, deciding which one to use for your JavaScript-based project can be difficult. You have a ton of options depending on your objectives or stack, including libraries like TypeORM, Sequelize, and Mongoose.

In this article, we'll be delving deeply into a different choice: one that provides a ton of fascinating features, a distinctive "ORM" experience, and a vibrant, committed team of developers supporting and working on it. It's called Prisma.

What is Prisma?

Prisma is a modern, open-source tool that enables developers to build scalable, high-performance data access layers with ease. Whether you're building a web or mobile application, Prisma has everything you need to connect to your databases and manage your data with confidence. It consists of the following parts:

Prisma Client: Auto-generated and type-safe query builder for Node.js & TypeScript
Prisma Migrate: Migration system
Prisma Studio: GUI to view and edit data in your database.

How does Prisma work?

Prisma makes use of a Prisma Schema File, which allows developers to define their application models in an intuitive data modeling language. It also contains the connection to a database and defines a generator.

// Relational Database
datasource db {
  provider = "postgresql"
  url      = env("DATABASE_URL")
}

generator client {
  provider = "prisma-client-js"
}

Let's briefly discuss on datasource db and Generator in the schema file

In Prisma, a "data source" refers to the database that you are connecting to in order to manage your data. This data source could be a relational database such as MySQL or PostgreSQL, or it could be a NoSQL database such as MongoDB or Cassandra. Prisma provides a unified API for connecting to and querying your data source, regardless of the type of database you are using.

The generator in Prisma is a tool that generates code based on your database schema. In other words, it creates code that corresponds to the structure of your database, including the tables, columns, and relationships between tables.

Auto-formatting

Prisma provides a dedicated extension that assists with code highlighting and code formatting automatically if you are using an IDE like vscode. Use your IDE's extension pane shortcut or the keyboard shortcut "ctrl+shift+X" to get there.

If you install this extension, your Prisma files should automatically format; otherwise, just complete the next step to set up auto-formatting.

File >> Preferences >> edit settings has JSON (top right corner)

// Copy and paste it into your settings.json
"[prisma]": {
        "editor.defaultFormatter": "Prisma.prisma",
        "editor.formatOnSave": true
    },

With this change, Prisma will automatically format your code upon saving, which I find to be really helpful. And don't worry, Prisma has a built-in formatter if you are unable to do all of this.

npx prisma format

Jumping In

To get started with Prisma, the first step is to install it. Prisma provides a CLI that makes it easy to install and manage the various components of the platform. The installation process is straightforward and only takes a few minutes to complete. Simply run the following command in your terminal:

npm install -g prisma
// This will install prisma globally on your machine

Initialize

Once you have installed Prisma globally on your machine, you can then initialize it inside your project with the following command in your terminal:

npx prisma init

This command will create a schema folder in your project directory and a `.env` file, which will be used to store the database URL. The schema folder will house the database structure or model, which is used to define the desired state of the database.

It is important to note that the `npx prisma init` command sets up the necessary infrastructure for using Prisma in your project, including the creation of the schema folder and the .env file. By defining the database structure in the schema folder, you can effectively communicate the desired state of the database to the Prisma CLI, which can then be used to generate the necessary code for accessing and manipulating the database.

Migration

The migration system in Prisma is a key feature that allows you to make changes to your database schema over time. Prisma migrations ensure that your database stays in sync with your application's data model, making it easy to add, modify, or delete tables, columns, and relationships. For example, you can use the following code to create a table in your database:

generator client {
  provider = "prisma-client-js"
}

datasource db {
  provider = "postgresql"
  url      = env("DATABASE_URL")
}

model User {
  id        String   @id @default(uuid())
  email     String   @unique
  name      String
  createdAt DateTime @default(now())
  updatedAt DateTime @updatedAt
}

npx migrate dev --name init

The command above is used to create a new migration in your Prisma project. The migrate sub-command is used to manage migrations for your project, and the dev option specifies that the migration should be created in the development environment. The --name option is used to specify a custom name for the migration, in this case init.

When you run this command, Prisma will create a new migration in your project's migrations directory with the specified name. This migration will contain a set of instructions for updating your database schema. You can then use the Prisma CLI to apply the migration to your database by using the npx prisma migrate up command.

npx prisma migrate up

npx prisma migrate up is a specific sub-command of the migrate command. It is used to apply pending migrations to your database. When you run npx prisma migrate up, Prisma will look at the migrations that have been created for your project and apply any that have not yet been executed in your database. This will update your database schema to reflect the changes you have made in your migrations

Prisma Client

The Prisma Client is an important component of the Prisma ecosystem and plays a critical role in the interaction between your application and database. It is a client library that provides a type-safe and convenient API for querying and managing your data in Node.js and TypeScript applications.

To install the Prisma client, you can run the following commands in your terminal:

npm i --save-dev prisma@latest
npm i @prisma/client@latest

The Prisma client is generated based on the data model defined in your Prisma schema and is always in sync with the structure of your database. This guarantees type safety and prevent runtime errors, making it easier to write and maintain your code.

Using the Prisma client in your application is straightforward. To get started, you can create an instance of the PrismaClient class and use its methods to interact with your database.

import { PrismaClient } from '@prisma/client';
const prisma = new PrismaClient();

async function getAllUsers() {
    const allUsers = await prisma.user.findMany();
    return allUsers;
}
getAllUsers().catch(err => console.error(err));

With a few lines of code, you can retrieve all the records from the User table in your database and log the results.

The Prisma Client also provides a number of other methods for querying, updating, and managing your data, including methods for creating, updating, and deleting records and for working with transactions.

Prisma Studio

Prisma Studio is a graphical user interface (GUI) tool that provides a convenient and intuitive way to interact with your Prisma-powered database. With Prisma Studio, you can view and manage your data, run queries, and perform other database-related tasks, all from a user-friendly web interface.

npx prisma studio

This will start a local instance of Prisma Studio, which you can access by navigating to http://localhost:5555 in your web browser.

One of the key features of Prisma Studio is its visual data explorer, which provides an intuitive interface for querying and manipulating your data. You can easily view your data in a tabular format, run queries, and perform other data-related tasks, all from a single interface.

Relationships

Relationships are an essential aspect of most databases, and Prisma makes it easy to define and manage relationships between tables in your database. Relationships in Prisma can be one-to-one, one-to-many, or many-to-many, and are defined using the @relation directive in your Prisma schema.

Let's update our schema a bit in other to grasp the concept around relationships and see how Prisma handles them

model User {
  id               String          @id @default(uuid())
  email            String          @unique
  name             String
  posts            Post[]
  userPreferences  userPreference?
  createdAt        DateTime        @default(now())
  updatedAt        DateTime        @updatedAt
}

model Post {
  id        String   @id @default(uuid())
  title     String
  content   String
  authorId  String
  author    User     @relation(fields: [authorId], references: [id])
  tags      Tags[]
  createdAt DateTime @default(now())
  updatedAt DateTime @updatedAt
}

model Tags {
  id    String @id @default(uuid())
  tags  String
  posts Post[]
}

model userPreference {
  id           String  @id @default(uuid())
  emailUpdate Boolean
  userId       String  @unique
  user         User    @relation(fields: [userId], references: [id])
}

I updated the schema with 3 extra models Post, Tags, UserPreference

`User` and `Post`: One-to-Many

The User model has a one-to-many relationship with the Post model. This relationship is established through the posts field in the User model and the author field in the Post model.

The posts field in the User model is an array of Post records, representing the posts created by that user. On the other hand, the author field in the Post model is a single User record representing the author of the post.

`User` and `UserPreference`: One-to-One

The User model also has a one-to-one relationship with the UserPreference model. This relationship is established through the userPreferences field in the User model and the user field in the UserPreference model.

The userPreferences field in the User model is a single UserPreference record representing the preferences of the user. The user field in the UserPreference model is a single User record representing the user that the preferences belong to.

`Post` and `Tags`: Many-to-Many

Finally, the Post model has a many-to-many relationship with the Tags model. This relationship is established through the tags field in the Post model and the posts field in the Tags model.

The tags field in the Post model is an array of Tags records representing the tags associated with the post. The posts field in the Tags model is an array of Post records representing the posts associated with the tag.

Wrapping Up

Prisma opens up a wealth of opportunities for creating and maintaining efficient and effective data structures. With its rich set of features and intuitive syntax, Prisma makes it easier than ever to connect, store, and manipulate data in a way that's both powerful and straightforward. If you want to dig deeper and expand on what we talked about, check out the Prisma Documentation.

Let's Connect

Reach out to me on Linkedin
Reach out to me on the Bird app ( Kindly follow I'll follow back immediately )
We can also connect in the comment section below (Also link to the GitHub repo can be found here)

SSL 101 - Securing Your Online Presence

Tijani Ayomide — Sat, 21 Jan 2023 15:49:23 +0000

Introduction

When you visit a website, the information that you share, such as your personal or financial data, needs to be kept private and secure. That's where Secure Socket Layer (SSL) certificates come in. These certificates are an essential component of online security and are necessary to establish a secure connection between a website and its visitors. In this article, we will discuss the importance of SSL certificates, how they work, the different types available, the difference between SSL and its successor, Transport Layer Security (TLS), and the various ways to obtain an SSL certificate.

Why do you need an SSL certificate?

An SSL certificate is necessary to establish a secure connection between a website and its visitors. This is particularly important when sensitive information, such as personal or financial data, is exchanged. Without an SSL certificate, this information can be intercepted and read by third parties (attackers), leading to potential fraud or identity theft. In addition, many modern web browsers will display a warning message when attempting to access a website without an SSL certificate, which can damage the trust of the website's visitors.

Understanding how SSL certificates work

An SSL certificate works by creating an encrypted connection between a website's server and the visitor's web browser. When a visitor accesses a website with an SSL certificate, their browser verifies the certificate's authenticity and establishes an encrypted connection. This encrypted connection ensures that any information exchanged between the website and the visitor is kept private and secure.

Under the hood the browser verifies that the certificate provided by the website:

Is valid for the same domain as the one being accessed.
Is been issued by a trusted CA (Certificate Authority).
Is valid and has not passed its expiration date.

Once the user's browser has verified the validity of the SSL certification, the connection continues to be secure. If not, you will get a “not secure” warning in your browser, or it will deny access to the site. If successful, the browser and website server exchange the necessary details to form a secure connection and the site loads.

What are the different kinds of SSL?

There are several different types of SSL certificates available, each with varying levels of security and validation. They include

Domain Validated (DV) Certificates

This is the most basic type of SSL certificate and only verifies the ownership of the domain. The certificate authority (CA) will check that the person requesting the certificate has the authority to request it for the domain. Once the domain ownership is verified, the CA will issue the certificate. DV certificates are typically the quickest and easiest to obtain, but they offer the lowest level of trust and security.

Organization Validated (OV) Certificates

OV certificates provide a higher level of validation than DV certificates. In addition to verifying the ownership of the domain, the CA will also verify the organization behind the domain. This includes checking the organization's legal existence, address, and phone number. OV certificates take a bit longer to obtain than DV certificates, but they offer a higher level of trust and security.

Extended Validation (EV) Certificates

EV certificates offer the highest level of validation and trust. In addition to verifying the ownership of the domain and the organization behind the domain, the CA will also conduct a rigorous vetting process to verify the organization's identity. This process includes checking government-issued documentation, conducting background checks on the organization's officers, and physically verifying the organization's address. EV certificates can take several days to a couple of weeks to obtain, but they offer the highest level of trust and security.

Wildcard Certificates

A Wildcard certificate is a type of certificate that allows the certificate holder to secure any subdomains under a single domain name. For example, if you own the domain "example.com" and have a wildcard certificate, you can secure "sub1.example.com", "sub2.example.com", "sub3.example.com" etc. with a single certificate. It can be a cost-effective solution for organizations that have multiple subdomains and want to secure them all with a single certificate.

Multi-Domain Certificates (SAN)

Multi-Domain Certificates (SAN) are also called “Subject Alternative Name” (SAN) certificates. It allows the certificate holder to secure multiple domain names with a single certificate. For example, if you own the domains "example.com" and "example.net," you can secure both of them with a single SAN certificate.

Self-Signed Certificates

Self-signed certificates are issued by the entity that runs the server rather than a trusted third-party CA. These types of certificates are suitable for internal use and testing, but they are not recommended for public-facing websites because they are not trusted by web browsers and may generate security warnings for visitors.

It's important to note that, depending on the level of security and trust you need, you should choose the appropriate type of SSL certificate for your website

What is TLS and how does it differ from SSL?

Transport Layer Security (TLS) is the successor to SSL and is considered to be more secure. Both SSL and TLS are cryptographic protocols that provide secure communication over the internet, but TLS has several key differences. One of the main differences is that TLS uses a stronger encryption algorithm, making it more difficult for third parties to intercept and read the information exchanged between a website and its visitors. Additionally, TLS requires the use of a certificate for server identification, which is not required for SSL.

Different ways one can obtain an SSL certificate

There are several ways to obtain an SSL certificate, including purchasing one from a reputable certificate authority (CA) such as DigiCert, GlobalSign, or Comodo, or using a free certificate from Let's Encrypt. Additionally, some web hosting providers may offer SSL certificates as part of their hosting packages.

Conclusion

SSL certificates are a vital component of online security, ensuring that sensitive information exchanged between a website and its visitors is kept private and secure. There are various types of SSL certificates available, each with varying levels of security and validation. Its successor, TLS, provides a stronger encryption algorithm and requires a certificate for server identification. Obtaining an SSL certificate can be done by purchasing it from a reputable certificate authority, using a free certificate, or using one that some web hosting providers offer as part of their hosting package.

Let's Connect

Reach out to me on Linkedin
Reach out to me on the Bird app ( Kindly follow I'll follow back immediately )
We can also connect in the comment section below (Leave your thought...)

Building your own LinkedIn Profile Scraper

Tijani Ayomide — Tue, 17 Jan 2023 19:15:55 +0000

Scraping is a computer technique for retrieving information from a web page and reusing it in another context.

Using bots to retrieve and extract information and content from a website is known as "web scraping." Web scrapers are programs that search databases for information by using software, scripts, and other methods.

Why Scrape LinkedIn ❓

As was already indicated, scraping can be a helpful tool for extracting information from a website. Online data can be retrieved by scraping. For instance, you can utilize LinkedIn's enormous user base of more than 828 million members by gathering data from its members' public profiles...

These members' profiles can then be queried in an excel or CSV file, allowing you to undertake operations that are specific to your needs. For instance, I discovered that it may be quite time-consuming to search through a ton of recruiter profiles in an effort to find any information, such as emails, contact information, and more. As a result, I considered how I could use Python to help automate this process. We all have different reasons for doing things, and yours might include gathering contact information for people who work at a company you'd love to work for and obtaining their emails so you can set up phone calls and start a conversation, among other things.

People have a misperception about scraping that makes it appear criminal; nonetheless, web scraping is completely legal. Although it has drawbacks, those won't be covered in this post; instead, you'll learn how to use Python packages and tools to make a LinkedIn profile scraper.

Pre-requisites 🔎

Make sure you have the following libraries and languages installed on your machine

Python
BeautifulSoup
Selenium
Chrome WebDriver

Beautiful Soup

Beautiful Soup is a Python library for pulling data out of HTML and XML files. It creates parse trees from the XML or HTML file for easy traversal and manipulation, and allows the user to search the parse tree using a variety of filters and search queries to extract the desired data.

$ pip install beautifulsoup4

Selenium

Selenium is a popular open-source tool that is commonly used for automated testing of web applications. Selenium provides a way to interact with web pages through a web driver, which can simulate a user interacting with the page, such as clicking buttons, filling out forms, and navigating through pages.

$ pip install selenium

Chrome WebDriver

Chrome WebDriver is a separate executable that WebDriver clients can use to interact with the Chrome browser. It is a part of the Selenium project and is used for automating web applications for testing purposes.

P.S Find your browser's version on the settings page, then download the driver for the particular version you own.

Python

Python is a high-level, interpreted programming language that is widely used for a variety of tasks such as web development, data analysis, artificial intelligence, and scientific computing. Python is a cross-platform language, which means that it can run on multiple operating systems such as Windows, macOS, and Linux.

Implementation 🛠️

We can start putting our scraper into action to scrape profiles as soon as you have the essential libraries loaded into your machine.

// Importing modules
import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_condition as EC
from selenium.webdriver.support.ui import webDriverWait
from bs4 import BeautifulSoup as bs

"""
Note: The path included is the current location of the driver downloaded
"""
// The location where my driver is installed
PATH = "C:Program Files (x86)"  chromedriver.exe' 
driver = webdriver.Chrome(PATH)

// Webpage to be accessed
driver.get("https://www.linkedin.com/")

The task can be completed more rapidly by splitting the steps the bot will take. The breakdown consists of the following:

Authentication
Search
Accessing a profile URL
Getting useful information from the accessed profile and storing them inside a CSV file

Authentication

Create a function to handle the authentication aspect of the process

def authenticate():
    try:
        email_field = driver.find_element(By.XPATH, "//*[@id="session_key"]")
        email_field.send_keys("User/email address")

        // Program should sleep for 3 secs
        time.sleep(3)

        password_field = driver.find_element(By.XPATH, "//*[@id="session_password"]")
        password_field.send_keys("Password")

        time.sleep(3)

        login_button = driver.find_element(By.XPATH, "//*[@id="main-content"]/section[1]/div/div/form/button")
        login_button.click()
    except:
        print('An error occurred')
        driver.quit()

authenticate()

The XPATH is used to access each field (email, username, and password) in the authentication function above. This can be obtained by "inspecting" the field you've specified using your browser's "inspect" menu.

Selenium by default gives us access to attributes that can be used to access particular HTML components. i.e.

By.CLASS_NAME
By.XPATH
By. ID
By.TAG_NAME
By.NAME

Search

The next step in the breakdown process is searching; you can query the role of jobs and then filter by people to get a list of profiles that have that role in their profile.

time.sleep(2)

def search():
    try:
        // Confirm the page is loaded before searching
        WebDriverWait(driver, 10).until(
            EC.presence_of_element_located((By.CLASS_NAME, 'authentication-outlet'))
        )
        search = driver.find_element(By.XPATH, '//*[@id="global-nav-typeahead"]/input')

        // Search Input
        search.send_keys('Software Developer')

        // Search Input: ENTER
        search.send_keys(Keys.RETURN)

        WebDriverWait(driver, 10).until(
            EC.presence_of_element_located((By.XPATH, '/html/body/div[6]/div[3]/div[2]/section/div/nav/div/ul/li[2]/button'))
        )

        people_btn = driver.find_element(By.XPATH, '/html/body/div[6]/div[3]/div[2]/section/div/nav/div/ul/li[2]/button')
        people_btn.click()
    except:
        print("An error occurred")
        driver.quit()

search()

The purpose of using time.sleep() is to simulate human time lag behavior; otherwise, the bot will execute all of its orders immediately, raising a flag that could result in your profile being blocked by Linkedin.

Accessing Profiles

Taking each profile URL and saving it in a list so that it may be retrieved

def profile():
    try:
        WebDriverWait(driver, 10).until(
            EC.presence_of_element_located((By.CLASS_NAME, 'search-results-container'))
        )
        page_source = bs(driver.page_source, 'lxml')
        profiles = page_source.find_all('a', class_ = 'app-aware-link')
        all_profile = []
        for profile in profiles:
            profile_ID = profile.get('href')
            if profile_ID not in all_profile:
                all_profile.append(profile_ID)
        return all_profile
    except:
        print('An error occurred')
        driver.quit()
profile()

Getting useful information

Looping over each profile URL to retrieve certain attributes that will be included in the CSV file

// import the CSV module
import csv

def convert_into_csv():
    try:
        page_url = profile()
        for urls in page_url:
            driver.get(urls)
            soup = bs(driver.page_source, 'lxml')
            profiles = soup.find_all('div', class_ = 'ph0 pv2 artdeco-card mb2')
            for profile in profiles:   
                name = profile.find('h1', class_ = 'text-heading-xlarge'). text
                current_position = profile.find('div', class_ = 'text-body-medium').text
                location = profile.find('div', class_ = 'pb2 pv-text-details__left-panel')[0].text

                with open('output.csv', 'w', newline='') as file_output:
                    headers = ['Name', 'Postion', 'Location', 'URL']
                    writer = csv.DictWriter(file_output, delimiter=',', lineterminator='\n', fieldnames=headers)
                    writer.writeheader()

                    writer.writerow({headers[0]: name, headers[1]: current_position, headers[2]: location, headers[3]: profile})
    except:
        print('An error occurred)
        driver.quit()

convert_into_csv()

Conclusion

The Selenium module provides a simple API for Selenium WebDriver function writing. The HTML can then be parsed using a different tool called Beautiful Soup. Both packages are dependable and useful companions for your web scraping endeavors.

In this article, you learned how to use Selenium and Beautiful Soup to scrape data from LinkedIn. You completed the entire web scraping procedure from beginning to end and created a script that retrieves and stores user profile data from LinkedIn. Feel free to explore the possibilities with this extensive pipeline in mind and these two strong libraries in your toolkit.

Note: It is worth noting that scraping data from a website without permission is a violation of the website's terms of service and can be illegal.

Let's Connect

Reach out to me on Linkedin
Reach out to me on the Bird app ( Kindly follow I'll follow back immediately )
We can also connect in the comment section below (Leave your thought...)

What is versioning all about?

Tijani Ayomide — Sat, 07 Jan 2023 15:55:31 +0000

I found the name "versioning" in an article I read not long ago. The article talked about best practices when creating a REST API. I assume we are all tech-savvy here😉 but for those who aren't find below the Wikipedia version of what a REST API is

A REST API (also known as a "RESTful API") is an application programming interface (API or web API) that conforms to the constraints of the REST architectural style and allows for interaction with RESTful web services.

Quite cumbersome if you would agree… In simpler terms, a REST API is a way for two computer systems to communicate with each other over the internet. It stands for "Representational State Transfer" and is a set of rules for how the systems should send data back and forth to each other.

Imagine you are trying to get a glass of water from the kitchen. You would go to the kitchen and ask someone to give you a glass of water. They would then get a glass and fill it with water from the tap before giving it to you. In this situation, you are like a computer system, and the person in the kitchen is like another computer system. You are sending a request for a glass of water, and the other system is responding by giving you the water.

A REST API works in a similar way. One system (like a website) can send a request to another system (like a server) and ask it to do something, like get some data or save some information. The second system will then do what was asked and send a response back to the first system with the result.

Does that make sense?

Let’s try not to stray too much from the topic. Like any good developer, I searched Google for the ambiguous term at hand and discovered that I had already been using “versioning” in some of my projects; however, I hadn't given it much thought at the time or had chosen to use it because I saw others using it in their code. (p.s. : If there is a term you don't seem to understand, try searching for it before implementing.)

What is versioning ❓

"Versioning" refers to the technique of making different versions of a web API available and accessible to developers. This enables the API to adapt and expand over time while remaining backward-compatible with existing integrations.

There are several standard techniques for API versioning. One common approach is to include the API's version number in the base URL. For instance, an API might have a base URL of "https://api.example.com/v1" for version 1 and "https://api.example.com/v2" for version 2. A developer only needs to use the correct base URL to use a certain version of the API.

Another common approach is to utilize the HTTP request's Accept header to identify the version of the API that the client wants to use. The server will then respond with the appropriate version of the API.

To elaborate on the techniques I briefly mentioned above, there are four common approaches you might take when versioning.

URI Path Versioning
URL Parameter Versioning
Versioning with Content-Type in Accept Header
Versioning with Custom Header

URI Path Versioning

In URI path versioning, the version of the API is included in the URL path. For example, an API might have a base URL https://api.example.com/v1 for version 1, and a base URL https://api.example.com/v2 for version 2. When a developer wants to use a specific version of the API, they just need to use the corresponding base URL.

Here is an example of making a request to version 1 of an API using URI path versioning

fetch('https://api.example.com/v1/endpoint')
  .then(response => response.json())
  .then(data => console.log(data));

URL Parameter Versioning

In URL parameter versioning, the version of the API is included as a query parameter in the URL. For example, an API might have a base URL of https://api.example.com and the version could be specified using a version query parameter, like https://api.example.com?version=1.

Here is an example of making a request to version 1 of an API using URL parameter versioning

const params = {
  version: 1
}
fetch('https://api.example.com/endpoint', { params })
  .then(response => response.json())
  .then(data => console.log(data));

Versioning with Content-Type in Accept Header

In this approach, the version of the API is specified using the Accept header of the HTTP request. The server will then return the appropriate version of the API in the response.

Here is an example of making a request to version 1 of an API using the Accept header

const headers = {
  Accept: 'application/app.v1.categories'
}
fetch('https://api.example.com/endpoint', { headers })
  .then(response => response.json())
  .then(data => console.log(data));

Versioning with Custom Header

In this approach, a custom header is used to specify the version of the API that the client wants to use. The server will then return the appropriate version of the API in the response.

Here is an example of making a request to version 1 of an API using a custom header

const headers = {
  'X-API-Version': 1
}
fetch('https://api.example.com/endpoint', { headers })
  .then(response => response.json())
  .then(data => console.log(data));

To use this approach, the server will need to be configured to look for the custom header and return the appropriate version of the API based on the value of the header.

This approach can be useful if the API needs to support multiple versions and the other approaches (such as URI path versioning or URL parameter versioning) are not suitable.

Why is versioning important ❓

Versioning is important because it allows an API to change and evolve over time without breaking existing integrations. When a client (such as a website or mobile app) integrates with an API, it usually depends on the API to function properly. If the API changes in a way that is not backward-compatible, it can break the client's integration and cause the client to stop working.

Versioning helps to prevent this problem by allowing multiple versions of the API to exist and be used concurrently. This means that when a new version of the API is released, existing clients can continue to use the old version until they are able to update their integration to use the new version.

Versioning is also important because it allows developers to understand the changes that have been made to the API over time. By clearly documenting the changes made in each version of the API, developers can more easily understand how to use the latest version of the API and how it differs from previous versions.

Overall, versioning helps to ensure that an API can change and improve over time without causing problems for existing clients.

Best practices to remember 🤔💭

There are a few best practices to keep in mind when versioning an API:

Use versioning to make backward-compatible changes to the API. If you need to make breaking changes, create a new version of the API rather than changing the existing version.
Make it easy for developers to discover and use the latest version of the API.
Clearly document the changes made in each version of the API.

Note : It's crucial to thoroughly outline the changes made in each API version so that developers may understand how the most recent version differs from earlier versions and choose which version to use.

When not to use versioning ❌✋

There are a few situations when it might not be necessary to use versioning in an API:

If the API is still in the early stages of development and is not yet being used by any external clients, versioning may not be necessary.
If the API is not expected to change significantly over time, versioning may not be necessary.
If the API is intended to be used by a small group of developers who are able to quickly update their integrations in response to changes, versioning may not be necessary.

That being said, it's generally a good idea to version an API from the start, even if it's not expected to change much. This allows the API to evolve over time without breaking existing integrations. It's easier to add versioning from the beginning than it is to retroactively add it later on.

Conclusion ✍🏽🚶🏽

In the preceding article, you learned why versioning is an important technique while developing and maintaining a REST API, approaches to versioning, the value of versioning, when not to use versioning, and so on. It's crucial to choose the appropriate approach for your API and to fully outline the changes made in each API version to improve developers experience.

I'd love to receive feedback of any kind. If there's anything you'd like to tell me don't hesitate to reach out. Thanks for reading!

Stealth Startups and the Power of Proxycurl

Tijani Ayomide — Fri, 30 Dec 2022 13:46:49 +0000

What is a stealth startup?

Stealth startups are companies that operate in secret, often keeping their business plans and products under wraps until they are ready to launch them or reveal them to the public.

Why go stealth?

Why would a company go stealthy? A company may decide to run as a stealth startup for a number of reasons, including:

To perfect their product or service: By operating in secret, a company can focus on developing and refining their product or service without the pressure or scrutiny of the public or competitors. This can help ensure that the product or service is as high-quality as possible before it is released to the market.
To maintain a competitive advantage: By keeping its plans d progress secret, a company can maintain a competitive advantage over its rivals. In fields where innovation or technology is a major differentiator, this can be extremely crucial.
To avoid drawing attention from competitors: By operating in secret, a company can avoid drawing attention from competitors who might try to copy their ideas or disrupt their plans. This can give the company time to build a strong foundation and establish a foothold in the market before facing significant competition.
To attract top talent: By operating in secret, a company can attract top talent drawn to the excitement and challenge of working on a secret project. This can help the company build a strong team and work towards its goals more effectively.
To build anticipation for the product or service: By operating in secret, a company can build anticipation and interest in its product or service before it is released to the public. This can generate buzz and interest in the product, which helps drive demand and sales when it is finally launched.

This is not to say that the company that took the stealth approach had a perfect outcome. There are also several potential drawbacks to operating as a stealth startup, including

Difficulty attracting investment: It can be difficult for a stealth startup to attract investment or funding without being able to share details about the company or its plans. Without a thorough understanding of a company's business model or product, investors could be hesitant to invest in it.
Difficulty attracting and retaining top talent: It can also be difficult for a stealth startup to attract and retain top talent if the company cannot share details about the work it will be doing. This can limit the company's ability to build a strong team and work towards its goals effectively.
Limited ability to build buzz or anticipation: Operating in secret can also limit a company's ability to build buzz or anticipation for its product or service before it is released. This can make it harder for the company to generate demand and sales when the product is finally launched.
Increased pressure to succeed: Finally, operating as a stealth startup can increase the pressure on the company to succeed, as it may have limited opportunities to build buzz or anticipation for its product or service before it is released. This can add additional stress and pressure on the company's founders and employees.

Where does ProxyCurl fit in all of these?

Proxycurl is a tool that can scrape publicly available data from the internet to gather information about individuals or companies. It can be used to track the movements of entrepreneurial talents by collecting data about their professional experience, education, and current employment. This information can be useful for companies looking to identify potential candidates for employment or investment opportunities.

Proxycurl can also be used to follow the movements of current employees by collecting data about their professional experience and current employment. This can be useful for companies looking to track the career progression of their employees or identify potential retention risks. Check out how their product can be used to “Uncover wealth of opportunities with stealth startups”

Finally, Proxycurl can be used to identify investment signals by collecting data about a company's financial performance, funding rounds, and partnerships. This information can be useful for investors looking to identify promising investment opportunities.

Curious Minded? Check out here for more information about stealth startups.

Data Bias: From Question to Conclusion

Tijani Ayomide — Sat, 17 Sep 2022 08:00:47 +0000

Introduction

So, recently, I've been learning a lot about data analysis, and it's been a wild ride thanks to the GoogleXCoursera scholarship program. You will be amazed by what you discover about data, from how it is turned from its raw state into actual information to how it may be used to help business decision-makers make more educated choices.

Did you know ❓❓

Between the beginning of time and 2003, Google generated five exabytes of data. This quantity of data was being generated every two days in 2010, and every 40 minutes by 2021.

Data analysis is somewhat similar to software development (Backend), where you evaluate the logic of things and try to determine how an application should flow. However, in this context, instead of thinking about the flow, you think about the data and try to figure out if that particular data is answering the question, you need it to answer, and if so, how can you modify that data into valuable information.

A lot of things are overlooked during the process which can cause your data to be biased.

Why are we talking about data bias ❓

As a data analyst, business owner, or as general person you must consider bias and fairness from the moment you begin data collection until the point at which you offer your conclusion.

An example of bias is when you are asked to participate in a poll to determine the number of people who would like to go fishing. You go ahead and presume you asked every student in the class, but you omitted the impacted (wheelchair-bound, disabled, etc.) individuals. That survey's data collection is systematically biased.

Another example is being asked to take part in a survey of employees in your organization to see how much employees enjoy working from the office rather than working remotely, but ever since the Covid 19 incident, businesses have adopted a work-from-home/ hybrid policy. As a result, the survey was only taken by those who were present in the office at the time. You can tell right away that the data is biased since it does not fully represent the population (people at the office).

What does bias mean ❓

Before providing more context, it is essential to understand what bias is. Using the above examples, you can see how being biased can cause data to be skewed.

Simply put, bias refers to a preference for or against a person, group of people, or object. Since our perspective is more data analytical, data bias is an error that methodically skews outcomes in one direction.

Forms of Bias

Sample Bias

When a sample doesn't accurately reflect the entire population. This can be avoided by ensuring that the sample is chosen at random.

Unbiased Sampling

A sample that is representative of the population being studied.

Comment down below an instance of an unbiased sample ...

Observer Bias

They are sometimes referred to as experimental bias or research bias. This is the tendency for two persons working on a project together to see things differently. It is more like the 6-9 case.

Interpretation Bias

The tendency to interpret ambiguous situations in either a good or negative light.

Confirmation Bias

The tendency to seek out or interpret information in a way that reinforces pre-existing ideas. People see what they want to see and ignore other types of information/data.

Explore data credibility

The more high-quality data you have the more confidence you can when making data-driven decisions. You can rest assured that your data is credible if you follow a methodology called ROCCC.

Which is an acronym for Reliable, Original, Comprehensive, Cited and Current.

R - With this kind of data, you can be sure that you're getting unbiased and precise information because it comes from a trusted source.

O - You will eventually have to work with first party to third-party data; to ensure that you are working with reliable data and confirm with the original source.

C - The most accurate data has all the relevant details required to provide an answer or a solution.

C - Consider these three factors when picking a data source: Who developed the dataset? Does it belong to a reputable organization? when the information was last updated.

C - The usefulness of data diminishes with time, and the finest data sources are current and relevant to the task at hand.

If your data source has all these attributes, then your data “ROCCC” pun intended 😄.

Let’s Connect 🖇️

Reach out to me on LinkedIn
Reach out to me on the Bird app (Kindly follow I'll follow back immediately)
We can also connect in the comment section below (Leave your thought...)

DEV Community: Tijani Ayomide

From Postman to Postman Extension

Introduction

What is Postman?

What is the Postman extension?

Features and Capabilities

Sending of HTTP and Multiprotocol Requests

Collection and Workspace Creation

Managing Environments and more

Install Postman Extension

Conclusion

Let's Connect

Understanding dependencies and dev-dependencies: Beginner’s Guide

What is a dependency❓

What is a dev-dependency❓

Updating Dependencies 🔃

Updating All Dependency

Updating a Specific dependency

Conclusion

Let's Connect 🔗

Which Case is Best for Your Coding Style

What is Casing?

What is Snake Case?

What is Camel Case?

What is Pascal Case?

What is Kebab Case?

Conclusion

Let’s Connect

API Limiting: Best Practices and Implementation

What is API Limiting?

Types of API Limiting

When to use API-Limiting

Best Practice for API Limiting

Determine appropriate limits

Set clear and informative error messages

Allow for flexibility and exceptions

Implementing rate limiting in a node js application

Prerequisites

Creating the Project

Schema of the Project

Implementing Rate Limiting

Conclusion

Let's Connect

Understanding the Basics of ACID Principles in Database Management

Transaction

A.C.I.D Properties of a transaction

Atomicity

Consistency

Isolation

Durability

Why are ACID Principles important?

Conclusion

Let’s Connect

Access Token and Refresh Token: A Comprehensive Guide

What are Tokens?

Types of Tokens

1. Access Token

2. Refresh Token

3. ID Token

4. Session Token

Using Access Tokens and Refresh Tokens

Prep work

Wrapping Up

Let’s Connect

Prisma in 500 Seconds

What is Prisma?

How does Prisma work?

Auto-formatting

Jumping In

Initialize

Migration

Prisma Client

Prisma Studio

Relationships

User and Post: One-to-Many

User and UserPreference: One-to-One

Post and Tags: Many-to-Many

Wrapping Up

Let's Connect

SSL 101 - Securing Your Online Presence

`User` and `Post`: One-to-Many

`User` and `UserPreference`: One-to-One

`Post` and `Tags`: Many-to-Many