DEV Community: Israel Ayanwola

Streamlining Big Number Calculations in C using GMP

Israel Ayanwola — Sat, 29 Jul 2023 11:01:29 +0000

Nowadays, technologies like Cryptography need computers to compute complex mathematical operations. These algorithms need the CPU to perform arithmetic operations on large numbers with high precision.

The amount of data a processor can execute at once depends on its number of virtual memory addresses. The maximum integer representation a 32-bit processor can work with is 4,294,967,295 (2^32 − 1), while for a 64-bit processor is 18,446,744,073,709,551,615 (2^64 − 1). These bit constraints affect C standard integer and float data types.

Most high-level languages, like Python, allows programmers to work with larger numbers. Allowing programmers to write programs for processing numbers larger than what the CPU can handle. It's possible to write programs for calculating big numbers in C, but it can get very complex and confusing.

GMP, GNU Multiple Precision Arithmetic Library, streamlines the complexity of working with big numbers in C. This article covers how you can use the GMP Library to perform basic mathematical operations on various kinds of numbers.

Prerequisites

To follow this article, you need to have;

A basic understanding of C,
GCC installed - for compiling C programs,
Basic understanding of GCC,
Access to the internet - for installing GMP, and
A text editor was installed.

What is GMP?

GMP is a free library for arbitrary precision arithmetic, operating on signed integers, rational numbers, and floating-point numbers. There is no practical limit to the precision except the ones implied by the available memory in the machine GMP runs on. GMP has a rich set of functions, and the functions have a regular interface. - gmplib.org

GMP allows you to calculate big numbers of many kinds with high precision. It represents numbers in its program as strings.

Installing GMP

To install GMP library, you can follow this informative guide. It explains how you can download, configure and build the library on Window, Mac and Linux.

Performing Arithmetical Operations Using GMP

After installing the library, you need to include the GMP header file in your program to use the library for calculations.

Addition

For adding two integers, you need to declare and initialize the variables in a special way before calculations.

Include GMP library header. This allows you to access the types and macros declared in the header file.

#include <gmp.h>

Declare and initialize the variables for the result, and the numbers to add together.

mpz_t res, num_a, num_b;

/* Initializing integers */
mpz_init(res), mpz_init(num_a), mpz_init(num_b);

Initializing the variables using mpz_init function tells GMP to allocate memory space to the variables. Then you can assign any large number to the initialized variables.

The lines below assigns 3479349783486297692763769376273723897236 to num_a and 9223372036854775807 to num_b. There is no limit to the amount of precision you get when using GMP for calculations.

/* Assigning integers */
mpz_set_str(num_a, "3479349783486297692763769376273723897236", 10);
mpz_set_str(num_b, "9223372036854775807", 10);

The mpz_set_str function takes the first argument as the variable to assign a value. The second argument is the string representation of the integer you want to assign. The third argument represents the base of the integer (which, in this case, is the second argument). In the above example, the integers are in base 10, meaning decimal numbers.

Now you can add the number together using the mpz_add function from the GMP library. The function sets res to num_a + num_b. Then you can use gmp_printf to print the values. It works like C standard library printf function. The only difference is that gmp_printf allows you to format larger numbers. The %Zd flag in the format string for gmp_printf is to format large integers.

/* Addition */
mpz_add(res, num_a, num_b);
gmp_printf("Addition:\n%Zd + %Zd = %Zd\n\n", num_a, num_b, res);

Now you need to clear the variables. Clearing the variables frees the memory allocated for the variables.

/* Clearing memory */
mpz_clear(res), mpz_clear(num_a), mpz_clear(num_b);

The full program should look like this.

// main.c

#include <gmp.h>

int main(void)
{
    mpz_t res, num_a, num_b;

    /* Initializing integers */
    mpz_init(res), mpz_init(num_a), mpz_init(num_b);

    /* Assigning integers */
    mpz_set_str(num_a, "3479349783486297692763769376273723897236", 10);
    mpz_set_str(num_b, "9223372036854775807", 10);

    /* Addition */
    mpz_add(res, num_a, num_b);
    gmp_printf("Addition:\n%Zd + %Zd = %Zd\n\n", num_a, num_b, res);

    /* Clearing memory */
    mpz_clear(res), mpz_clear(num_a), mpz_clear(num_b);

    return (0);
}

You need to link the GMP library when compiling your program. You can only link the dynamic or static library if you have installed GMP on your machine. If you haven’t, go back to the installation section above.

Run the command below to build your program. The -lgmp flag tells gcc to search for the GMP library when linking. GCC will search the list of standard directories to find the library.

gcc main.c -o main -lgmp

If it could not find the gmp library on your machine. Try with the build flags below. The -L~/gmp-6.2.1/.libs flag tells gcc to also check the ~/gmp-6.2.1/.libs directory if it could not find the library using the standard list. For you, this path is the location you installed GMP.

gcc main.c -o main -L~/gmp-6.2.1/.libs -lgmp -static

That covers how you can add large numbers in C and build your program.

Subtraction

You can use the mpz_sub function from the GMP library to substract large numbers. Ensure you initialize the variables and assign their integers respectively.

This subtracts num_b from num_a and assigns the result to res.

/* Subtraction */
mpz_sub(res, num_b, num_a);
gmp_printf("Substraction:\n%Zd - %Zd = %Zd\n\n", num_a, num_b, res);

Your full program might look like this.

// main.c

#include <gmp.h>

int main(void)
{
    mpz_t res, num_a, num_b;

    /* Initializing integers */
    mpz_init(res), mpz_init(num_a), mpz_init(num_b);

    /* Assigning integers */
    mpz_set_str(num_a, "3434234245245245252452452452", 10);
    mpz_set_str(num_b, "9999943444444434344444444434232323232322", 10);

    /* Subtraction */
    mpz_sub(res, num_a, num_b);
    gmp_printf("Substraction:\n%Zd - %Zd = %Zd\n\n", num_a, num_b, res);

    /* Clearing memory */
    mpz_clear(res), mpz_clear(num_a), mpz_clear(num_b);

    return (0);
}

Multiplication

For multiplying integers, use mpz_mul function from the GMP library.

This multiplies num_a and num_b and assigns result to res.

/* Multiplication */
mpz_mul(res, num_a, num_b);
gmp_printf("Multiplication:\n%Zd x %Zd = %Zd\n\n", num_a, num_b, res);

Division

For division, things will look a little different. Instead of using the type mpz_t to declare integers as you have seen before, you will use mpf_t instead to declare floats.

To divide large numbers using GMP you need to declare and initialize floats and not integers.

mpf_t div_res, div_a, div_b;

Then you initialize the variables using mpf_init.

/* Initializing floats */
mpf_init(div_res), mpf_init(div_a), mpf_init(div_b);

Assign values you want to the variables. They can be as large as needed. The last argument to mpf_set_str tells GMP that the string is in a decimal format.

/* Assign floats */
mpf_set_str(div_a, "7347973477376467734.34322335550000000000", 10);
mpf_set_str(div_b, "489384838948848.4783873847", 10);

Divide and print out the results using the functions below. The GMP function mpf_div divides div_a by div_b and assigns the result to div_res. Then prints out the formatted string.

/* Division */
mpf_div(div_res, div_a, div_b);
gmp_printf("Division:\n%Ff / %Ff = %Ff\n\n", div_a, div_b, div_res);

Your final code should look like this.

// main.c

#include <gmp.h>

int main(void)
{
    mpf_t div_res, div_a, div_b;

    /* Initializing floats */
    mpf_init(div_res), mpf_init(div_a), mpf_init(div_b);

    /* Assign floats */
    mpf_set_str(div_a, "7347973477376467734.34322335550000000000", 10);
    mpf_set_str(div_b, "489384838948848.4783873847", 10);

    /* Division */
    mpf_div(div_res, div_a, div_b);
    gmp_printf("Division:\n%Ff / %Ff = %Ff\n\n", div_a, div_b, div_res);

    /* Clearing memory */
    mpf_clear(div_res), mpf_clear(div_a), mpf_clear(div_b);

    return (0);
}

Square root

Ensure you declare and initialize the float variables. Then calculate the square root of that variable using mpf_sqrt function. This function only accepts two arguments. The variable to square root and the variable to assign the result.

/* Squre root */
mpf_sqrt(div_res, div_a);
gmp_printf("Square root:\nsqrt(%Ff) = %Ff\n", div_a, div_res);

Conclusion

You can perform other arithmetical operations that are not covered in this article using GMP. Check out the resources section below for their documentation to learn more. It covers how you can perform more Arithmetical operations with Logical and Bitwise operations. Also you can calculate using C int types and rational numbers.

Use the resources provided below to learn more about working with big numbers in C using the GMP library. If you are looking for a challenge try writing a C function to add two large numbers.

Thanks for reading this article. 😊

Resources

Linking libraries using GCC Compiler - https://gcc.gnu.org/onlinedocs/gcc/Link-Options.html
Installing GMP from the official website - https://gmplib.org/#DOWNLOAD
Guide to installing GMP on any machine - http://rstudio-pubs-static.s3.amazonaws.com/493124_a46782f9253a4b8193595b6b2a037d58.html

Mastering Django CI/CD using GitHub Actions

Israel Ayanwola — Sun, 02 Jul 2023 03:23:10 +0000

Testing and delivery is a common repetitive cycle in application development. This cycle consumes more time and resources as we fix or add more features to our application. Automating this repetitive process will reduce the time spent in application development.

Testing and Deployment are part of the cycle discussed earlier. We build, deploy and test our applications a lot. Doing this every time is vulnerable to human error. Automating our tests guarantees that every commit undergoes testing.

This article covers how we can use GitHub Actions to automatically test our Django applications.

Let’s dive in.

Prerequisites

To follow through with this article, we need the following:

Basic knowledge of Django, Git, and GitHub
A GitHub account
Python3 and Django installed on our operating system

Setup Demo Django Application

We need a Django application that has tests to understand how GitHub actions work. We can work with this demo Django project, which includes tests.

GitHub actions run on GitHub repositories and not on our machines. Only the owner of the repository can manage workflow runs. Later in the article, we will cover what Workflows are.

We need to fork the repository and clone it.



git clone <REPOSITORY_URL>

We should set up a new virtual environment.




# linux and mac
cd <PROJECT_DIR>
python3 -m venv venv
source venv/bin/activate




# windows
cd <PROJECT_DIR>
python3 -m venv venv
venv/Scripts/activate  # <-- difference

Now we need to install the project dependencies. Run this command in the project directory. This will install all the dependencies specified in requirements.txt file.



pip install -r requirements.txt

Now that our application setup is complete. Let’s run our tests will. The tests confirm the reliability of our application.



cd project
python manage.py test

The tests are successful. We can now go ahead and automate this process using GitHub actions.😊

Automated tests don’t only work when we use Django’s built-in test runners. We can write automation for any kind of test runners we are using. One popular test runner in the Python community is Pytest.

What are GitHub Actions?

To get started, we need to define a workflow. A workflow is a configurable automated process that will run one or more jobs. A step is a command or action that performs a basic task. An Action is a collection of steps, which makes an action reusable. A Job is an operation in a workflow that contains steps. Jobs can run in parallel or in sequence.

We define workflows by using a YAML file checked in the repository. A workflow will execute when triggered by an event in the same repository it exists.

GitHub workflows listen to many events, examples are push and pull requests. The events configured into our YAML file trigger our workflow. For example, when we push a new commit to the master branch, this can trigger a testing workflow.

Creating A Testing Workflow

Let’s create a workflow that executes our Django application tests.

Workflows are not for testing but for executing jobs which can be anything. An example of this is running tests. Our tests are already defined. We configure workflows to execute them. Other use cases are deployments and closing pull requests.

Create a new directory called .github/workflows in the root directory. Create a new file called test.yml in the .github/workflows directory. This file will be our workflow YAML configuration.




# test.yml
name: Run Django Tests
on:
  push:
    branches: [master]
jobs:
  test-django-app:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        python-version: ["3.9"]
    steps:
      - uses: actions/checkout@v3
        with:
          fetch-depth: 0
      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v3
        with:
          python-version: ${{ matrix.python-version }}
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
      - name: Run Django test
        run: |
          cd project
          python manage.py test

Let’s break down what each syntax means and how this executes the application tests.

We start by defining the workflow name by using the name syntax.



name: Run Django Tests

The workflow name helps us identify the workflow in our repository’s GitHub Actions tab.

The GitHub Actions tab helps us to manage workflow runs. See how to manage a workflow run on GitHub.

Next, we define the event that triggers the workflow.



on:
  push:
    branches: [master]

By using the on syntax, every commit we push to the master branch triggers the workflow run. We can specify many events and branches.

Next, we have the jobs syntax.



jobs:
  test-django-app:

A scenario to use this is when we want to deploy a web application after successful tests. We can create two jobs, one for testing and the other for deployment. Then configure the deployment to run after when tests are complete and successful.

The name of our job is test-django-app, we can name it anything. But make sure it’s descriptive, as it helps us identify the currently running job in the GitHub Actions tab.

Next, the runs-on syntax defines what machine will run the job. Here we instructed it to use ubuntu-latest version.



runs-on: ubuntu-latest

Next, the strategy syntax helps us define what Python versions we will use to test. In our workflow, we have defined only one version. The job will use each Python version declared in the list to execute itself in parallel.



strategy:
  fail-fast: false
  matrix:
    python-version: ["3.9"]

fail-fast applies to the entire matrix. If fail-fast is set to true, GitHub will cancel all in-progress and queued jobs in the matrix if any job in the matrix fails. This property defaults to true.

Assuming that we built a Python package and set many Python versions. We can ensure the package undergoes testing across defined Python versions.

Jobs contain a sequence of tasks called steps. Steps can run commands, set up docker, or run an action in our repository. These steps run one after the other based on their position in the YAML definition.

The following are the actions taken by each step in our workflow:

Checkout the repository on the machine defined for the job,
Setup Python using the current version from the strategy,
Install pip and the project dependencies,
Move into the project directory and run the tests.

We need to commit and push a new workflow to trigger our workflow.



git add .
git commit -m "Added tests automation using Github Actions"
git push

We should see a new running or completed workflow run on the GitHub actions tab in our forked repository.

This workflow would show us each job and the steps executed.

![GitHub workflow job management tab(https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5tlk6am6n0dhwjn7s2mn.png)

Now, any time we push a new commit to the master branch. Our workflow executes our Django application tests.

Understanding GitHub Actions is a complete guide to learning actions, workflows, and jobs. Check it out to master project development automation.

Conclusion

In this article, we have covered how GitHub actions execute Django application tests. Continuous Integration and Delivery is an important part of a modern development cycle. Automating development life cycles ensures we build reliable applications when working with collaborators.

Resources

Understanding GitHub Actions by GitHub
Introduction to GitHub Actions Training by Microsoft
GitHub Actions Tutorial for Beginners to Automate Your Workflow

Dynamically Managing Dependencies In Your Python Projects

Israel Ayanwola — Wed, 03 May 2023 08:28:43 +0000

In today's software development landscape, ensuring successful project deployment can be challenging due to the variety of environments programmers use to build their projects. Each environment presents unique challenges to project deployment from Windows to ArchLinux, Mac to Ubuntu.

One major challenge developers face is conflicting environment dependencies, which can cause project failure when deployed. Although Docker solves this problem, its complex setup requirements make it inaccessible to many developers.

This tutorial offers a solution by demonstrating how to dynamically install Python dependencies based on the environment in which the project will run.

Prerequisites

With only a basic understanding of Python and Git, any developer can follow the methods outlined in this tutorial to improve their project's deployment success.
The only requirement to have installed for this tutorial is Python. To install Python, visit their official downloads page.

What is the problem?

When using Postgres database with Django, psycopg2 database adapter is a required dependency for this to work. But there is a problem, psycopg2 will install successfully on a Windows environment but won't on a Linux environment, you will need to install psycopg2-binary instead.

Well, if your development and deployment environment is similar or you use Docker, you won't have a problem with this situation. The problem occurs when you have differing environments or multiple collaborators working on the project with you. You don't want them editing the requirements.txt file so as not to cause git conflicts.

How can this problem be solved?

Two major methods will be shared today to help you dynamically install Python dependencies based on your environment or Python version.

Conditional pip requirements syntax

Suppose that the dependencies listed in your requirements.txt file are as follows:

requests==2.26.0
six==1.16.0
urllib3==1.26.7
psycopg2==2.9.6

That can be modified to ensure that when you install the requirements on a Linux environment, the required database adapter for Linux will be installed.

requests==2.26.0
six==1.16.0
urllib3==1.26.7
psycopg2-binary==2.9.3; sys_platform == "linux"
psycopg2==2.9.6; sys_platform == "win32"

You can also program the dependencies to be installed based on the Python version installed on the environment.

requests==2.26.0; python_version > '3.0'
requests==2.26.0; python_version == '2.6'
six==1.16.0
urllib3==1.26.7
psycopg2-binary==2.9.3; sys_platform == "linux"
psycopg2==2.9.6; sys_platform == "win32"

They are defined in PEP 508 and PEP 0345 (Environment Markers), but the syntax appears to follow the draft PEP 0496.

To know all the possible values for sys_platform, visit this StackOverflow answer.

Now when you run pip install -r requirements.txt, depending on your environment, the appropriate dependency and version will be installed.

This is the easiest and recommended method for dynamically installing dependencies based on the environment or Python version.

Python install script

You could create an install.py script to install dependencies based on the environment or Python version.

Create a file install.py in your project's root directory

import pip

_all_ = (
    "requests==2.26.0",
    "six==1.16.0",
    "urllib3==1.26.7",
)  # All platforms

windows = ("psycopg2==2.9.6",)  # Windows

linux = ("psycopg2-binary==2.9.3",)  # Linux

darwin = []  # MacOS

def install(packages):
    print("Installing packages: {}".format(packages))
    for package in packages:
        pip.main(['install', package])

if __name__ == '__main__':

    from sys import platform

    print("Platform: {}".format(platform))

    install(_all_)

    if platform == 'win32':
        install(windows)
    if platform.startswith('linux'):
        install(linux)
    if platform == 'darwin':
        install(darwin)

    print("Completed installation of packages.")

This script works by checking what platform the program is running on using the Python built-in sys package. So it installs the defined dependencies for that platform by calling pip by script. This method is more complex and time-consuming, but it gives you more control and flexibility on the program control flow.

Run the program to install the packages
```
python install.py
```

Conclusion

In conclusion, conflicting environment dependencies can be a huge headache for software developers when deploying their projects. Fortunately, several solutions are available, including Docker and the dynamic installation of dependencies based on the environment. In this article, we have provided two different methods for dynamically installing Python dependencies based on the environment, which can help avoid conflicts and ensure the project runs smoothly across different platforms. These methods allow developers to save time, reduce errors, and focus on building great software. I hope that this article has been helpful in providing useful tips and tricks for managing environment dependencies in your Python projects.

Thanks for reading, I'd also love to connect with you at Twitter | LinkedIn | GitHub | Portfolio. 🤓

Building a Tech Course Curriculums Platform with MindsDB Models and Django

Israel Ayanwola — Sun, 30 Apr 2023 22:59:14 +0000

Introduction: As a developer, I often find myself overwhelmed by the abundance of free and paid online tech courses and resources. That's why I decided to build a platform that curates the best tech course curriculums and organizes them based on skill level, topic, and user feedback. To improve the user experience and accelerate app growth, I integrated MindsDB models for sentiment and text classification into the platform. In this tutorial, I'll walk you through how I used MindsDB models and Django for the backend and Next.js for the frontend to build this platform.

Background

I built this project for a hackathon with the goal of helping new programmers find quality tech courses and resources. The platform curates the best resources and organizes them into curriculums based on user feedback. I wanted to improve the user experience and accelerate app growth by implementing MindsDB models for sentiment and text classification.

How I used MindsDB Models

I used MindsDB models for sentiment and text classification to classify user curriculum reviews. The sentiment model determines if the review is positive, negative, or neutral, while the text classification model determines the topic or subject of the review, such as the quality of the resources, the user experience, or the difficulty level of the curriculum. The models are trained on a dataset of user reviews and are able to accurately classify new reviews with high accuracy.

To use MindsDB, I used some AI models from the Huggingface repository. I was able to utilize the models by using some SQL queries. I labeled the reviews based on their sentiment and topic and then used MindsDB to train the sentiment and text classification models on this labeled dataset. Once the models were trained, I integrated them into the backend API, which allows me to classify new user reviews in real-time.

Backend and Frontend Technologies

I used Django for the backend of my platform. Django is a powerful and popular Python-based web framework that allowed me to rapidly build a scalable and secure web application. I used Django to handle user authentication, curriculum enrollment, and progress tracking. I also added quizzes to the curriculums, which allow users to test their knowledge and track their progress.

For the frontend, I used Next.js, a React-based framework that allowed me to build high-performance and SEO-friendly web applications. I used Next.js to create a responsive and intuitive user interface that makes it easy for users to discover and enroll in the curated curriculums. I also used Next.js to integrate the MindsDB models into the platform, allowing me to classify user reviews in real-time.

Building The Project

I will cover the project build-up from the DB design to the product mvp completion. The repository for the completed project is listed below;

Database Design

By using diagram software, I designed a database diagram that I can follow.

This helped me get clarity on how I am going to structure the complex project. I could get tables that are related to each other and how to categorize the apps in Django. It saved me a lot of time instead of just jumping on the project without direction.

Application Design

I designed the frontend application using NextJs, Redux, Formik, and Chakra. I mustn't try to rebuild the wheel for form validation or basic components. Chakra helped to move faster with design, I won't say the app design is perfect but it was amazing to build.

I used redux to manage some of my states when working on the curriculum learning navigation. It was needed to send state changes to a parent component. I used redux for this purpose made it very easy to update the navigation based on the current tab being viewed by the learner.

Impact

By using MindsDB models, I'm able to quickly and accurately classify user curriculum reviews, allowing me to identify and address areas for improvement. Using the MindsDb cloud made it easier to integrate AI into the application. Doing this the manual way is not easy to manage.

This helps me to continually improve the platform and curate the best possible curriculums for users. Additionally, by using Django and Next.js, I'm able to build a scalable and secure platform that can handle a large number of users and provide a seamless user experience.

Conclusion

In this tutorial, I showed you how to use MindsDB models and Django for the backend and Next.js for the frontend to build a tech course curriculums platform. By implementing MindsDB models for sentiment and text classification, you can quickly and accurately classify user curriculum reviews and improve the user experience of your platform. With the power of Django and Next.js, you can build a scalable and secure platform that provides a seamless user experience.

Create and Manage Cryptographically Strong Tokens with Python for Better Web Application Security

Israel Ayanwola — Fri, 14 Apr 2023 20:48:01 +0000

Secret key management is a crucial aspect of a software engineer's daily routine. In this article, you will explore how to create and manage cryptographically secure keys in Python using the popular Pypi cryptography module. You will also learn the importance of securely storing these keys to ensure maximum protection of sensitive data in your web applications and software.

Prerequisites

You must install Python on your machine if you don't have it installed. Its the only needed external requirement. You can install Python from their official downloads page. Its recommended to install the latest version of Python for this tutorial.

To follow up with this tutorial, you are required to have some knowledge about writing simple Python programs. All the packages that will be discussed are from the Python standard library or Pypi package manager.

Secured Symmetrical Encryption Keys

When it comes to encryption and decryption in Python, the Pypi cryptography module is a popular choice. One of the easiest and most secure ways to implement symmetric cryptography using this module is through Fernet symmetrical encryption. However, to use Fernet keys for symmetric encryption, they must be generated and stored securely.

Let's dive in.

Start by creating a new Python project and set up your virtual environment to separate your project dependencies from your global dependencies.

python -m venv venv
# or
python3 -m venv venv

The Python command above is possible with Python3. If you have any errors setting this up, check out Python's official venv documentation to learn about creating a virtual environment.

Activate the created environment

# Windows: cmd.exe
venv\Script\activate

# Linux: bash or Mac: zsh
source venv/bin/activate

Depending on your shell, the command for activating your virtual environment will be different. Check this reference for the command that will work for your shell.

Now you need to install cryptography

pip install cryptography

If you encounter any installation issues, you can check out its installation guide.

Enter the Python shell and run the following commands to generate the key.

>>> from cryptography.fernet import Fernet
>>> key = Fernet.generate_key()
b'RAuFlYBGswBmBOccV13UNYXxJTi19LCGhUOLZOi6oFY='

The value RAuFlYBGswBmBOccV13UNYXxJTi19LCGhUOLZOi6oFY= in the byte string is the important key. It's stored like that because that's what the cryptography library likes to work with. So, if you generate a new key in the future, you need to convert it to bytes before passing it to any hash function.

Is Fernet.generate_key() cryptographically secure?

The answer is a resounding YES!

The implementation of Fernet.generate_key method uses the function os.urandom, which has more sufficient randomness than the random function from the built-in random module.

Why? It's because os.urandom cannot be seeded like random and draws its source of entropy from many unpredictable sources, making it more random. For more understanding, you can check this StackOverflow answer.

Now you can use your securely generated key to encrypt and decrypt data.

# Initialize a Fernet object
>>> f = Fernet(key)

# Generate a cryptographically secure and url-safe fernet token
>>> token = f.encrypt(b"Hashnode Neptune is the best")
>>> print(token)
b'gAAAAABkDv-R51WpztocZoMMat3UyKg8jz6KgCQgCq4g9SU36OF7kiPhqQwjLXPT-39lbb5cL-MlUWSmoDLKXlkOZo2Od_Icp_6jPFLDgF32f2r9agrRr50='

There you have it. You have generated a secure token that you can use to ensure your data security in your projects.

If you don't want to use the generate_key function to create your Fernet tokens. You can use this resource to generate a more secure and strong token using the PBKDF2HMAC algorithm.

Securely Managing Your Secret Keys

To ensure complete security, you must store your keys safely. Never share with anyone.

One easy way to do this is to use Pypi decouple.

Let's start by installing it in our environment.

pip install python-decouple

Create a file .env in your project directory. Add the key generated earlier to the file.

SECRET_KEY="RAuFlYBGswBmBOccV13UNYXxJTi19LCGhUOLZOi6oFY="

Save the file.

Create a new Python program, main.py, in your project directory. Then paste the code added below into the file.

from decouple import config

SECRET_KEY = config('SECRET_KEY')

print(SECRET_KEY)

Run the main.py python program.

python main.py

The secret key stored in the environment file would be displayed. Decouple is a project used to store the environment and secret keys. You can read more at Pypi Decouple.

What makes this approach better than using an operating system environment? Because it makes it easier to share secrets when collaborating with others. They only need to clone the project and copy the .env file.

Also, it eliminates the collision of similar keys on multiple projects on the same machine. As a bonus, you can create a file env.example, this file will contain the keys from the actual .env file but with fake values.

Your collaborator can then;

Create their .env file,
Copy the contents of the env.example file into their .env file,
Generate a secret key,
Replace the sample (fake) key in the .env file with the newly generated key.

Ensure you don't forget to add .env file to .gitignore so it doesn't get pushed to your GitHub repository, thereby exposing your keys. It's better to have a sample file like env.example which can be pushed to keep track of what keys a collaborator needs to create when running the project on their machine.

Using Python Built-in Secrets Library

From Python Documentation: The secrets module is used for generating cryptographically strong random numbers suitable for managing data such as passwords, account authentication, security tokens, and related secrets.

Use the Secrets library for security purposes to generate something random, like random tokens, digits, or strings. Its randomness is cryptographically secure.

Creating a secure hex token

Run your Python shell.

>>> import secrets
>>> secrets.token_hex()
'20a3f7333abd0668e474d393870a0b47463a6935e2eb730343820767eaf77226'

That's a simple way to create a very secure token. The longer the token the more sufficient the randomness.

The function token_hex accepts a parameter, n_bytes, for how long you want the token.

>>> import secrets
>>> secrets.token_hex(64
'5222cffc8c2881afcbf219c90c6e2f3a8b168c7547f790c2e852a047e9f4c8094577d8198a1d802ee053d13987e9111b317a7771c9ce87597a67203311afc69f')

Conclusion

Securing web applications and software requires effective management of secret keys. Python offers libraries like cryptography and secrets that simplify the process of generating cryptographically secure tokens. To ensure maximum protection of sensitive data, it is crucial to store these keys securely using tools like Pypi Decouple and AWS Secrets Manager. By adhering to best practices for managing secret keys, developers can guarantee the security of their projects and prevent unauthorized access to sensitive information.

Follow me on Twitter @netrobeweb, Hashnode, and Dev.to where I post amazing projects and articles.

Thanks for reading, 🤓😊

How to Build a Speech Authentication System with Django and Next JS - Part 1

Israel Ayanwola — Fri, 07 Apr 2023 21:02:06 +0000

Learn how to build a speech authentication system with Django in this comprehensive tutorial. With the help of a Python package that converts speech to text, users can enter their details by speaking. The system includes a registration and login page and once logged in, users will be greeted with a welcome voice. Follow the step-by-step guide to set up the backend API using Django and the frontend using Next.js and Chakra UI.

TL;DR

The system includes a registration and login page and once logged in, users will be greeted with a welcome voice. Follow the step-by-step guide to set up the backend API using Django and the frontend using Next.js and Chakra UI. The article also covers the prerequisites, project setup, installations, and building of the API. The API includes endpoints for Registration, Login, JWT Token Refresh, User Retrieve, and Speech to Text.

Prerequisites

Hey there! To make the most of this article, it would be helpful to have a solid understanding of Django, Django Rest Framework, and Next.js. Don't worry though, we'll do our best to explain things clearly and make it easy to follow along.

Getting Started

First of all, we would work on the backend. Django will be used for the Backend and Next.js + Charkra UI for the front end. Django is used because we would be using a Python package that helps us convert speech to text.

Application Design

The overall application design will be very simple. The backend will be an API which the frontend will consume.

A user's voice will be recorded on the frontend.
The frontend sends the recorded wav file to the backend in Base64 format.
The backend will process the data and convert it to text.
The text is returned to the frontend as a response,
then text is entered into the UI input box.

That's it, very simple.

Project Setup

This project is already available on GitHub. Clone it and follow the installation guide in the Readme to set up the project.

Installations

For building this project, we will need to have the following installed;

Python 3.9 - This is used for the backend, and Django is used as the framework.
NodeJs & NPM - This is used for the frontend.

Building the API

Let's clone the project.

git clone https://github.com/devvspaces/speech_to_text_auth
cd speech_to_text_auth

For now, we would be working on the API. Move into the api directory.

Set up your virtual environment

python -m venv venv
# Or python3 if you have that installed
python3 -m venv venv

Check out this python documentation to learn about virtual environments.

Activate the environment.

# windows: cmd.exe
venv\Scripts\activate

# bash or zsh
source venv/bin/activate

If the above commands don't work, check out this python documentation for help activating virtual environments. If you encounter an issue activating your virtual environment, here is a list of commands for common shells.

Installing requirements

# This will install the requirements in the requirements.txt file
pip install -r requirements.txt

Move into the src directory. There we have the account application, config, tests, and utils directories.

Now we need to create our .env file. This is where our secret configurations are kept. Decouple is used for accessing this file in our config files e.g manage.py, wsgi.py, etc.

Create a new file, .env and copy the contents of .env.example into it.

Now we need to create a Postgres DB for our project. By default, the project uses PostgreSQL but can be changed in the settings file. We can also create a DB for tests if we want to run tests. It's customary to have a separate database for development and testing because well testing uses autogenerated data which we don't want in our development database.

The config folder is where our settings are located. This directory contains a settings directory which contains two Python files, base.py and test.py. Having our settings file arranged like this helps us to write separate settings for different environments.

We could

develop using the base.py,
test using the test.py,
maybe create a file staging.py for our staging environment in a live server
and production.py for our production environment.

The tests folder contains our tests.

Accounts Application

This is the only application on the project and the most important because we will store users' data here. The API views, models, and serializers are designed here.

models.py

This is a basic Django custom user model implementation. By inheriting the Django AbstractBaseUser model, we can build our User model and make email required.

This is the DBML structure used for the diagram above.

// Use DBML to define your database structure
// Docs: https://www.dbml.org/docs
// DB Diagram tool: https://dbdiagram.io/home

Table users {
  pk integer [primary key]
  email varchar
  active boolean
  staff boolean
  admin boolean
  created timestamp
}

Table profile {
  pk integer [primary key]
  user_id integer
  fullname varchar
  sex varchar
  phone varchar
  country varchar
}

Ref: profile.user_id - users.pk // one-to-one

The User model has a one-to-one relationship with the Profile model. This is a very common way to design a User table. The reason for this is to make sure that when there is a change in business logic that will slightly affect how the User is created. We will update the Profile model instead so that we won't have migration issues with the User model.

When a user is created, we automatically create its profile by listening to the post_save signal. Learn more about Django Signals.

forms.py

We need a custom user registration form because we need a way to make the Django admin create a User correctly. This form allows the admin to enter the user's profile data when creating the user. The default Django admin form will have only fields for the User model.

By overriding the model form save method, we get the validated profile data and populate the already created user profile. As soon as the user.save() is invoked, the profile will be created by the post_save signal from earlier.

Learn more about creating Django Model Form.

admin.py

One important thing here is, we create a UserAdmin class for overriding the default that will be created by Django. Read more about Customizing Django Model Admin.

Django BaseUserAdmin is a special ModelAdmin that allows us to override the form used for creating a new User instance in the admin. So we pass the form we created earlier here.

config/settings/base.py

An important thing to note here is that after creating a Custom Django User Model we need to tell Django we want to use it instead of the default.

The line below tells Django to use our new Custom User Model instead of its default User Model. This is a very common implementation in Django, here is a Django Rest Framework GitHub template that we can use as a project boilerplate.

AUTH_USER_MODEL = 'account.User'

Decouple is used to access your environment keys, .env. This documentation shares how to effectively use decouple with Django.

Views and Serializers

Now we will cover the Django rest framework views and serializers. We need the following endpoints;

Registration [POST],
Login [POST],
JWT Token Refresh [POST],
User Retrieve [GET], and
Speech to Text [POST].

Registration [POST]: The endpoint will be used to create a new user. The data that will be received through all the endpoints will be in JSON format. The endpoint uses the imported RegisterSerializer to convert post data into native Python datatypes and validate them. An important thing to note here is that we are using JSON Web Token (JWT) for authenticating our users. As soon as a new account is created, new access and refresh tokens are included in the response. Read more about Simple JWT, A JSON Web Token authentication plugin for the Django REST Framework.

Login [POST]: Validates the user data and returns a response that contains the tokens and logged-in user data.

JWT Token Refresh [POST]: This is used to get a new access token using a valid refresh token. As JWT is a stateless authentication method, meaning there is no session stored for the user. Every 5 mins the access token previously generated is expired which requires the user to generate a new access token using the refresh token which is valid for 24 hours. The validation time for both the access and refresh tokens can be edited from the settings.py file. Learn more about this at Simple JWT.

User Retrieve [GET]: We need to be able to get the data of the currently authenticated user. This endpoint returns the data of the authenticated user.

Speech to Text [POST]: This endpoint used the SpeechRecognition package, to convert audio binary to text. This package supports Google, IBM, and Microsoft Speech-to-text APIs and many more. Here we are using Google Speech-to-text. Because of the abstraction, the package brings, we don't need to call any actual API to upload our audio files for processing.

class SpeechToTextView(APIView):

    @swagger_auto_schema(
        request_body=serializers.SpeechBody,
    )
    def post(self, request, *args, **kwargs):
        """
        Convert speech to text, using Google Speech Recognition API
        Accepts a base64 encoded string of audio data
        Returns:
            Response: Response object with text
        """
        try:
            record: str = self.request.data.get('record')
            decoded_b64 = base64.b64decode(record)

            # Convert decoded_b64 to in-memory bytes buffer
            r = sr.Recognizer()
            with sr.AudioFile(io.BytesIO(decoded_b64)) as source:
                # listen for the data (load audio to memory)
                audio_data = r.record(source)
                # recognize (convert from speech to text)
                text = r.recognize_google(audio_data)

            return Response(data={'text': text})
        except Exception:
            return Response(
                data={'message': "Error converting speech to text"},
                status=status.HTTP_400_BAD_REQUEST)

The data that will be passed to this endpoint will be in base64 format because it's faster and easier to transfer and process in this form.

decoded_b64 = base64.b64decode(record)

This takes the data received and decodes it using the built-in base64 library. The decoded data returned is in binary form. This is done so that we can convert it into a Buffered stream in memory using io.BytesIO(decoded_b64) , the built-in io library which is used for working with Input and Output (I/O) streams.

What are Buffered I/O streams in Python?

Buffered I/O streams in Python's io module are classes that help improve the performance of I/O operations by buffering data in memory. Think of buffering as a way of temporarily storing data in a queue so that it can be accessed more efficiently later on. Read more on Buffered streams to deeply understand how it works.

When working with objects like Buffers, Files, or Database connections in Python Context Managers are used to gain access to these objects because the connection is closed automatically even if there is an unhandled exception in the context manager.

with sr.AudioFile(io.BytesIO(decoded_b64)) as source:
    # listen for the data (load audio to memory)
    audio_data = r.record(source)
    # recognize (convert from speech to text)
    text = r.recognize_google(audio_data)

If an error occurs while trying to record the audio source, which is the Buffer we created earlier in the memory, the I/O connection will be closed safely. This article explains why it's important to use Context Managers when dealing with objects like this.

So the buffer is recorded and passed to google for speech-to-text processing. If all is well, we return the text in the response else we raise a BadRequest error.

Conclusion 🤓

In conclusion, this article provides a comprehensive tutorial on how to build a speech authentication system with Django. The system allows users to enter their details by speaking and includes a registration and login page. The article covers the step-by-step guide to setting up the backend API using Django. The article also covers the necessary installations, project setup, and the design of the application. Overall, this article is a great resource for developers looking to build a speech authentication system.

To make this article brief the next part which is Building the Frontend with Next.js will be covered in another article.

You can follow me on Twitter, Hashnode, Dev.to, and Github, where I post amazing projects and articles.

Thanks for reading, 😉.

Streamline Your File Uploads with Cloudinary and NestJS

Israel Ayanwola — Thu, 23 Mar 2023 10:06:02 +0000

Cloudinary is a powerful serverless cloud-based storage infrastructure that allows easy file uploading for NestJS projects. In this article, we will explore the various methods of file uploading to Cloudinary and how to set it up in your NestJS project. Whether you are uploading file buffers, file URLs, or Base64 encoded strings, the Cloudinary NodeJS SDK and the steps outlined in this article make it easy to incorporate advanced file uploading into your NestJS project.

If you don't know how to handle file upload in NestJs, this article explains how to efficiently incorporate file upload and validation into your NestJS project.

Creating a Cloudinary Module

One way to easily set it up in your NestJs projects is by creating a module for it. This allows you to use Dependency Injection design to use it in other providers or controllers.

Installation

Install Cloudinary NodeJs SDK

npm install cloudinary

Create a new module, cloudinary.

Create file constants.ts.

export const CLOUDINARY = 'Cloudinary';

Add file cloudinary.provider.ts, paste the code below into the file.


import { ConfigService } from '@nestjs/config';
import { v2 } from 'cloudinary';
import { CLOUDINARY } from './constants';

export const CloudinaryProvider = {
  provide: CLOUDINARY,
  useFactory: (config: ConfigService) => {
    return v2.config({
      cloud_name: config.get('CLOUDINARY_CLOUD_NAME'),
      api_key: config.get('CLOUDINARY_API_KEY'),
      api_secret: config.get('CLOUDINARY_API_SECRET'),
    });
  },
  inject: [ConfigService],
};

ConfigService is used to assess keys in your secret file. For example, a .env file.

Don't know how to set up env configurations in NestJs? Read more.

Don't know how to create factory providers in NestJs? Read more.

The CloudinaryProvider provider is a Factory Provider. The useFactory allows you to create providers dynamically. You can also inject other module-imported services into the useFactory function.

Create the Cloudinary Service Injectable

Add a new file, cloudinary.service.ts

import { Injectable } from '@nestjs/common';

@Injectable()
export class CloudinaryService {   
}

This service will contain methods for uploading files to Cloudinary.

Create the module file

Add a new file, cloudinary.module.ts

import { Module } from '@nestjs/common';
import { ConfigModule } from '@nestjs/config';
import { CloudinaryProvider } from './cloudinary.provider';
import { CloudinaryService } from './cloudinary.service';

@Module({
    imports: [ConfigModule],
    providers: [CloudinaryProvider, CloudinaryService],
    exports: [CloudinaryProvider, CloudinaryService],
})
export class CloudinaryModule {}

The ConfigModule is added to the module imports because ConfigService in the CloudinaryProvider.

Setup .env file

Create a .env in your root folder. Depending on how you structure your project, a general root folder could be the folder containing your package.json file or where you run the command to start your server.

CLOUDINARY_CLOUD_NAME=test
CLOUDINARY_API_KEY=test
CLOUDINARY_API_SECRET=test

Now you need to create a free Cloudinary account. After creating your account, you can follow this guide to get your API key, secret and cloud name.

Setup ConfigModule

In your app.module.ts you will need to configure the ConfigModule. This makes sure it finds the .env file and populate itself with the keys and values, thereby making you access them with ConfigService.

import {
  Module,
  NestModule,
} from '@nestjs/common';
import { AppController } from './app.controller';
import { AppService } from './app.service';
import { ConfigModule } from '@nestjs/config';

@Module({
  imports: [
    ConfigModule.forRoot(),
  ],
  controllers: [AppController],
  providers: [
    AppService,
  ],
})
export class AppModule implements NestModule {
}

Using Form data - File Buffers

Single File Upload

Now that you have your CloudinaryService. When you create a route that accepts file buffers using Multer. You can upload this file using the service. You just have to integrate the upload method from the Cloudinary package.

Add the method below to your service.

async uploadFile(
  file: Express.Multer.File,
): Promise<UploadApiResponse | UploadApiErrorResponse> {
  return new Promise((resolve, reject) => {
    v2.uploader.upload_stream(
      {
        resource_type: 'auto',
      },
      (error, result) => {
        if (error) return reject(error);
        resolve(result);
      }
    ).end(file.buffer)
  })
}

This simple method uses v2.uploader.upload_stream to upload your file to Cloudinary in chunks, which is good for uploading large files in chunks instead of one long request.

An UploadApiResponse or UploadApiErrorResponse is returned by the function. You can use this to know if the file was successfully uploaded or not.

To access your uploaded file URL, use the example of a route for single file upload below to understand the process.

import { UploadedFile, UseInterceptors } from '@nestjs/common';
import { FileInterceptor } from '@nestjs/platform-express';

@Post('upload/single')
@UseInterceptors(FileInterceptor('file'))
@ApiOkResponse({ description: 'Upload image', type: String })
public async uploadSingleImage(@UploadedFile() file: Express.Multer.File) {
  const result = await this.cloudinaryService.uploadFile(file);
  return result.secure_url;
}

The secure_url property is the URL for the file uploaded.

Uploading multiple file buffers

Loop through all the files and upload each one after the other. The response is the list of secure URLs.

async uploadFiles(
  files: Express.Multer.File[],
) {
  const urls = await Promise.all(files.map(async (file): Promise<string> => {
    const { secure_url } = await this.uploadFile(file);
    return secure_url;
  }));
  return urls
}

The files were mapped over and uploaded, and their URLs were returned by the map callback function.

Uploading File with URLs

What if you have a file URL and not a file buffer? It's simpler to upload that using Cloudinary.

async uploadFileFromUrl(
  url: string,
): Promise<UploadApiResponse | UploadApiErrorResponse> {
  return v2.uploader.upload(url)
}

The URL passed to this would be uploaded to Cloudinary, which will then return a response containing the secure_url if successful.

Uploading multiple file urls

By following the same idea from uploading multiple file buffers. You can map through the urls, upload each and return the secure_url for each. Then the function will return a list of URLs.

async uploadFilesFromUrl(
  urls: string[],
) {
  return Promise.all(urls.map(async (url: string): Promise<string> => {
    const { secure_url } = await this.uploadFileFromUrl(url);
    return secure_url;
  }));
}

Uploading Base64 encoded strings

Base64 is a binary-to-text encoding scheme that represents binary data in an American Standard Code for Information Interchange (ASCII) string format. An example of binary data is an image. Base64 can be uploaded too as a file, it's a string and not a buffer, which makes it easier to parse and upload to Cloudinary than file buffers.

By utilizing the upload function from the Cloudinary package, a base64 string can be uploaded to Cloudinary and get a URL for the uploaded file.

async uploadFileFromBase64(
  data: string,
): Promise<UploadApiResponse | UploadApiErrorResponse> {
  return v2.uploader.upload(data)
}

async uploadManyBase64(
  files: string[],
) {
  const urls = await Promise.all(files.map(async (file): Promise<string> => {
    const { secure_url } = await this.uploadFileFromBase64(file);
    return secure_url;
  }));
  return urls
}

Conclusion

In conclusion, Cloudinary is a powerful tool for file uploading and storage in NestJS projects. With the help of the Cloudinary NodeJS SDK and the steps outlined in this article, developers can easily set up a module for Cloudinary and upload files using various available methods including file buffers, file URLs, and Base64 encoded strings. While Cloudinary is free to set up initially, developers may need to pay for used services once they start using it past some limit.

I'd love to connect with you on Twitter | LinkedIn | GitHub | Portfolio

See you in my next blog article. Take care!!!

Creating Admin Panels for NestJs and Prisma Made Easy with AdminJS

Israel Ayanwola — Tue, 21 Mar 2023 12:52:51 +0000

Are you tired of spending too much time creating admin panels for your NestJs and Prisma applications? Look no further than AdminJS! This powerful framework allows you to create a fully functional admin dashboard with just a few lines of code. With its compatibility with various ORM and ODM models, it simplifies the process of performing CRUD operations on your models.

Plus, with customization options available, you can tailor the dashboard to meet your specific project needs. Keep reading to learn how to get started with AdminJS and take control of your application's data management.

What is ORM

Object-relational mapping (ORM, O/RM, and O/R mapping tool) in computer science is a programming technique for converting data between incompatible type systems using object-oriented programming languages.

What is ODM?

Object Document Mapper (ODM) is a library that helps you to work with MongoDB databases. It provides an abstraction layer between the database and the application code. It allows you to use the object-oriented paradigm to work with the database.

Getting Started

AdminJs/NestJs => AdminJs/Prisma => Resources and Customizations

Above is a very simple flow that describes the process of setting up AdminJs on your NestJs project.

Install Requirements



yarn add adminjs @adminjs/nestjs @adminjs/express express-session express-formidable

app.module.ts



import { Module } from '@nestjs/common'
import { AdminModule } from '@adminjs/nestjs'

import { AppController } from './app.controller'
import { AppService } from './app.service'

const DEFAULT_ADMIN = {
  email: 'admin@example.com',
  password: 'password',
}

const authenticate = async (email: string, password: string) => {
  if (email === DEFAULT_ADMIN.email && password === DEFAULT_ADMIN.password) {
    return Promise.resolve(DEFAULT_ADMIN)
  }
  return null
}

@Module({
  imports: [
    AdminModule.createAdminAsync({
      useFactory: () => ({
        adminJsOptions: {
          rootPath: '/admin',
          resources: [],
        },
        auth: {
          authenticate,
          cookieName: 'adminjs',
          cookiePassword: 'secret'
        },
        sessionOptions: {
          resave: true,
          saveUninitialized: true,
          secret: 'secret'
        },
      }),
    }),
  ],
  controllers: [AppController],
  providers: [AppService],
})
export class AppModule {}

Don't want authentication?

You might want your admin panel to not require authentication depending on the project you are working on. By removing the auth and sessionOptions from the object returned by the factory provider.



...
AdminModule.createAdminAsync({
  useFactory: () => ({
    adminJsOptions: {
      rootPath: '/admin',
      resources: [],
    },
  }),
}),
...

Start your server



nest start --watch
# OR
nest start

Visit 127.0.0.1:<your port>/admin . If you have authentication enabled it will automatically redirect you to the login page.

Enter the details from the DEFAULT_ADMIN object in the app.module.ts program.

Adding Database models

Upon logging in, if you're unable to locate your models on the dashboard, resources can be of great help. AdminJs leverages the resources added to its options to effectively showcase, evaluate, and utilize them.

Assuming you have this Prisma schema,



generator client {
  provider = "prisma-client-js"
}

datasource db {
  provider = "postgresql"
  url = env("DATABASE_URL")
}

model User {
  pk Int @id @default(autoincrement())
  id String @unique @default(uuid()) @db.Uuid
  email String @unique
  password String
  active Boolean @default(true)
  staff Boolean @default(false)
  admin Boolean @default(false)
  profile Profile?

  @@unique([pk, id])
  @@map("users")
}

model Profile {
  pk Int @id @default(autoincrement())
  id String @unique @default(uuid()) @db.Uuid
  firstName String? @map("first_name")
  lastName String? @map("last_name")
  userPk Int @unique @map("user_pk")
  userId String @unique @map("user_id") @db.Uuid
  user User @relation(fields: [userPk, userId], references: [pk, id], onDelete: Cascade, onUpdate: Cascade)

  @@unique([pk, id])
  @@unique([userPk, userId])
  @@map("profiles")
}

app.module.ts



import { Module } from '@nestjs/common'
import { AdminModule } from '@adminjs/nestjs'
import { Database, Resource } from '@adminjs/prisma';
import AdminJS from 'adminjs'
import { DMMFClass } from '@prisma/client/runtime';
import { PrismaService } from '@database/prisma.service';
import { DatabaseModule } from '@database/database.module';

import { AppController } from './app.controller'
import { AppService } from './app.service'

const DEFAULT_ADMIN = {
  email: 'admin@example.com',
  password: 'password',
}

const authenticate = async (email: string, password: string) => {
  if (email === DEFAULT_ADMIN.email && password === DEFAULT_ADMIN.password) {
    return Promise.resolve(DEFAULT_ADMIN)
  }
  return null
}

AdminJS.registerAdapter({ Database, Resource });

@Module({
  imports: [
    AdminModule.createAdminAsync({
      imports: [DatabaseModule],
      useFactory: (prisma: PrismaService) => {
        const dmmf = (prisma as any)._baseDmmf as DMMFClass;

        return {
          adminJsOptions: {
            rootPath: '/admin',
            resources: [
              {
                resource: { model: dmmf.modelMap['User'], client: prisma },
                options: {},
              },
              {
                resource: { model: dmmf.modelMap['Profile'], client: prisma },
                options: {},
              },
            ],
          },
          auth: {
            authenticate,
            cookieName: 'adminjs',
            cookiePassword: 'secret'
          },
          sessionOptions: {
            resave: true,
            saveUninitialized: true,
            secret: 'secret'
          },
        }
      },
      inject: [PrismaService],
    }),
  ],
  controllers: [AppController],
  providers: [AppService],
})
export class AppModule {}

Explore the basic Resource object format and effortlessly add multiple resources to your project.



{
  resource: { model: dmmf.modelMap[{Model name here}], client: prisma },
  options: {},
}

Make sure you replace {Model name here} with the name of the model you want to use. For example, if you have a model Post, you would replace {Model name here} with Post.

How does this work?

The Database Model Mapper (DMMF) object comprises a comprehensive depiction of your database schema. Its functionality extends to facilitating the extraction of your model from Prisma. The DMMF achieves this by mapping model names to their respective model objects. For example, to retrieve the User model from the map, one can utilize the syntax dmmf.modelMap['User']. Once the model object is obtained, it can be assigned to the resource's model property.

The Prisma client serves as the client property, facilitating connectivity to the database for conducting CRUD operations.

You can add as many resources as you need. For example, if you have a Post model in your Prisma schema that you want to be displayed in the Admin Dashboard. You just need to add it to the resources array;

app.module.ts



...
{
  resource: { model: dmmf.modelMap['Post'], client: prisma },
  options: {},
},
...

Now restart your server if it's not on fast reload.

Simplify adding resources

When managing a substantial number of database models, it is common to encounter the need for duplicating codes while incorporating new resources for each model.

Did you know that the Builder pattern can help you add multiple resources to your code without repeating the same code? This makes your code more efficient and easier to maintain. So why not give it a try?



import { Module } from '@nestjs/common'
import { AdminModule } from '@adminjs/nestjs'
import { Database, Resource } from '@adminjs/prisma';
import AdminJS from 'adminjs'
import { DMMFClass } from '@prisma/client/runtime';
import { PrismaService } from '@database/prisma.service';
import { DatabaseModule } from '@database/database.module';
import { ResourceWithOptions, ResourceOptions } from 'adminjs/types/src';

import { AppController } from './app.controller'
import { AppService } from './app.service'

const DEFAULT_ADMIN = {
  email: 'admin@example.com',
  password: 'password',
}

const authenticate = async (email: string, password: string) => {
  if (email === DEFAULT_ADMIN.email && password === DEFAULT_ADMIN.password) {
    return Promise.resolve(DEFAULT_ADMIN)
  }
  return null
}

AdminJS.registerAdapter({ Database, Resource });

class CResource {
  model: any;
  options: ResourceOptions;

  constructor(model: any, options?: ResourceOptions) {
    this.model = model;
    this.options = options || {};
  }
}

class CResourceBuilder {
  private readonly resources: Array<CResource> = [];
  dmmf: DMMFClass;

  constructor(private readonly prisma: PrismaService) {
    this.dmmf = ((prisma as any)._baseDmmf as DMMFClass)
  }

  /**
   * Adds a resource to the builder
   * 
   * @param resource string
   * @param options ResourceOptions
   * @returns this
   */
  public addResource(resource: string, options?: ResourceOptions): this {
    const obj = new CResource(this.dmmf.modelMap[resource], options);
    this.resources.push(obj);
    return this;
  }

  /**
   * Compiles the resources into an array of objects
   * that can be passed to the AdminJS module
   * 
   * @returns Array<ResourceWithOptions | any>
   */
  public build(): Array<ResourceWithOptions | any> {
    return this.resources.map((resource) => {
      return {
        resource: {
          model: resource.model,
          client: this.prisma,
        },
        options: resource.options,
      }
    })
  }
}

@Module({
  imports: [
    AdminModule.createAdminAsync({
      imports: [DatabaseModule],
      useFactory: (prisma: PrismaService) => {
        const dmmf = (prisma as any)._baseDmmf as DMMFClass;

        return {
          adminJsOptions: {
            rootPath: '/admin',

            // updated here
            resources: new CResourceBuilder(prisma)
              .addResource('User')
              .addResource('Profile')
              .addResource('Post')
              .build(),
          },
          auth: {
            authenticate,
            cookieName: 'adminjs',
            cookiePassword: 'secret'
          },
          sessionOptions: {
            resave: true,
            saveUninitialized: true,
            secret: 'secret'
          },
        }
      },
      inject: [PrismaService],
    }),
  ],
  controllers: [AppController],
  providers: [AppService],
})
export class AppModule { }

You can always export the classes, functions and objects from an external file.

Mastering AdminJs

Conclusion

In conclusion, AdminJS is a powerful tool for creating admin dashboards for NestJS and Prisma applications. With its adapters for various ORM and ODM models, it simplifies the process of performing CRUD operations on models. It also provides customization options for the dashboard, making it easy to tailor it to specific project needs. By following the steps outlined in the article, developers can quickly set up AdminJS and start using it to manage their application's data.

Follow me on Twitter @netrobeweb and Hashnode where I post amazing projects and articles.

Thanks for reading, 😉.

Mastering File Upload and Validation in NestJS with Multer

Israel Ayanwola — Sat, 18 Mar 2023 12:04:39 +0000

This guide will show you how to efficiently incorporate file upload and validation into your NestJS project. The tutorial will walk you through the steps of receiving and verifying files using the Multer middleware package with the Express Adapter. By following this guide, you can create a personalized file upload and validation process that works best for your NestJS application.

Accepting files

File processing in NestJs is handled with Multer. It has a built-in Multer middleware package for Express Adapter. The only required package to install is @types/multer, which is for better type safety.

npm i -D @types/multer @nestjs/platform-express

Start by creating an endpoint for single file uploads.

import { Post, UploadedFile, UseInterceptors, Controller } from '@nestjs/common';
import { FileInterceptor } from '@nestjs/platform-express';

@Controller('files')
export class FileController {
    @Post('upload')
    @UseInterceptors(FileInterceptor('file'))
    uploadFile(@UploadedFile() file: Express.Multer.File) {
        console.log(file);
    }
}

Make sure to use @nestjs/common: "^9.0.0", it's required to use UploadedFile.

If you use swagger, you can specify ApiConsumes decorator to multipart/form-data. This tells swagger that this route accepts form data.

import { Post, UploadedFile, UseInterceptors, Controller } from '@nestjs/common';
import { FileInterceptor } from '@nestjs/platform-express';
import { ApiConsumes, ApiTags } from '@nestjs/swagger';

@ApiTags('Uploding Files')
@Controller('files')
@ApiConsumes('multipart/form-data')
export class FileController {
    @Post('upload')
    @UseInterceptors(FileInterceptor('file'))
    uploadFile(@UploadedFile() file: Express.Multer.File) {
        console.log(file);
    }
}

When it comes to filing storage, Multer has a default memory setting. While this is a viable choice, it may not suit your needs if you plan to bypass disk storage and instead upload files directly to external storage solutions like Cloudinary or AWS S3.

How to Save Data to Disk Storage

To upload your files to disk storage, you can provide the file destination in the FileInterceptor. The interceptor accepts Multer Options.

const UPLOAD_DIR = './upload/files/';

@ApiTags('Uploding Files')
@Controller('files')
@ApiConsumes('multipart/form-data')
export class FileController {
    @Post('upload')
    @UseInterceptors(FileInterceptor('file', { dest: UPLOAD_DIR }))
    uploadFile(@UploadedFile() file: Express.Multer.File) {
        console.log(file);
    }
}

The upload directory path is relative to your root directory.

Hey there! When dealing with a lot of users, we want to make sure we avoid any issues that might come up with file names. To prevent this from happening, we recommend assigning a separate folder for each upload. That way, everything stays organized and everyone can easily find what they need.

There is a helpful tip to share with you. Are you aware that every user can have their unique folder in the upload directory? What's more, you can enhance this feature by configuring a storage engine for Multer, which allows you to create a dynamic upload path. Why not give it a try?

import { Post, UploadedFile, UseInterceptors, Controller } from '@nestjs/common';
import { FileInterceptor } from '@nestjs/platform-express';
import { ApiConsumes, ApiTags } from '@nestjs/swagger';
import { Request } from 'express';
import { diskStorage } from 'multer';
import * as path from 'path';

const UPLOAD_DIR = './upload/files/';

// User interface of the authenticated user
interface User {
  id: string;
}

/**
 * You can use this function to generate a unique filename for each file
 * User id is used to generate a unique filename
 * The User object can be attached to the request object in the auth middleware
 */
const defaultConfig = diskStorage({
  destination: UPLOAD_DIR,
  filename: (req: Request & { user: User }, file, cb) => {
    const uid = req.user.id;
    cb(null, `${uid}${path.extname(file.originalname)}`)
  }
})

@ApiTags('Uploding Files')
@Controller('files')
@ApiConsumes('multipart/form-data')
export class FileController {

  @Post('upload')
  @UseInterceptors(FileInterceptor('file', { storage: defaultConfig }))
  uploadFile(@UploadedFile() file: Express.Multer.File) {
    console.log(file);
  }

}

In this example, the user object could be attached to the request manually in the auth middleware.

This is a perfect example of how to accept files and save them in a super cool and unique path!

The file would be saved to disk before console.log(file) runs.

The next section provides a way of adding file processing like validation of files uploaded.

Validate Your Files with ParseFilePipe and ParseFilePipeBuilder

Using ParseFilePipe with Validator classes

NestJs Pipes offer an efficient way to validate files. To perform file validation, simply create a validator class such as MaxFileSizeValidator or FileNameValidator, and pass it to the ParseFilePipe.

import { ..., ParseFilePipe } from '@nestjs/common';

...

@Post('upload')
@UseInterceptors(FileInterceptor('file', { storage: defaultConfig }))
uploadFile(
  @UploadedFile(
    new ParseFilePipe({
      validators: [
        // ... Set of file validator instances here
      ]
    })
  )
  file: Express.Multer.File
) {
  console.log(file);
}

...

Create a FileValidator class to use with ParseFilePipe.

import { FileValidator } from '@nestjs/common';

class MaxFileSize extends FileValidator<{ maxSize: number }>{
  constructor(options: { maxSize: number }) {
    super(options)
  }

  isValid(file: Express.Multer.File): boolean | Promise<boolean> {
    const in_mb = file.size / 1000000
    return in_mb <= this.validationOptions.maxSize
  }
  buildErrorMessage(): string {
    return `File uploaded is too big. Max size is (${this.validationOptions.maxSize} MB)`
  }
}

The interfaceFileValidator has two methods needed to create its class. The method isValid and buildErrorMessage.

The method, isValid will contain your logic that defines if the file is valid or not depending on the options passed in the constructor.

From the constructor definition, maxSize can be passed as an option and is used in the isValid method to know what is considered the maximum file size.

/**
 * Builds an error message in case the validation fails.
 * @param file the file from the request object
 */
abstract buildErrorMessage(file: any): string;

Above is the interface of the buildErrorMessage function. It's used to build unique or general error messages for uploaded files.

Pass an instance of the MaxFileSize class in the ParseFilePipe. The maxSize can be passed to the constructor during instantiation.

...

const MAX_UPLOAD_SIZE = 10; // in MB
...

@Post('upload')
@UseInterceptors(FileInterceptor('file', { storage: defaultConfig }))
uploadFile(
  @UploadedFile(
    new ParseFilePipe({
      validators: [
        // ... Set of file validator instances here
        new MaxFileSize({
          maxSize: MAX_UPLOAD_SIZE
        }),
      ]
    })
  )
  file: Express.Multer.File
) {
  console.log(file);
}

...

The code should look like this now

import { Post, UploadedFile, UseInterceptors, Controller, ParseFilePipe } from '@nestjs/common';
import { FileInterceptor } from '@nestjs/platform-express';
import { ApiConsumes, ApiTags } from '@nestjs/swagger';
import { Request } from 'express';
import { diskStorage } from 'multer';
import * as path from 'path';
import { FileValidator } from '@nestjs/common';

const UPLOAD_DIR = './upload/files/';
const MAX_UPLOAD_SIZE = 10; // in MB

// User interface of the authenticated user
interface User {
  id: string;
}

class MaxFileSize extends FileValidator<{ maxSize: number }>{
  constructor(options: { maxSize: number }) {
    super(options)
  }

  isValid(file: Express.Multer.File): boolean | Promise<boolean> {
    const in_mb = file.size / 1000000
    return in_mb <= this.validationOptions.maxSize
  }
  buildErrorMessage(): string {
    return `File uploaded is too big. Max size is (${this.validationOptions.maxSize} MB)`
  }
}

/**
 * You can use this function to generate a unique filename for each file
 * User id is used to generate a unique filename
 * The User object can be attached to the request object in the auth middleware
 */
const defaultConfig = diskStorage({
  destination: UPLOAD_DIR,
  filename: (req: Request & { user: User }, file, cb) => {
    const uid = req.user.id;
    cb(null, `${uid}${path.extname(file.originalname)}`)
  }
})

@ApiTags('Uploding Files')
@Controller('files')
@ApiConsumes('multipart/form-data')
export class FileController {

  @Post('upload')
  @UseInterceptors(FileInterceptor('file', { storage: defaultConfig }))
  uploadFile(
    @UploadedFile(
      new ParseFilePipe({
        validators: [
          // ... Set of file validator instances here
          new MaxFileSize({
            maxSize: MAX_UPLOAD_SIZE
          }),
        ]
      })
    )
    file: Express.Multer.File
  ) {
    console.log(file);
  }

}

If your validators are getting much, you can create them in a separate file and import them here as a named constant like myValidators. - NestJs Docs

Using ParseFilePipeBuilder

Wow, this is amazing! It's so much better because it comes with built-in validators for file size and type! Plus, there's a way to add even more validators using the addValidator method for complex file validation! How cool is that?!

import { ..., ParseFilePipeBuilder, HttpStatus } from '@nestjs/common';
...

@Post('upload')
@UseInterceptors(FileInterceptor('file', { storage: defaultConfig }))
uploadFile(
  @UploadedFile(
    new ParseFilePipeBuilder()
      .addFileTypeValidator({
        fileType: /(jpg|jpeg|png|gif)$/,
      })
      .addMaxSizeValidator({
        maxSize: 1000
      })
      .addValidator(
        new MaxFileSize({
          maxSize: MAX_UPLOAD_SIZE
        }),
      ) 
      .build({
        errorHttpStatusCode: HttpStatus.UNPROCESSABLE_ENTITY
      })
  )
  file: Express.Multer.File
) {
  console.log(file);
}

...

errorHttpStatusCode is the HTTP status code you want to be returned when validation fails.

Simplify the process of file validation with the help of ParseFilePipeBuilder. For straightforward validations like file types and sizes, this tool is highly recommended. However, for more intricate validations, it's best to create file validator classes.

Optional File Uploads

File upload parameters can be made optional or not required by passing an extra option in the ParseFilePipeBuilder build method.

new ParseFilePipeBuilder()
.addFileTypeValidator({
  fileType: "png",
})
.build({
  fileIsRequired: false // It's required by default
})

Accepting multiple files

Simply change the file interceptor and upload the file decorator.

import { UploadedFiles } from '@nestjs/common';
import { FilesInterceptor } from '@nestjs/platform-express';

const MAX_FILES_COUNT = 10;

...

@Post('upload/multiple')
@UseInterceptors(
  FilesInterceptor('files', MAX_FILES_COUNT, { storage: defaultConfig })
)
uploadFiles(
  @UploadedFiles(
    new ParseFilePipeBuilder()
      .addFileTypeValidator({
        fileType: /(jpg|jpeg|png|gif)$/,
      })
      .addMaxSizeValidator({
        maxSize: 1000
      })
      .addValidator(
        new MaxFileSize({
          maxSize: MAX_UPLOAD_SIZE
        }),
      ) 
      .build({
        errorHttpStatusCode: HttpStatus.UNPROCESSABLE_ENTITY
      })
  )
  files: Express.Multer.File[]
) {
  console.log(files);
}

...

It's now UploadedFiles and FilesInterceptor. FilesInterceptor accepts max files number that can be uploaded through the endpoint.

Make sure to accept files as an argument, not just file.

Below, you can find the complete code.

import { Post, UploadedFile, UseInterceptors, Controller, ParseFilePipe, ParseFilePipeBuilder, HttpStatus, UploadedFiles } from '@nestjs/common';
import { FileInterceptor, FilesInterceptor } from '@nestjs/platform-express';
import { ApiConsumes, ApiTags } from '@nestjs/swagger';
import { Request } from 'express';
import { diskStorage } from 'multer';
import * as path from 'path';
import { FileValidator } from '@nestjs/common';

const UPLOAD_DIR = './upload/files/';
const MAX_UPLOAD_SIZE = 10; // in MB
const MAX_FILES_COUNT = 10; // Maximum number of files that can be uploaded at once

// User interface of the authenticated user
interface User {
  id: string;
}

class MaxFileSize extends FileValidator<{ maxSize: number }>{
  constructor(options: { maxSize: number }) {
    super(options)
  }

  isValid(file: Express.Multer.File): boolean | Promise<boolean> {
    const in_mb = file.size / 1000000
    return in_mb <= this.validationOptions.maxSize
  }
  buildErrorMessage(): string {
    return `File uploaded is too big. Max size is (${this.validationOptions.maxSize} MB)`
  }
}

/**
 * You can use this function to generate a unique filename for each file
 * User id is used to generate a unique filename
 * The User object can be attached to the request object in the auth middleware
 */
const defaultConfig = diskStorage({
  destination: UPLOAD_DIR,
  filename: (req: Request & { user: User }, file, cb) => {
    const uid = req.user.id;
    cb(null, `${uid}${path.extname(file.originalname)}`)
  }
})

@ApiTags('Uploding Files')
@Controller('files')
@ApiConsumes('multipart/form-data')
export class FileController {

  @Post('upload')
  @UseInterceptors(FileInterceptor('file', { storage: defaultConfig }))
  uploadFile(
    @UploadedFile(
      new ParseFilePipeBuilder()
        .addFileTypeValidator({
          fileType: /(jpg|jpeg|png|gif)$/,
        })
        .addMaxSizeValidator({
          maxSize: 1000
        })
        .build({
          errorHttpStatusCode: HttpStatus.UNPROCESSABLE_ENTITY
        })
    )
    file: Express.Multer.File
  ) {
    console.log(file);
  }

  @Post('upload/multiple')
  @UseInterceptors(
    FilesInterceptor('files', MAX_FILES_COUNT, { storage: defaultConfig })
  )
  uploadFiles(
    @UploadedFiles(
      new ParseFilePipeBuilder()
        .addFileTypeValidator({
          fileType: /(jpg|jpeg|png|gif)$/,
        })
        .addValidator(
          new MaxFileSize({
            maxSize: MAX_UPLOAD_SIZE
          }),
        )
        .build({
          errorHttpStatusCode: HttpStatus.UNPROCESSABLE_ENTITY
        })
    )
    files: Express.Multer.File[]
  ) {
    console.log(files);
  }
}

For a comprehensive configuration of Multer file upload, refer to the NestJs File Upload Documentation. Read on to learn more.

Conclusion

In conclusion, this article provides a comprehensive guide on how to implement file upload and validation in NestJS using the Multer middleware package for the Express Adapter. It covers accepting and validating files, saving files to disk storage, file validation with ParseFilePipe and ParseFilePipeBuilder, and accepting multiple files. By following the steps outlined in this article, developers can easily design their custom file upload and validation flow in NestJS.

Building an ECDSA wallet with JavaScript

Israel Ayanwola — Sun, 29 Jan 2023 19:05:24 +0000

Blockchain is an amazing technology to dive into. It has been around for some time now and powering very popular technologies like Ethereum.

To make this article as simple as possible, the definitions and terminologies would be summarized to just key points needed for the project you will build in this article. But links to amazing educational articles will be provided for each.

By the end of this article, you would understand Public Key Cryptography, Elliptic curve digital signatures and how they can be used to make the world a better place. I will take you one step at a time on how to build a blockchain node.

What is Cryptography

Cryptography is a way of securing a piece of information. It's the study of cryptographic hash functions.

Cryptographic hash functions are mathematical functions that take an input of any size and convert it to a fixed-sized output. It's a one-way ticket, you can't get the input from the output.

Cryptography is at the heart of blockchain and Web3. Here is a link to an online tool containing lots of hash functions you can play with.

Public Key Cryptography

It's the use of two keys, a public key (a key that is known to everyone), and a private key (a key that is kept secured). The private key can encrypt a message that can only be decrypted by the public key, verifying that the message was indeed sent by someone with the right private key and vice versa.

Public key cryptography is also called asymmetric cryptography because it uses a pair of related keys.

RSA and ECDSA are two popular algorithms used for public cryptography. RSA is based on the idea that you can quickly find the product of two prime numbers, but extremely hard to factor out the prime numbers once you have the products. Read more about RSA on Wikipedia.

ECDSA (Elliptic curve digital signature algorithm) uses elliptic curves. Read more about it on Wikipedia.

Why use ECDSA

ECDSA provides the same level of security as RSA but with small key sizes. RSA keys can be very large making it take a long to transfer over a network.

ECDSA is the algorithm used by Bitcoin, which is the secp256k1 curve.

Building an ECDSA node

This is an example of a blockchain node that uses the ECDSA signing algorithm. The project is a simple react frontend application that allows users to send money to each other. There will be a single server node, which makes this centralized but there will be an article later on how to deploy a program on Ethereum.

Goal

The server would manage the balances of the users. The goal is to enable the server to verify who is sending the money to make sure that a person can only send money from their wallet.

Project prerequisites

You will need to have a basic knowledge of React as it is used for the front end. With a basic understanding of Express.

Application Architecture

There is the front end, which has the UI part of the project that allows users to enter their wallet addresses and attempt to send money to others. The Idea here is that, when a user tries to send money, we would generate a transaction for them to sign with their private key, then input the signed transaction which would be sent to the server with their public key (wallet address), address where money should be sent, and the amount of money to be sent.

The server would take this signed transaction and the public key, then verify if the transaction was indeed signed using the right private key of the public key. If it's correct, it updates the users (sender and recipient) balances, else it returns an error message.

Client-Side React App

We would use Vite for our front-end tooling.

Create a new directory called wallet-node and move into the directory.

Run npm create vite@latest, enter client as your project name, react as the framework and JavaScript as the variant.

Move into the client directory just created by Vite. Run npm i to install the packages from the template.

Install Sass as a dev dependency using this command npm i -D sass. This will install the sass package as we will be using sass for our CSS styling.

The last installments are:

Axios - npm i axios
Ethereum Cryptography - npm i ethereum-cryptography

Now, edit the value of the title tag in the index.html file to ECDSA Wallet or anything you want.

Move into the src directory, delete the assets directory, index.css and App.css files.

Edit the main.jsx file, remove the line for import './index.css'.

Edit App.jsx by removing everything in it and pasting in the following code;

import Wallet from "./Wallet";
import Transfer from "./Transfer";
import "./App.scss";
import { useState } from "react";

function App() {
  const [balance, setBalance] = useState(0);
  const [address, setAddress] = useState("");

  return (
    <div className="app">
      <Wallet
        balance={balance}
        setBalance={setBalance}
        address={address}
        setAddress={setAddress}
      />
      <Transfer setBalance={setBalance} address={address} />
    </div>
  );
}

export default App;

We will create a Wallet and Transfer component, and our App.scss file for the sass styles.

From the above code, two stateful values were created, balance and address. The balance will be used to keep track of and manage an address balance, while the address will be used to share a user address among the Wallet and Transfer components.

The transfer component accepts a prop setBalance which would be used to set the state of the user's balance after the money has been transferred successfully.

Create an App.scss file in the src directory, then paste the following code;

body {
  font-family: "Muli", sans-serif;
  font-weight: 300;
  background-color: #e2e8f0;
  padding: 40px;
}

label {
  display: flex;
  flex-direction: column;
  letter-spacing: 0.05rem;
  font-size: .8em;
  font-weight: 400;
  color: #222;
}

.app {
  display: flex;
  max-width: 1400px;
  flex-wrap: wrap;
  gap: 12px;
  margin: 0 auto;
}

.container {
  flex-grow: 1;
  margin: 0 20px;
  background-color: #fff;
  border: 1px solid #cbd5e0;
  box-shadow: 0 1px 2px 0 rgba(0, 0, 0, 0.05);
  border-radius: 0.375rem;
  padding: 40px;

  label {
    margin: 10px 0;
  }

  .button {
    margin-top: 10px;
  }
}

input {
  padding: 10px 0;
  border-radius: 0.125rem;
  border: 1px solid rgb(226, 232, 240);
  background-color: #fdfdfe;
  padding-inline-start: 0.75rem;
  font-size: 0.875rem;
}

.button {
  background-color: #319795;
  border-radius: 0.125rem;
  padding: 10px 20px;
  color: white;
  display: inline-flex;
  text-transform: uppercase;
  letter-spacing: 1px;
  font-weight: 400;
  font-size: .9em;
  &:hover {
    cursor: pointer;
  }
}

.wallet {
  display: flex;
  flex-direction: column;

  .balance {
    text-transform: uppercase;
    letter-spacing: 1px;
    font-weight: 400;
    font-size: .9em;
    display: inline-flex;
    margin-top: 10px;
    padding: 0.75rem;
    background-color: #f4f6f8;
  }
}

.transfer {
  display: flex;
  flex-direction: column;
}

Create a server.js file and paste the code below;

import axios from "axios";

const server = axios.create({
  baseURL: "http://localhost:3042",
});

export default server;

This creates an Axios instance that uses http://localhost:3042 as the base URL, this is the URL our server will be listening on. The Axios instance is exported to be used by other services.

Now we would need to create a very important function. Here we would use cryptographic functions 😁. Specifically, the keccak256 hash function, which would be used to hash our transaction messages.

Create a services.js file, and paste the code below;

import { keccak256 } from "ethereum-cryptography/keccak"
import { utf8ToBytes, toHex } from "ethereum-cryptography/utils"

export async function hashMessage(message){
    return toHex(keccak256(utf8ToBytes(message)))
}

Now let's talk about what is happening here.

We have created a hashMessage function, which accepts a message (string) and returns a hash (string).

Keccak256, the function is imported from the Ethereum cryptography package installed earlier, it accepts bytes (Uint8Array) and returns a hash of fixed-sized output. The return type of this hash is a byte. The utf8ToBytes converts the message passed to the function to bytes which are then passed to the keccak256 function.

Finally, the function, toHex, takes the returned bytes from the keccak256 function and converts it to a hexadecimal string, which is then returned by the hashMessage function.

Its purpose is to encrypt a transaction message which would then be sent to the user to sign with their private key 🔑.

Creating the Wallet Component

The functionality of this component is to allow the user to input any address and see its balance. Create a Wallet.jsx file in the src directory, then copy and paste the code below into the file.

import server from "./server";

function Wallet({ address, setAddress, balance, setBalance }) {
  async function onChange(evt) {
    const address = evt.target.value;
    setAddress(address);
    if (address) {
      const {
        data: { balance },
      } = await server.get(`balance/${address}`);
      setBalance(balance);
    } else {
      setBalance(0);
    }
  }

  return (
    <div className="container wallet">
      <h1>Your Wallet</h1>

      <label>
        Wallet Address
        <input placeholder="Type an address, for example: 0x1" value={address} onChange={onChange}></input>
      </label>

      <div className="balance">Balance: {balance}</div>
    </div>
  );
}

export default Wallet;

Every time the input changes it will invoke the onChange event handler which will request the address balance from the server. If successful it will update the balance state using the setBalance action. The Balance and Address states are created from the App.jsx App component because their states will be used by the Transfer component too.

Transfer Component

This component will be used by the user to send any amount to any recipient. Create a Transfer.jsx file in the src directory, then copy and paste the code below into the file.

import { useState } from "react";
import server from "./server";
import { hashMessage } from "./services"

function Transfer({ address, setBalance }) {
  const [sendAmount, setSendAmount] = useState("");
  const [recipient, setRecipient] = useState("");

  const setValue = (setter) => (evt) => setter(evt.target.value);

  async function getSignature(evt){
    evt.preventDefault();

    try {
      let data = {
        recipient,
        amount: parseInt(sendAmount)
      }
      let msgHex = await hashMessage(JSON.stringify(data))
      let signature = prompt(`Sign message (${msgHex}) and provide signature:`)
      if (signature === null){
        alert("You did not provided a signature")
        return
      }
      await transfer(signature)
    } catch (ex) {
      alert(ex.response.data.message);
    }

  }

  async function transfer(signature) {
    const {
      data: { balance },
    } = await server.post(`send`, {
      sender: address,
      amount: parseInt(sendAmount),
      recipient,
      signature,
    });
    setBalance(balance);
    alert("Funds transferred successfully!")
  }

  return (
    <form className="container transfer" onSubmit={getSignature}>
      <h1>Send Transaction</h1>

      <label>
        Send Amount
        <input
          placeholder="1, 2, 3..."
          value={sendAmount}
          onChange={setValue(setSendAmount)}
        ></input>
      </label>

      <label>
        Recipient
        <input
          placeholder="Type an address, for example: 0x2"
          value={recipient}
          onChange={setValue(setRecipient)}
        ></input>
      </label>

      <input type="submit" className="button" value="Transfer" />
    </form>
  );
}

export default Transfer;

This is a lot, so let's break it down.

...
function Transfer({ address, setBalance })
...

The transfer component accepts two values;

address - state object which is the sender's (user's) address
setBalance - state action which will be used to update the user's balance

const [sendAmount, setSendAmount] = useState("");
const [recipient, setRecipient] = useState("");

These states are used to set the amount to be sent and the recipient's address.

...
const setValue = (setter) => (evt) => setter(evt.target.value);
...

This function takes in a callable, setter as an argument and returns an event handler. The function of the event handler is to pass the target value of the event object as an argument to setter.

It's used here;

...
<label>
  Send Amount
  <input
    placeholder="1, 2, 3..."
    value={sendAmount}
    onChange={setValue(setSendAmount)}
  ></input>
</label>

<label>
  Recipient
  <input
    placeholder="Type an address, for example: 0x2"
    value={recipient}
    onChange={setValue(setRecipient)}
  ></input>
</label>
...

The function, setValue , is used for the onChange events of both inputs above. Which updates the states of both amount and recipient with the values of their respective inputs.

Now, the function, getSignature , will be demystified. The goal here is that when the user enters a recipient address, the amount to be sent, and submits the form. A transaction object will be created which will contain the amount and recipient address, this transaction will then be displayed to the user so that they can sign it with their private key 🔑. Then the signature will be passed to the transfer function.

...
async function getSignature(evt){
  evt.preventDefault();

  try {
    let data = {
      recipient,
      amount: parseInt(sendAmount)
    }
    let msgHex = await hashMessage(JSON.stringify(data))
    let signature = prompt(`Sign message (${msgHex}) and provide signature:`)
    if (signature === null){
      alert("You did not provided a signature")
      return
    }
    await transfer(signature)
  } catch (ex) {
    alert(ex.response.data.message);
  }
}
...

The transfer function;

...
async function transfer(signature) {
  const {
    data: { balance },
  } = await server.post(`send`, {
    sender: address,
    amount: parseInt(sendAmount),
    recipient,
    signature,
  });
  setBalance(balance);
  alert("Funds transferred successfully!")
}
...

This sends the signature and the transaction data to the server. If the signature is valid for the transaction data, then the money will be transferred to the recipient's address.

That's all for the client-side application.

Server-Side NodeJS Application

The backend will be an express application. It's going to have two simple handlers;

GET /balance/:address - to get an address balance
POST /send - to transfer funds from one address to another.

Also, we would need to create some useful scripts for generating new random wallets and signing transactions. The latter is meant to help users sign transaction messages with their private key 🔑.

Firstly, let's set up the server directory. If not already created, create a new directory, server , in the root directory. Move into the directory, and run npm init -y to initialize the default package.json file.

Copy and paste the content below into the package.json file.

{
  "name": "server",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "generate": "node ./scripts/gen.js",
    "sign": "node ./scripts/signer.js",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "cors": "^2.8.5",
    "ethereum-cryptography": "^1.1.2",
    "express": "^4.18.1",
    "yargs": "^17.6.2"
  }
}

Then run npm install to install dependencies.

Before creating the scripts for generate and sign, we will be creating services.js first to define shared functions.

Services

Create a new file service.js in the server directory. Copy and paste the code below into it.

const secp = require("ethereum-cryptography/secp256k1");
const { keccak256 } = require("ethereum-cryptography/keccak")
const { utf8ToBytes, toHex } = require("ethereum-cryptography/utils")

function extractPublicKey(fullKey){
    let kec = keccak256(fullKey.slice(1, fullKey.length));
    return toHex(kec.slice(kec.length - 20, kec.length))
}

function verifySignature(sig, msg, pubKey){
    const msgHash = keccak256(utf8ToBytes(msg));
    let actualSignature = sig.slice(0, sig.length - 1)
    let recoveryBit = parseInt(sig[sig.length - 1])
    const sigPubKey = secp.recoverPublicKey(msgHash, actualSignature, recoveryBit);
    const mainKey = extractPublicKey(sigPubKey);
    return mainKey == pubKey
}

module.exports = {
    verifySignature,
    extractPublicKey
}

The function, extractPublicKey, accepts bytes array as an argument, this is the byte format of a full public key. Then hashes it with keccak256 and returns the hexadecimal string of the last 20 bytes of the hash. Ethereum docs explains why it's required. It's to make it shorter 😉.

The last function, verifySignature, accepts a signature, a transaction message, and the sender's public key. All this data is required to verify that the transaction message was indeed signed using the private key of the owner of the public key.

The function hashes the transaction message and used [recoverPublicKey](https://github.com/ethereum/js-ethereum-cryptography#secp256k1-curve:~:text=%3A%20boolean%0Afunction-,recoverPublicKey,-(msgHash%3A to get the public key of the signer. After extracting the short format using extractPublicKey, it compares it will the public key, pubKey, passed to the function.

Script - generate: Generating new random public and private key pairs

const secp = require("ethereum-cryptography/secp256k1")
const { toHex } = require('ethereum-cryptography/utils')
const { extractPublicKey } = require('../services')

let privateKey = secp.utils.randomPrivateKey();
let pubKey = secp.getPublicKey(privateKey)

pubKey = extractPublicKey(pubKey)

console.log("Private key:", toHex(privateKey))
console.log("Public key:", pubKey)

A new private key is generated using the randomPrivatekey function. The public key is also extracted from the private key. The mathematical property that made this possible is so amazing. The private key can never be known from the public key.

Script usage example:

npm run generate

Console output:

Script - sign: Signing transaction messages

const yargs = require('yargs/yargs')
const { hideBin } = require('yargs/helpers')
const secp = require('ethereum-cryptography/secp256k1')
const { toHex } = require("ethereum-cryptography/utils")

let args = yargs(hideBin(process.argv))
    .option('private_key', {
        alias: 'p',
        type: 'string',
        description: 'Your Private Key',
        demandOption: true
    })
    .option('data', {
        alias: 'd',
        type: 'string',
        description: 'Payload to sign',
        demandOption: true
    })
    .parse()

let privKey = args.private_key
let msgHash = args.data

secp.sign(secp.utils.hexToBytes(msgHash), privKey, { recovered: true }).then(data => {
    const [signature, recovery_bit] = data
    let sig = toHex(signature);
    console.log("Your Signature:", sig)
    console.log("Your Recovery Bit:", recovery_bit)
    let fullSig = sig + recovery_bit.toString()
    console.log("Copy and paste this as the full signature, this has the recovery bit attached to the end:\n", fullSig)
})

This is more of a CLI tool. It accepts the private key and message hash as command-line arguments. The sign function from the module, ethereum-cryptography/secp256k1 , is used to sign the message hash, msgHash. The result of the signature and recovery_bit received from the data are later concatenated to form a single string, signature, which is logged to the console.

The signature is expected in the component, Transfer , in the client-side application.

So the user can copy it from the console and paste it into the prompt.

Script usage example

npm run sign -- -p 3ebefedbd43cbd88f0504acd101df139ddce0656da699b8350c1db9eaf193484 -d 3ebefedbd43cbd88f0504acd101df139ddce0656da699b8350c1db9eaf178970

Console output:

Now create the index.js file, which will contain our API routes and handlers, in the src directory.

Follow the steps below and paste the codes gradually.

const express = require("express");
const app = express();
const cors = require("cors");
const port = 3042;
const { verifySignature } = require("./services")

app.use(cors());
app.use(express.json());

const balances = {
  "KEY_A": 100,
  "KEY_B": 50,
  "KEY_C": 75,
  // KEY_N: Any amount
};

The object, balances , is currently acting as the database. The keys of the object will be public keys of different wallets, and the values will be their respective balance.

New public and private key pairs can be generated using the generate script. After creating any amount of key pairs, update the object, balances, with the public keys. Make sure to save their respective private keys too, so they can be used later for signing transaction messages.

GET - /balance/:address

app.get("/balance/:address", (req, res) => {
  const { address } = req.params;
  const balance = balances[address] || 0;
  res.send({ balance });
});

This is the route used to get an address balance.

POST - /send

app.post("/send", (req, res) => {
  const { sender, recipient, amount, signature } = req.body;

  const msg = JSON.stringify({
    recipient,
    amount
  })
  let isValid = verifySignature(signature, msg, sender);
  if (isValid === false){
    res.status(400).send({ message: "Invalid Signature!" })
    return
  }

  setInitialBalance(sender);
  setInitialBalance(recipient);

  if (balances[sender] < amount) {
    res.status(400).send({ message: "Not enough funds!" });
  } else {
    balances[sender] -= amount;
    balances[recipient] += amount;
    res.send({ balance: balances[sender] });
  }
});

This route verifies the amount and recipient, against the signature and public key of the sender. If the signature is valid, it tries to debit the sender, if this is successful it will update the recipient balance.

Lastly,

app.listen(port, () => {
  console.log(`Listening on port ${port}!`);
});

function setInitialBalance(address) {
  if (!balances[address]) {
    balances[address] = 0;
  }
}

Start the server to listen on any given port.

If the port is changed, make sure it's also updated in the client application.

The function, setInitialBalance, checks if an address exists in the database. If the address does not exist, it adds the address to the database with a balance of zero. This is a very nice method, as we don't need to manually add new users to our database.

Project codes are Github

Resources

Cryptographic tools for Ethereum - Ethereum Cryptography

Node CLI tool - Yargs

Play with Cryptographic hashes - SHA256 Online tool

Public Key Cryptography - RSA Algorithm & Elliptic Digital Curves (ECDSA)

Cryptography in HTTPS - Diffie Hellman Key Exchange & TLS handshake for HTTPS

Conclusion

The idea presented in this article can also be used in several fields on technology to maintain data integrity when data is transferred between several components through a network.

One flaw of the server built today is that it's a single node, which makes the database a centralized instance. This is where a blockchain ledger would be useful because it's managed by more nodes which are all bound by rules (consensus) making the whole system decentralized. An account with a public key, TEST, on node A, would have the same balance on node B.

Building applications on the blockchain will be covered later in new articles, subscribe to the newsletter to receive notifications when new articles drop.

Follow me on Twitter @netrobeweb.

Build an Online Bookshop with Xata and Cloudinary

Israel Ayanwola — Wed, 23 Nov 2022 11:19:37 +0000

Hi, there. Today, We will learn how to use Xata and Cloudinary to build Bookay, which is Jamstack architecture. Bookay is a platform for techies in Nigeria to buy technical books and sell used ones. It's an online bookshop. With Bookay, you can purchase books from the comfort of your home or sell your used books to earn more profits.

Xata is a serverless database platform with no need for servers. It's one of the features I love about this product. As a backend developer / DevOps engineer, managing relational databases daily can be very stressful. But by using Xata, You don't have to deal with database migrations again, database management, and performance tuning. You just need to focus on what you need to do and let Xata focus on Data.

Cloudinary helps with images and videos. You can use it to manage image and video uploading, deliver images anywhere on the web, and transform them. An example of image transformation is converting an image to a thumbnail. Cloudinary is used in Bookay to easily upload book images and edit and deliver images to users searching for books. We would use Cloudinary pluggable image uploader available as a React component; this comes with out-of-the-box functionality to set up image uploading swiftly. Image transformation can happen right in the element used to display them. Book images can be uploaded once and transformed anytime, anywhere.

This article will teach you how to build a web application where you can buy books and sell your books online. We would go through Bookay's architecture, teaching you how to use Xata's database-less platform and Cloudinary's image upload features to build your online bookshop. Building your applications using the platforms above, can make your application development less affected by changes in business logic.

Project Idea: Bookay

Finding technical books is not easy in Nigeria. Books are available, but it's not easy to find. The idea behind Bookay, an online bookshop, is to make books ready and available online. It is a platform where you can easily find any available technical or non-technical books you want to buy. Bookay allows sellers to list their books on the forum, allowing buyers to search and buy books easily.

Architecture Overview: Xata + Cloudinary + NextJs

Using just NextJs, Bookay was built. Uploading of data to Xata is done with APIs. Xata comes with built-in automated full-text search functionality on all created tables ❤️. Images are uploaded to Cloudinary, from which you can quickly transform them while viewing it. For example, Image thumbnails can be used when displaying a list of books from search results, and larger images are used when showing a single book.

Although, security is needed when working with APIs. NextJs allows developers to make API calls on the server side, secluding API authentication details from the client side. This is why NextJs can easily be the best framework for building Jamstack products. But, Xata does not depend on any language or framework, It is language-agnostic. You can use any of your favourite languages or frameworks to use Xata.

Building Bookay: Your Online Bookshop 🛩️

Let's go through how you can build your own online bookshop. Bookay is already deployed on Vercel and it's Open source on GitHub.

Let's get started by setting it up on our computer.

Prerequisites:

Node v12.18.3 or higher,
npm v6.14.6 or higher.

Dependencies

Xata: Set up your database for free
Cloudinary: Quickstart with Cloudinary React

Installation:

A step-by-step series of examples that tell you how to get a development env running.

Clone the project and make sure you are in the project root directory

    git clone https://github.com/devvspaces/bookay
    cd bookay

Installing dependencies

    npm install

The above command would install all necessary dependencies to run Bookay on your computer.

Setting up your own Xata and Cloudinary account is needed. After installing the required dependencies, run the command, xata auth login, to authenticate with Xata and create your DB. For more information on using Xata CLI.

When starting your New project, Xata will automatically set up your development environment.

Run your development server

    npm run dev

Usage

Visit your running server URL; this will likely be http://localhost:3000
Login / Register: This link allows you to log in or automatically create an account if it doesn't exist.
Register as Seller: This allows you to create a seller account specifically.
You can now add new books, update books, and view and delete books.
You can check your sellers' store orders and normal orders for users.
And the rest is history!

Development 💪🏽

The best feature of the project development is code reusability.

Models

In this project, we will use write model classes. These are JavaScript objects that contain methods that serve as utility methods for interacting with a table on Xata. Model classes will use the Xata SDK API to read from or write to the database. Xata's concept revolves around tables. Tables can relate to one another like a relational database. Most of the data and business logic reside here. It also allows the code to be tested without calling Xata during tests. This can be achieved just by mocking the model classes and overriding their methods. An example of the test procedure can be found in my other repo, the Xata demo todo app.

Features on your online bookshop made available by Xata are database CRUD management, full complete type safe typescript SDK, and full-text search. With these features, you are not limited to building only an online bookshop. The sky is your limit.

API Routes and Server Side Props.

API routes and Server Side Props are the only communication methods with Xata API. This is the best and recommended way because it makes sure your Xata API requests are secured and hidden. So that API Keys won't get sniffed out. Data fetching in NextJs runs on the Server Side. This development pattern available with NextJs can help you build complete web applications with a single framework. Furthermore, the app will run most of its life cycle in the client's browsers, making it blazingly fast.

Cloudinary for Image Transformation

As said earlier, this is the easiest to set up. You just install Cloudinary react components, and that's all. Using Cloudinary's documentation, you can easily set up your image book uploader and editor. It's right there in the documentation. It doesn't require complex requirements like API routes or Server Side Props. With React components, you can have a complete image upload and preview set up in 10 min.

Deployment ❤️

You can deploy your new online bookshop on Vercel or Netlify in minutes. Just connect the platform to your GitHub account. Give access to your project, and you will get a smooth developer experience for deployment. Deployment has never been this interesting and easy. It's fast. It's also pre-built with CI / CD integrations, allowing your online bookshop to automatically be redeployed to new URLs every time you update any branch. Tutorial to do that here.

Book's source code is well documented; going through the code alone can help you learn more about the project. Installing, running, and testing the project on your computer can give you more practical experience in understanding the project's architecture. You will get more experience building Xata and Cloudinary applications by adding your features to the app.

Conclusion 😎

The developing experience of using frameworks or platforms is essential. Apart from making your development faster, It means you can develop web applications while paying less to no attention to the platform's inner workings. This is Xata and Cloudinary. Focus less on how you will set up that database or that user's profile image. They will handle it. Thanks for reading.

Working with strings effectively with Golang

Israel Ayanwola — Sun, 09 Oct 2022 01:26:46 +0000

What are Strings? 🤔

The way strings is stored in memory makes them immutable, making it difficult to perform simple operations like changing the value of an index in a string. For example you can't perform an index assignment operation on a string, but this is possible with arrays. Golang has a built-in library that can help us work with strings. We are able to read and manipulate strings in various ways.

In this tutorial series, we would discuss about ways we can manipulate a string in Golang. By looking into

Peeking,
Traversing,
Mutating, and
Sorting

of strings in Golang, you should be confident enough to work with them in any aspect of programming you come across.

Working with Strings 🛩

Peeking

In this use case Peeking means inspecting a string trying to find what it looks like. For example, checking the first character of a string, the character at an index, the characters in a range of two index, etc.

Testing the first and last characters of a string

We can test what starts (prefixes) and ends (suffixes) a string.

package main

import (
    "fmt"
    "strings"
)

func main() {
    var str string = "Hashnode is a very easy tool to use for blogging"
    fmt.Printf("T/F? \nDoes the string \"%s\" have prefix %s? ", str, "Th")
    fmt.Printf("\n%t\n\n", strings.HasPrefix(str, "Th")) // Finding prefix

    fmt.Printf("Does the string \"%s\" have suffix %s? ", str, "ting")
    fmt.Printf("\n%t\n\n", strings.HasSuffix(str, "ing")) // Finding suffix
}

Run code live here

Output:

T/F? 
Does the string "Hashnode is a very easy tool to use for blogging" have prefix Th? 
false

Does the string "Hashnode is a very easy tool to use for blogging" have suffix ting? 
true

From the output above, the function HasPrefix checks whether the string in its first parameter starts with the Th. The function HasSuffix does the opposite of that, it checks the end of the string. The HasPrefix function returns false because the string in variable str doesn't start with Th. While HasSuffix returned true because the string str ends with ing.

The above methods show a simple way to test what starts and ends any string.

Indexing a string

For beginners, indexing a string in Go for the first time will seem weird. When you index a string in Go you get a rune. The Go language defines the word rune as an alias for the type int32. Read more about string in Go

Take the following code sample as an example;

package main

import (
    "fmt"
)

func main() {
    var str string = "Hashnode is a very easy tool to use for blogging"
    fmt.Println(str[0])
}

Run code live here

Output

Converting the rune to a string gives the actual character at that index.

package main

import (
    "fmt"
)

func main() {
    var str string = "Hashnode is a very easy tool to use for blogging"
    fmt.Println(string(str[0]))
}

Run code live here

Output

Testing if a string contains a substring

To check if a string contains some string, do the following

package main

import (
    "fmt"
    "strings"
)

func main() {
    var str = "I code Python, Golang, JavaScript, TypeScript and PHP"
    fmt.Println(strings.Contains(str, "Golang"))
    fmt.Println(strings.Contains(str, "Rust"))
}

Run code live here

Output

true
false

The function Contains returns true if the second parameter is found in the first parameter. Otherwise, it returns false.

Locating the index of a substring or character in a string

We can locate the first or last occurrence of a substring or character in a string.

package main

import (
    "fmt"
    "strings"
)

func main() {
    var str string = "Hi, I'm Marc, Hi."
    fmt.Printf("The position of the first instance of\"Marc\" is: ")
    fmt.Printf("%d\n", strings.Index(str, "Marc")) // Finding first occurence
    fmt.Printf("The position of the first instance of \"Hi\" is: ")
    fmt.Printf("%d\n", strings.Index(str, "Hi")) // Finding first occurence
    fmt.Printf("The position of the last instance of \"Hi\" is: ")
    fmt.Printf("%d\n", strings.LastIndex(str, "Hi")) // Finding last occurence
    fmt.Printf("The position of the first instance of\"Burger\" is: ")
    fmt.Printf("%d\n", strings.Index(str, "Burger")) // Finding first occurence
    fmt.Printf("%d\n", strings.IndexRune(str, 'H')) // Finding first occurence
}

Run code live here

Output

The position of the first instance of"Marc" is: 8
The position of the first instance of "Hi" is: 0
The position of the last instance of "Hi" is: 14
The position of the first instance of"Burger" is: -1
0

Index returns the index of the first instance of the second parameter in the first parameter. LastIndex searches from the end of the string, returning the index of the last occurrence of the character or substring in the searched string. IndexRune returns the index of the first instance of the Unicode code point 'H' in the string str.

To be continued

Hope you learned something new today!. This part of the String series focuses on how you can test the values of a string in Go. You can practice more of the functions covered in this tutorial. To master and do more practice check out the Strings module documentation. Learn on your own how to use strings.Count to count the number of occurences of a character or substring in a string.

In the next part, we would look at ways a string can be manipulated in Go. Thanks for reading.