DEV Community: chryzcode

Google Authentication in Nodejs using Passport and Google Oauth

chryzcode — Tue, 23 Apr 2024 17:07:05 +0000

Google authentication is one of the seamless and fastest modes of authenticating users in an application. It saves the user the time of filling out forms with his/her details and verifying email addresses.

In this article, we will implement Google authentication in our NodeJS application.

Create Your NodeJS Application

Install Node.js

You should have Nodejs installed on your laptop and if not, check the Node.js official website, and download/ install the latest and stable release.

To verify if node.js was successfully installed, paste the command below on your Command Line Interface (CLI) to check the version of node.js installed.
```
// terminal
node --version
```

Set up a simple Node.js app

Create a directory for your application, I will create mine on the desktop using the CLI. I am using the Windows operating system and VS Code editor.

// terminal 
cd desktop 
mkdir my-nodejs-app //create a project directory
cd my-nodejs-app // navigate to the app directory
npm init -y // create a default package.json file
npm install express dotenv passport passport-google-oauth20 express-session // install necessary dependencies
code . // to open the directory on my code editor (VS Code). You can do it manually.

Modify package.json file

Configure the JSON file to get connected to the server.js file, for the application to use the ECMAScript 6 (ES6) modules, and to run a development server using the --watch flag.

//package.json
{
  "name": "my-nodejs-app",
  "version": "1.0.0",
  "description": "",
  "type": "module", // add this to use the ES6 
  "main": "server.js", // update this to 
  "scripts": {
    "dev": "node --watch server.js" // script to run a development server
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "dotenv": "^16.4.5",
    "express": "^4.19.2",
    "express-session": "^1.18.0",
    "passport": "^0.7.0",
    "passport-google-oauth20": "^2.0.0"
  }
}

Create the Server

Create an server.js file in the base of the directory to set up our project server.

// server.js

// import necessary depenencies
import "dotenv/config";
import express from "express";
import passport from "passport"; 
import { Strategy as GoogleStrategy } from "passport-google-oauth20";
import session from "express-session";

// intialize app and define the server port
const app = express();
const port = process.env.PORT || 8000;

// a middleware to access json data
app.use(express.json());

// a view to check if the server is running properly 
// check `http://127.0.0.1:${port}/` -> http://127.0.0.1:800
app.get("/", (req, res) => {
  res.send(`My Node.JS APP`);
});

// a function to start the server  and listen to the port defined
const start = async () => {
  try {
    app.listen(port, () => console.log(`server is running on port ${port}`));
  } catch (error) {
    console.log(error);
  }
};

// call the function
start();

Generate a Google client Oauth credentials

You have to generate a Google client Oauth credentials(Client ID and Secret), if you are new to this, check out tutorials on how to go about it.

Create a.envfile

Make sure dotenv dependency is installed and imported into your server.js file. Use the format below to set the .env file.

//.env
// make sure these values are correct
GOOGLE_CLIENT_ID= // your google client id
GOOGLE_CLIENT_SECRET= // your google client client
SESSION_SECRET= // any randome secure characters
PORT= 8000 // define port from the env file

Createpassport.jsfile

Create an utils folder in the project directory base and create passport.js file. This is where the connection of our Node.js app to the Google Oauth App created on the Google console takes place.

// utils/passport.js

// import necessary dependencies
import passport from "passport";
import { Strategy as GoogleStrategy } from "passport-google-oauth20";

//initialize 
passport.use(
  new GoogleStrategy(
    {

      clientID: process.env.GOOGLE_CLIENT_ID, // google client id
      clientSecret: process.env.GOOGLE_CLIENT_SECRET, // google client secret
      // the callback url added while creating the Google auth app on the console
      callbackURL: "http://localhost:8000/auth/google/callback", 
      passReqToCallback: true,
    },

// returns the authenticated email profile
 async function (request, accessToken, refreshToken, profile, done) {
 // you can write some algorithms here based on your application models and all
 // an example - not related to this application

/*
   const exist = await User.findOne({ email: profile["emails"][0].value });
   if (!exist) {
        await User.create({
        email: profile["emails"][0].value,
          fullName: profile["displayName"],
          avatar: profile["photos"][0].value,
          username: profile["name"]["givenName"],
          verified: true,
        });
      }
    const user = await User.findOne({ email: profile["emails"][0].value });
 return done(null, user);
*/
     return done(null, profile);
    }
  )
);

// function to serialize a user/profile object into the session
passport.serializeUser(function (user, done) {
  done(null, user);
});

// function to deserialize a user/profile object into the session
passport.deserializeUser(function (user, done) {
  done(null, user);
});

Update the server.js

The application session(middleware) must be initialized and configured to Google passport. The route for the authentication and the callback needs to be defined.

// server.js

// import necessary depenencies
import "dotenv/config";
import express from "express";
import passport from "passport";
import session from "express-session";
// import passport.js
import "./utils/passport.js"

// intialize app and define the server port
const app = express();
const port = process.env.PORT || 8000;

// a middleware to access json data
app.use(express.json());

// use the session middleware
app.use(
  session({
    secret: process.env.SESSION_SECRET, // session secret
    resave: false,
    saveUninitialized: false,
  })
);

// initialize passport and session
app.use(passport.initialize());
app.use(passport.session());

// a view to check if the server is running properly
app.get("/", (req, res) => {
  res.send(`My Node.JS APP`);
});

// authetication route
app.get(
  "/auth/google",
  passport.authenticate("google", {
    scope: ["email", "profile"],
  })
);

// Call back route
app.get(
  "/auth/google/callback",
  passport.authenticate("google", {
    access_type: "offline",
    scope: ["email", "profile"],
  }),
  (req, res) => {
    if (!req.user) {
      res.status(400).json({ error: "Authentication failed" });
    }
    // return user details
    res.status(200).json(req.user);
  }
);

// a function to start the server  and listen to the port defined
const start = async () => {
  try {
    app.listen(port, () => console.log(`server is running on port ${port}`));
  } catch (error) {
    console.log(error);
  }
};

// call the start function
start();

Test your Node.JS Application

Run server(dev)

Based on the custom configuration in the package.json file, to run the development use the command npm run dev
Authenticate a user

Click this link http://127.0.0.1:8000/auth/google if your server is running on port 8000. You should see a page like the one below.

After selecting an email address, it should return JSON data of that particular account.

Conclusion

I hope you found the article helpful and were able to implement the Google authentication in your Node.js application. You can check out the tutorial source code.

If yes, do well to like and share this piece and comment with me on Linkedin, Twitter and GitHub. And if you like what you read and want to show support, you can buy me coffee😉.

JWT Custom Authentication for Django Application

chryzcode — Mon, 18 Mar 2024 11:53:48 +0000

Introduction

What is the good of a web application without an efficient authentication system?

In this article, I will share how to create a simple and effective authentication for your Django web application.

By default, Django uses the built-in basic authentication system, i.e. (using an email/ username and password to log in directly) how efficient can this be with an Application Programming Interface(API) without tokens for authentication?

Introducing Simple JSON Web Token (JWT) it provides a JSON Web Token authentication backend for a Django REST Framework application.

Let's start building

Create Django Application

Navigate to the directory of your choice on the terminal and create your Django application.

cd desktop # navigated to the desktop directory (windows)
django-admin --version # check if django is installed
django-admin startproject drf_auth_proj # create django project
cd drf_auth_proj # naviage to the project directory
django-admin startapp drf_app # create a django application

Create a Virtual Environment(optional)

virtualenv venv # created a virtual environment named venv
venv\Scripts\activate # activate virtual environment (windows)

Install Necessary Dependencies

pip install djangorestframework # install drf(installs django by default)
pip install djangorestframework-simplejwt # install simple jwt
# add dependencies to requirements.txt file
pip freeze > requirements.txt #make sure the virtulal environment is activated

Application Configuration

We are adding some configurations in the settings.py file(Django app, dependencies).

#settings.py
INSTALLED_APPS = [
    # add 
    'drf_app', # django application
    'rest_framework', # django rest framework
    'rest_framework_simplejwt.token_blacklist', #(JWT BLACKLIST CONFIG)
]

# set simple JWT to default authentication
REST_FRAMEWORK = {
    'DEFAULT_AUTHENTICATION_CLASSES': (
        'rest_framework_simplejwt.authentication.JWTAuthentication',
    )
}

Create a URL file in the application directory(drf_app)

#drf_app/urls.py
from django.urls import path
from rest_framework_simplejwt.views import (
    TokenObtainPairView,
    TokenRefreshView,
) # import views from JWT 

urlpatterns =  [
    #JWT
    path('token/', TokenObtainPairView.as_view(), name='token_obtain_pair'),
    path('token/refresh/', TokenRefreshView.as_view(), name='token_refresh'),
]

Add the application URL to the project URL file

# drf_auth_proj/urls.py
from django.urls import path, include # import include
urlpatterns = [

    path('api/v1/', include('drf_app.urls')), #add app urls file
]

Create customized token-obtain serializer(optional)

This is to add more data/information to a token. The code below adds/encrypts a user's email, and first and last name to the token

# drf_app/views.py

from rest_framework_simplejwt.serializers import TokenObtainPairSerializer

class MyTokenObtainPairSerializer(TokenObtainPairSerializer):
    @classmethod
    def get_token(cls, user):
        token = super().get_token(user)

        # Add custom claims based on your user model fields
        token['email'] = user.email
        token['username'] = user.username
        #token['first_name'] = user.first_name
        # token['last_name'] = user.last_name
        return token

More JWT configurations

# drf_auth_proj/seetings.py

from datetime import timedelta

# Read more abou this on oficial docs
# https://django-rest-framework-simplejwt.readthedocs.io/en/latest/settings.html
SIMPLE_JWT = {
    "ACCESS_TOKEN_LIFETIME": timedelta(minutes=5), 
    "REFRESH_TOKEN_LIFETIME": timedelta(days=30),
    "ROTATE_REFRESH_TOKENS": True,
    "BLACKLIST_AFTER_ROTATION": True,
    "UPDATE_LAST_LOGIN": False,

    "ALGORITHM": "HS256",
    "VERIFYING_KEY": "",
    "AUDIENCE": None,
    "ISSUER": None,
    "JSON_ENCODER": None,
    "JWK_URL": None,
    "LEEWAY": 0,

    "AUTH_HEADER_TYPES": ("Bearer",),
    "AUTH_HEADER_NAME": "HTTP_AUTHORIZATION",
    "USER_ID_FIELD": "id",
    "USER_ID_CLAIM": "user_id",
    "USER_AUTHENTICATION_RULE": "rest_framework_simplejwt.authentication.default_user_authentication_rule",

    "AUTH_TOKEN_CLASSES": ("rest_framework_simplejwt.tokens.AccessToken",),
    "TOKEN_TYPE_CLAIM": "token_type",
    "TOKEN_USER_CLASS": "rest_framework_simplejwt.models.TokenUser",

    "JTI_CLAIM": "jti",

    "SLIDING_TOKEN_REFRESH_EXP_CLAIM": "refresh_exp",
    "SLIDING_TOKEN_LIFETIME": timedelta(minutes=5),
    "SLIDING_TOKEN_REFRESH_LIFETIME": timedelta(days=1),

    "TOKEN_OBTAIN_SERIALIZER": "rest_framework_simplejwt.serializers.TokenObtainPairSerializer",
    "TOKEN_REFRESH_SERIALIZER": "rest_framework_simplejwt.serializers.TokenRefreshSerializer",
    "TOKEN_VERIFY_SERIALIZER": "rest_framework_simplejwt.serializers.TokenVerifySerializer",
    "TOKEN_BLACKLIST_SERIALIZER": "rest_framework_simplejwt.serializers.TokenBlacklistSerializer",
    "SLIDING_TOKEN_OBTAIN_SERIALIZER": "rest_framework_simplejwt.serializers.TokenObtainSlidingSerializer",
    "SLIDING_TOKEN_REFRESH_SERIALIZER": "rest_framework_simplejwt.serializers.TokenRefreshSlidingSerializer",

    #JWT TOKEN CUSTOM(in drf_app.views.py)
    "TOKEN_OBTAIN_SERIALIZER": "drf_app.views.MyTokenObtainPairSerializer",

}

Migrate the changes to the database

# on the terminal
python manage.py migrate

Create Super User
```
python manage.py createsuperuser
```

Check the Admin Page(optional)

If you can remember, we did some token blacklist configuration. It saves all tokens generated and makes sure it is used only once.

Generate Token

Make sure the server is running python manage.py runserver. Then, open the Token Obtain Pair URL on a browser and enter the necessary details(username and password).

Decode the Token Generated

Visit the JSON Web Token website https://jwt.io/ and paste any of your tokens there. The email and username field in the image below results from the custom Token Obtain Pair Serializer views.

Here is the link to the project's repository for reference sake.

Conclusion

I hope you found the article helpful and created a JWT authentication for your Django application.

If yes, do well to like and share this piece and comment with me on Linkedin, Twitter and GitHub. And if you like what you read and want to show support, you can buy me coffee😉.

What is SQL and SQLite?

chryzcode — Fri, 19 Jan 2024 06:23:04 +0000

Why this title/ subject matter?

As a software engineer, I had to use SQLite in a product; I learnt something new and saw the importance of using it because I use PostgreSQL more, and I made a post about it on X(Twitter)

Then, an account replied with a comment seeking clarity about SQL and SQLite.

)

More detailed content would be more insightful because why not?

What is SQL?

Structured Query Language is a programming language designed and utilized for storing, managing, processing and manipulating relational databases. A relational database uses rows and columns to store data in a tabular form. E.g. MySQL, PostgreSQL, SQLite, Microsoft SQL Server, and Oracle Database.

It was initially known as the Structured English Query Language (SEQUEL).

What is SQLite?

SQLite is a C-language library for implementing a small, fast, self-contained, high-reliability, full-featured SQL database engine (according to www.sqlite.org).

It is a library that binds to many programming languages and provides a relational database management system (RDBMS). The entire database is stored in a single disk file.

Differences between SQL and SQLite

There are quite a lot of differences, but a few basic ones will be outlined:

SQL is used by different SQL databases like MySQL, SQLite and PostgreSQL, while SQLite uses SQL
SQL is a query language, while SQLite is a database resource.
SQL is used to query a Relational Database System, while SQLite is a Relational Database Management System.
SQL is a standard for creating relational schema, while SQLite is file-based.

Use Cases of SQL and SQLite

SQL and SQLite might seem to have much in common to the extent of the naming, but they have different use cases. A basic few will be mentioned below:

SQL

Management of relational databases

The Relational Database Management System (RDMS) is software for managing relational databases and was built using the Structured Query Language (SQL). Therefore, the management and maintenance of relational databases can be done with SQL.
Performing basic database operations

SQL uses specific keywords to perform basic database functionalities like Creating, Reading, Updating and Deleting (CRUD) data.
SQL for websites

Most interactive and e-commerce websites, especially those with lots and tons of data, use SQL to fetch, retrieve and store data from their database when needed or required.
Integration with other scripting languages

SQL is integrated and supported by scripting languages like Python and R. Therefore, SQL can be used to manage databases in such languages.

SQLite

Websites

SQLite database engine can be used for storing the data of a website. It has a large bandwidth to operate with high-traffic websites and applications.
Fast and quick server-side database integration

The SQLite database can be the primary or secondary storage engine or database for server-side applications. It is also advisable to use SQLite database for mini-based applications, development environments, and Internal and temporary usage because of its flexibility, simplicity and speed.

Developers report that SQLite is often faster than a client/server SQL database engine.
Experimental SQL language extensions

The simplicity and modular design of the SQLite database make it a good platform for prototyping new, experimental database language features or ideas.

Conclusion

After going through and reading this article, I hope you were able to know and understand more about what SQL and SQLite are, their differences and use cases.

If yes, do well to like and share this piece and comment with me on Linkedin, Twitter and GitHub. And if you like what you read and want to show support, you can buy me coffee😉.

Best Practices For Deploying Django Applications(Production Environment and Hosting)

chryzcode — Tue, 21 Nov 2023 12:30:41 +0000

Introduction

Hosting and deploying projects is a core and important phase of software and web development. What's the joy, happiness and pride in building a project or product and not being able to share it with the world? How can constructive feedback that promotes improvement be gotten? This has been an issue or problem faced by many developers (myself included), especially newbies.

Building a project in a local environment is different from that of a production environment because of a lot of configurations involved which will shared in this article for Django projects using a PostgreSQL database on the Heroku platform.

Why PostgreSQL Database and Heroku?

You might be wondering why a PostgreSQL database and Heroku for deploying and hosting. If you are curious about this continue reading and if not you can skip it.

PostgreSQL Database

The default database for a Django project is SQLite, it is efficient to use in a local environment but not a production environment. These are a few of the reasons that justify the statement aforementioned:

The bandwidth of PostgreSQL is wider and stronger.
There are lots of hosting platforms that offer PostgreSQL database services.
The PostgreSQL database file does not need to be in the project folder or directory before functioning, unlike SQLite. (The Host of the database made it possible).
- Databases should not be exposed/ revealed publicly(i.e. you can not push your database to a version control system. It needs to be hidden and this makes an SQLite database unavailable for deployment.
PostgreSQL is versatile. It can be used in both a local and production environment(Synchronization and unity of data)

There are other options when it comes to the choice of database like Oracle, MySQL, MariaDB etc. but we will not be heading in that direction in this article.

Heroku

There is a need for a hosting and deploying platform and there are quite a variety of them like Render, Fly, BlueHost, AWS, Google Cloud, Railway etc. Below are a few reasons why I chose Heroku despite their change of billing policies which has been for a while now:

The configuration process is not difficult and frustrating.
Heroku has 3 deployment methods, deploying using the Heroku CLI, GitHub(through the website) and Container Registry. In this article, we will be using GitHub which is the easiest of all.
Creation Heroku applications are free but to use addons(resources), like Heroku PostgreSQL you need to add a debit/ credit card and the plans are fair.

Getting started

Prerequisites and Requirements

Here are the prerequisites and requirements to get started deploying your Django application on Heroku:

Our Django Project
An Heroku account(make sure a debit/credit card is attached to the account)
Install the Heroku CLI

Configuring settings for static files

What are static files?

They are files that can not be converted, modified, changed or processed when an application is running. Examples are Javascript and CSS files, images etc. To make Heroku serve these files as they ought to be, the following steps need to be taken.

Install Whitenoise

With a couple of lines of config, WhiteNoise allows your web app to serve static files, making it a self-contained unit that can be deployed anywhere without relying on nginx, Amazon S3 or any other external service. (Especially useful on Heroku, OpenShift and other PaaS providers.)#make sure.
```
#make sure your project virtual environment is activated
pipenv install whitenoise
```

Configuring whitenoise in settings.py file

#setting the directory and root of the staticfiles for accesibility
STATICFILES_DIRS = [os.path.join(BASE_DIR, 'staticfiles')]

#whitenoise configuration
STATICFILES_STORAGE = "whitenoise.storage.CompressedManifestStaticFilesStorage"

Setting Heroku and project up for deployment

To deploy the application, both the Heroku CLI and the website will be used. To verify if you have Heroku installed properly, use the command below;

#check for the version of Heroku installed
heroku -v

#install django-heroku package in your project
pipenv install django_heroku

#install gunicorn
pipenv install gunicorn

# to use postgresql in our django hosted project
pipenv install psycopg2-binary

Create a Procfile in your project

It is a file that specifies the commands that are executed by a Heroku app on startup. Make sure it is located at the base path of the project. Insert the code below into the Procfile
```
# it connects to the wsgi file of the project
web: gunicorn my_project.wsgi --log-file -
```
Create an application on Heroku
Create a Postgres Database on Heroku

While on the newly created application click on the Resources tab search and click Heroku Postgres on Add-ons search bar. Choose your desired plan and click on it after it has been created to see the database credentials details.

Install django-decouple

It is a Python library aimed at making it easier for developers to separate their configuration settings from code
```
pipenv install python_decouple
```

settings.py configuration

#add this at the begining of settings.py file
from decouple import config
import django_heroku

# update these
SECRET_KEY = config('SECRET_KEY')

# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = config("DEBUG", default=False, cast=bool)

# the last value is the url of the Heroku app, remove the https:/
ALLOWED_HOSTS = ["127.0.0.1", "localhost", "my-test-django-project-9a47545252ae.herokuapp.com"]

#update the database from sqlite to postgres
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        "NAME": config("DB_NAME"),
        "USER": config("DB_USER"),
        "PASSWORD": config("DB_PASSWORD"),
        'HOST': config("HOST"),
        'POST': '5432',
    }
}

# Activate Django_heroku.
django_heroku.settings(locals())

Create .env and .gitignore file

A .env file is where the secrets and sensitive keys of a project are kept to prevent exposure especially when pushed to GitHub, GitLab etc. Link to a git ignore file

See to it that both files are located at the base path of the project.
```
DEBUG=True
SECRET_KEY=jferjrjtgotuggfuopfddl #just a sample 
DB_NAME=#database name
DB_USER=#user
DB_PASSWORD=#password
HOST=#the host
```
Delete sqlite database db.sqlite3 and migrate
```
python manage.py migrate
```
Add .env details

Go to the application on the Heroku website and click on settings tab and the Reveal Config Vars button to add all the details in the .env files.

Deployment

As said earlier, we will be deploying using the GitHub method on Heroku, so make sure to push your code to GitHub. Navigate to the deploy tab to connect the application to the GitHub repository. Then, scroll down and click the deploy branch button to start the deployment process.

You should have your hosted project running live on Heroku. Hope you enjoyed the hands-on tutorial-based-article

Build a Custom Functional Django Project

chryzcode — Mon, 13 Nov 2023 07:55:28 +0000

Introduction

In this article, we will build a note project to cover the major scope and aspects of building real-world projects using Django.

Create a virtual environment

A virtual environment is a tool that helps store dependencies for a specific Django application, not on a global scope on your laptop or computer.

#if you have python installed properly, pip should be available automatically
#pip is a python package manager/ installer
pipenv shell
#the command above should create a virtual environment if ran on your terminal in the project path

Installing Django

pipenv install django

After successfully running the command above a Pipfile and Pipefile.lock file will be created automatically.

Pipfile

Pipfile is the file used by the Pipenv virtual environment to manage project dependencies and keep track of each dependency version.
Pipfile.lock

Pipfile.lock leverages the security of package hash validation in pip. This guarantees you're installing the same packages on any network as the one where the lock file was last updated, even on untrusted networks.

Create note models

from django.db import models
#use django user user model
from django.contrib.auth.models import User

# Create your models here.
#Note model
class Note(models.Model):
    user = models.ForeignKey(User, on_delete=models.CASCADE)
    title = models.CharField(max_length=200)
    body = models.TextField(null=True, blank=True)
    updated = models.DateTimeField(auto_now=True)
    created = models.DateTimeField(auto_now_add=True)

    # ordering notes in list view with the just created or updated ones
    class Meta:
        ordering = ['-updated', '-created']

    # if a note object is called 50 characters of the title will shown
    def __str__(self):
        return self.title[0:50]

Add Django auth URLs

Since we will not be creating custom views authentication add the code below to your project my_project URL file

from django.contrib import admin
from django.urls import path, include


urlpatterns = [
    path('admin/', admin.site.urls),
    path('', include('my_app.urls')),
#Django auth default url
    path('', include('django.contrib.auth.urls')),
]

Create migration

The command below creates a file which will have a database schema based on the models in the models.py file in the migration folder.

python manage.py makemigrations

#This should be the response diplayed after running it on the terminal
#Migrations for 'my_app':
  #my_app\migrations\0001_initial.py
    #- Create model Note

Create database

To create the project database run the command below. This command also creates necessary tables like notes and schemas like user , sessions etc.

python manage.py migrate

#and you can run the server
python manage.py runserver

Create a superuser

In a Django project, there are superusers and normal users and the major difference between them is that superusers have access and permission to the project admin page to manage the project's models and data.

python manage.py createsuperuser

Add notes model to Django admin

To access your notes models and play around it using the Create, Read, Update, Delete (crud) functions on the Django admin, the notes model needs to be registered on the admin.py file.

from django.contrib import admin
from .models import Note

# Register your models here.
admin.site.register(Note)

To have a view of the Django admin page make sure the server is up and running and open this link on the browser http://127.0.0.1:8000/admin/. You can only log in with a superuser account.

The CRUD functions can now be done on the Notes models.

Create custom application interfaces and functions

Instead of using the Django admin page, a custom interface and functions for the project will be created.

Create forms(forms.py)

The file forms.py will be created customarily in the application folder my_app

from django import forms
from .models import Note

#create a form based on the Note model
class noteForm(forms.ModelForm):
    class Meta:
        model = Note
        #form fields/inputs that will be shown
        fields = ['title', 'body']
        #added a css class and html placeholder to the form inputs
        #the form-control css class is from bootstrap
        widgets={
                'title':forms.TextInput(attrs={'class':'form-control', 'placeholder':'Note title'}),
                'body':forms.Textarea(attrs={'class':'form-control',  'placeholder':'Start writing...'}),
        }

Create the project functions(views.py)

from django.shortcuts import render, redirect
from .models import Note
#import the noteform
from .forms import noteForm
from django.contrib.auth.decorators import login_required
from django.contrib.auth.models import User

# Create your views here.

#login_required makes sure a user is authenticated before a view

#get all the notes of the currently logged in user
@login_required(login_url='login')
def myNotesList(request):
    context = {}
    notes = Note.objects.filter(user=request.user)
    context['notes'] = notes
    return render(request, 'note-list.html', context)

#add a new note
@login_required(login_url='login')
def addNote(request):
    context = {}
    form = noteForm
    if request.method == 'POST':
        note = Note.objects.create(
            user = request.user,
            title = request.POST.get('title'),
            body = request.POST.get('body')
        )
        return redirect('view-note', pk=note.pk)
    context['form'] = form
    return render(request, 'add-note.html', context)

#edit a specific note
@login_required(login_url='login')
def editNote(request, pk):
    note = Note.objects.get(pk = pk)
    if request.user.id == note.user.id:
        form = noteForm(instance=note)
        if request.method == 'POST':
            form = noteForm(request.POST, instance=note)
            if form.is_valid():
                form.save()
                return redirect('view-note', pk=note.pk)
        context = {'form':form}
        return render(request, 'edit-note.html', context)
    return redirect('notes-list')

#view a specific note
@login_required(login_url='login')
def viewNote(request, pk):
    context = {}
    note = Note.objects.get(pk = pk)
    if request.user.id == note.user.id:
        context['note'] = note
        return render(request, 'view-note.html', context)
    return redirect('notes-list')

#delete a specific note
@login_required(login_url='login')
def deleteNote(request, pk):
    note = Note.objects.get(pk=pk)
    if request.user.id == note.user.id:
        note.delete()
        return redirect('notes-list')
    return redirect('notes-list')

##delete the logged in user account
@login_required(login_url='login')
def deleteUser(request):
    user = request.user
    user.delete()
    return redirect('login')

Create URLs patterns

#urls.py
from django.urls import path
from . import views

urlpatterns =[
    path('', views.myNotesList, name='notes-list'),
    path('add-note/', views.addNote, name='add-note'),
    path('edit-note/<str:pk>/', views.editNote, name='edit-note'),
    path('view-note/<str:pk>/', views.viewNote, name='view-note'),
    path('delete-note/<str:pk>/', views.deleteNote, name='delete-note'),
    path('delete-user/', views.deleteUser, name='delete-user'),
]

Add the Login and Logout Redirect URL

This piece of code is to be in the settings.py and its purpose is to redirect a user to a certain URL/page after logging in or logging out.

#settings.py
import os

#where your static files will be located e.g css, ja and images
STATIC_ROOT = os.path.join(BASE_DIR, 'static')

LOGIN_REDIRECT_URL = "notes-list"
#we can access the login url through the Django defauly auth urls
LOGOUT_REDIRECT_URL = "login"

Create template and static files(`.html` and `.css`)

This should be an overview of how the static and template files should be arranged.

Create styles.css file

This is the file where we write the stylings of the app(CSS).

Create a static folder in the application folder my_app

Inside the static folder create a css folder

Then create the styles.css file inside the css folder.

* {
  margin: 0;
  padding: 0;
}

body {
  font-family: "Montserrat", sans-serif;
  background-color: #1f2124;
  color: white;
}
.fa.fa-google {
  margin-right: 7px;
}

.page-header {
  text-align: center;
  font-size: 30px;
  font-weight: 600;
  color: orange;
}

.login-page-container {
  margin-top: 40px;
}

.login-container {
  border: 3px solid #252629;
  border-radius: 8px;
  margin-left: auto;
  margin-right: auto;
  margin-top: 50px;
  width: 30%;
  height: 40vh;
}

.login-container > p {
  text-align: center;
  font-size: 22px;
  margin-top: 15px;
}

.login-btn {
  margin-left: auto;
  margin-right: auto;
  display: block;
  width: 65%;
  margin-top: 70px;
}

.login-btn:hover {
  background-color: #252629;
}

.notes-container {
  margin-left: auto;
  margin-right: auto;
  margin-top: 60px;
  display: block;
  border: 3px solid #252629;
  width: 500px;
  border-radius: 8px;
}

.note-title {
  margin-left: 15px;
  font-size: 18px;
  padding: 15px;
  color: white;
  text-decoration: none;
}

.note-title > a {
  margin-left: 15px;
  font-size: 18px;
  color: white;
  text-decoration: none;
}

.note-title > a:hover {
  margin-left: 10px;
  color: orange;
  text-decoration: none;
}

.note-header {
  margin-top: 20px;
  padding-left: 30px;
  padding-right: 50px;
  color: orange;
  border-bottom: 3px solid orange;
}

.note-header > a {
  text-decoration: none;
  color: orange;
}

.note-logo {
  margin-right: 15px;
}

.note-count {
  font-size: 19px;
  float: right;
  padding-right: 10px;
}

.note-created {
  display: flex;
  justify-content: right;
  align-items: center;
  font-size: 10px;
}

.note-list {
  border-bottom: 1px solid #252629;
  height: 60px;
}

.note-list:hover {
  background-color: #252629;
}

#add-note-icon {
  display: flex;
  justify-content: right;
  align-items: center;
  font-size: 30px;
}

label {
  display: none;
}

.a-note-title {
  color: orange;
  text-align: center;
  padding: 10px;
}

.btn.btn-secondary {
  justify-content: right;
  align-items: center;
  background-color: rgba(255, 166, 0, 0.576);
}

#back-to-notes-list {
  font-size: 30px;
  /* padding: 8px; */
}

.note-detail-header {
  padding-left: 10px;
  padding-right: 10px;
}

.icons > a {
  color: white;
  text-decoration: none;
}

.note-detail-header > a {
  color: white;
  text-decoration: none;
}

.icons {
  display: flex;
  align-items: center;
  justify-content: right;
}

#edit-note {
  font-size: 20px;
}

#delete-note {
  font-size: 20px;
}

.note-body {
  padding: 10px;
  margin-left: 7px;
  margin-right: 7px;
  display: block;
}

.markdown-support-note {
  margin-top: 10px;
  color: orange;
  text-align: center;
}

.dev-info {
  display: flex;
  justify-content: space-between;
  align-items: center;
  margin-top: 100px;
  font-size: 25px;
  color: orange;
}

.dev-info > span > a {
  text-decoration: underline !important;
  color: orange !important;
}

.dev-info > a > i {
  font-size: 40px;
  color: orange;
}

@media all and (min-width: 100px) and (max-width: 600px) {
  .notes-container {
    width: 100%;
  }

  .login-container {
    width: 100%;
  }
}

Create the custom views template

Create a template folder in the application folder my_app and

Create registration folder

Create login.html file

{% extends 'base.html'%}
{% load static %}
<title>{% block title %}Sign In || My Note App{% endblock %}</title>
{% block content %}

{% if user.is_authenticated %}
<p></p>
{% else %}
<div class="login-page-container">
    <h3 class="page-header" >Sign In</h3>
    <div class="login-container">
        <p>My Notes App</p>
        {% if not user.is_authenticated %}
    {% if messages %} {% for message in messages %}
    <div class="text-center alert alert-{{ message.tags }}">{{ message|safe }}</div>
    {% endfor %} {% endif %} {% if form.errors %}
    <p style="font-size: 20px; margin-left: 25px">Your username and password did not match. Please try again.</p>
    {% endif %}
    <form action="" method="post">
      {% csrf_token %}
      <div class="mb-3">
        <label id="login-label" for="username" class="form-label">Username:</label>
        <input id="login-form" type="text" class="form-control" name="username" id="username" placeholder="Username" />
      </div>
      <div class="mb-3">
        <label id="login-label" for="password" class="form-label">Password:</label>
        <input
          id="login-form"
          type="password"
          class="form-control"
          name="password"
          id="password"
          placeholder="Password"
        />
      </div>
      <div id="login-btn">
        <input class="btn btn-secondary" type="submit" value="Login" />
      </div>
    </form>
    <h5 style="text-align: center; bottom: -50px; position: relative;">
      Yet to have an account? <a href="register/">Sign up</a>
    </h5>
    {% else %}
    <br /><br />
    <p style="font-size: 30px; text-align: center; font-weight: 600">You are already logged in.</p>
    {% endif %}
    </div>
</div>
<p class="markdown-support-note">markdown supported</p>
{% endif %}
{% endblock %}

Create add-note.html

{% extends 'base.html' %}
<title>{% block title %}Add note || My Note App{% endblock %}</title>
{% block content %}

<div class="notes-container">
  <h4 class="a-note-title">Add a note</h4>
  <form method="POST">
    {% csrf_token %} {{ form.as_p }}
    <button class="btn btn-secondary sm" type="submit">save</button>
  </form>
</div>

{% endblock %}

Create base.html

This file contains the <head>base.html</head> and the nav of all the template pages

{% load static %}
<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <link href="https://unpkg.com/tailwindcss@^2/dist/tailwind.min.css" rel="stylesheet" />
    <link
      href="https://cdn.jsdelivr.net/npm/bootstrap@5.1.3/dist/css/bootstrap.min.css"
      rel="stylesheet"
      integrity="sha384-1BmE4kWBq78iYhFldvKuhfTAU6auU8tT94WrHftjDbrCEXSU1oBoqyl2QvZ6jIW3"
      crossorigin="anonymous"
    />
    <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.7.0/css/font-awesome.min.css">
    <link rel="stylesheet" href="https://fonts.googleapis.com/icon?family=Material+Icons">
    <link href="https://fonts.googleapis.com/css2?family=Montserrat&display=swap" rel="stylesheet" />
    <link rel="stylesheet" href="{% static 'css/styles.css' %}" />
   <title>{% block title %}My Note App{% endblock %}</title>
  </head>
  <body>
    <nav class="navbar navbar-expand-lg navbar-dark bg-dark">
  <div class="container-fluid">
    <span class="navbar-brand" style="color: orange;">&#9782; My Note App</span>
    <button class="navbar-toggler" type="button" data-bs-toggle="collapse" data-bs-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
      <span class="navbar-toggler-icon"></span>
    </button>
    <div class="collapse navbar-collapse" id="navbarSupportedContent">

    {% if user.is_authenticated %}

      <ul class="navbar-nav me-auto mb-2 mb-lg-0"">
        <li class="nav-item"><a class="nav-link" href="{% url 'add-note' %}">Add a note</a></li>
      </ul>
      {% else %}
      <ul class="navbar-nav me-auto mb-2 mb-lg-0"">
        <li class="nav-item"><a class="nav-link" href="{% url 'add-note' %}">Add a note</a></li>
      </ul>
{% endif %}

  {% if user.is_authenticated %}
  <div class="btn-group">
    <ul class="nav justify-content-end">
      <li class="nav-item dropdown" >
        <a style="color: orange;" class="nav-link dropdown-toggle" class="nav-link" href="#" id="navbarDropdown" role="button" data-bs-toggle="dropdown" aria-expanded="false">
          {{ request.user }}
        </a>
        <ul class="dropdown-menu bg-dark" aria-labelledby="navbarDropdown">
          <li class="nav-item"><a style="color: orange;" class="nav-link" href="{% url 'logout' %}">Log out</a></li>
          <li><a style="color: orange;" class="dropdown-item" href="{% url 'delete-user' %}">Delete acc</a></li>
      </ul>
  </div>
  {% endif %}
    </div>
</nav>

    <div class="container">{% block content %} {% endblock %}
      <script src="https://cdn.jsdelivr.net/npm/bootstrap@5.0.1/dist/js/bootstrap.bundle.min.js" integrity="sha384-gtEjrD/SeCtmISkJkNUaaKMoLD0//ElJ19smozuHV6z3Iehds+3Ulb9Bn9Plx0x4" crossorigin="anonymous"></script>
    </div>
  </body>
</html>

Create edit-note.html

{% extends 'base.html' %}
<title>{% block title %}Edit note|| My Note App{% endblock %}</title>
{% block content %}

<div class="notes-container">
    <h4 class="a-note-title" >Edit note</h4>
    <form method="POST">
    {% csrf_token %}
    {{ form.as_p }}
    <button class="btn btn-secondary sm" type="submit">save</button>
</form>
</div>

{% endblock %}

Create note-list.html

{% extends 'base.html' %}
<title>{% block title %}Note list || My Note App{% endblock %}</title>
{% block content %}
<div class="notes-container">
  <div class="note-header">
    <h5>
      <span class="note-logo">&#9782;</span>{{request.user}}'s note
      <span class="note-count">{{notes.count}}</span>
    </h5>
    <a href="{% url 'add-note' %}"><span id="add-note-icon" class="material-icons">add_circle</span></a>
  </div>
  {% for note in notes %}
  <div class="note-list">
    <div class="note-title">
      <a href="{% url 'view-note' note.pk %}">{{ note }}</a>
      <small class="note-created">{{note.created}}</small>
    </div>
  </div>

  {% endfor %}
</div>
{% endblock %}

Create view-note.html

{% extends 'base.html' %}
<title>{% block title %}Note || My Note App{% endblock %}</title>
{% block content %}
<div class="notes-container">
  <div class="note-detail-header">
    <a href="{% url 'notes-list' %}"><span id="back-to-notes-list" class="material-icons">chevron_left</span></a>
    <span class="icons">
      <a href="{% url 'edit-note' note.pk%}"><span id="edit-note" class="material-icons">edit</span></a>
      <a href="{% url 'delete-note' note.pk%}"><span id="delete-note" class="material-icons"> delete </span></a>
    </span>
  </div>
  <h4 class="a-note-title">{{note}}</h4>
  <div class="note-body">{{ note.body}}</div>
</div>

{% endblock %}

Our note application should be up, running and functional. I hope you enjoyed reading this article and building along side.

Building a Custom Django Project/ Application(Folders and Files 2)

chryzcode — Sat, 04 Nov 2023 03:02:44 +0000

Introduction

What is a project or application that is not custom-made and per the intent of the developer or client?

A dream project or application is primarily engineered from the thought or what can be called inspiration.

In addition, this article will convey custom-made Django folders and files, the continuation of The Django Project Folder and Files Structure and Architecture 1.

Building Custom Django Application

Create an application URL file

Firstly, a urls.py file needs to be created in the Django application my_app because it is not always provided by Django(default).

The code below gives the file its full potential apart from its name; without it, the Django server will throw an error. The code below helps to initiate the process of creating the URL pattern of the application.

#the application url file
from django.urls import path

urlpatterns = [

]

Connect the application URL file to the project URL file.

In the project folder, my_project there is a URL file that needs to connect with the URL file of all the project's applications.

from django.contrib import admin
from django.urls import path, include


urlpatterns = [
    path('admin/', admin.site.urls),
#added the application url as the project base url pattern
    path('', include('my_app.urls')), 
]

include needs to be imported, and can be called the connection conjunction key in this case.

Connect the app to the project.

All Django applications need to be connected to the project, done in the settings file. settings.py

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',

    # put the name of the application
    'my_app',
]

Create our first Django web page.

We will override the default Django web page to a simple custom-based one. These changes will be made through the application views.py and urls.py files.

#the application urls.py file
from django.urls import path
#import everything in the application views file
from my_app.views import *

urlpatterns = [
    #base url pattern connected to customHomePage view
    #you can name a url path using the python keyword name
    path('', customHomePage, name='custom_home_page')
]

Django custom view using HttpResponse

The view returns an HttpResponse object that contains the generated response. Each view function is responsible for returning an HttpResponse object.

#the application views.py file
from django.shortcuts import render
#import HttpResponse
from django.http import HttpResponse

#a simple function-based function
def customHomePage(request):
    #HttpResonse is to render contents/ strings on the page
    return HttpResponse('My Django application custom home page')

Django custom view using Template

This requires creating an HTML template file and connecting to the view to render content on the web page.

def customHomePage(request):
    # return render(request, template_name='customHomePage.html')
    return render(request, 'customHomePage.html')

Your Django server http://127.0.0.1:8000/ should display the page shown below.

I hope you created your Django custom-made project with less difficulty and ease.

The Django Project Folder and Files Structure and Architecture 1

chryzcode — Fri, 27 Oct 2023 21:33:23 +0000

Introduction

Django is an extensive framework of the language Python. Therefore, on this note, it should not be surprising that a Django project will include many significant files and folders if needed. This article will address the default folders and files provided by Django.

You might want to read this article of mine, How to Create a Django Project and Project, for the project sample that will be examined in this article or, better still, clone it from the GitHub website Django sample project.

Overview of the Django Project

Image of the Project

Below is an image of the Django project and application. Specifically, this is the project's image from a code editor, VSCode.

The Application Folder(`my_app`)

This is a Django application folder, and it is possible to have more than a Django application based on the project's scope, but our sample project is one application-based. It hosts all the files and folders related to the application and even more, if the files are imported into other applications.

Migrations folder

This is the folder where the schema files for creating and updating a database are kept. This is the command for making migrations. python manage.py makemigrations
- The Init file __init__.py
  
  An experienced Python developer should know the work of this file. It is a Python file that marks a directory as a Python package. It is used to initialize the package when imported(i.e., it helps connect files from a Django folder/ application to one another).
  
  This file is for displaying your models on the project admin page. It is always in a Django application by default. It makes it easy for one to test models using a page/form created by the Django framework.
The Admin file admin.py

This file is for displaying your models on the project admin page. It is always in a Django application by default. It makes it easy for one to test models using a page/form created by the Django framework.
The Models file models.py

This is where you carve the tables and schema for the project and application database.
The Tests file tests.py

For writing tests, check for errors and see that the codes run perfectly.

Check one of my articles on Why Write Tests?
The Views file views.py

The File for the application functions and rendering/ displaying data content on the project web application.

The Project Folder(`my_project`)

The folder that hosts files and folders related to the project in general. All applications in the project must be connected to this folder.

The Pycache folder __pycache__

This folder consists of interpreted and optimized versions of codes. When executed, it converts to bytecode and ends with either a .pyc or .pyo extension.
The Asynchronous Server Gateway Interface (ASGI) file asgi.py

It is for sending requests to asynchronous-capable Python programming language frameworks and applications. It allows developers to write web applications that can handle multiple requests at once without blocking the main thread.
The settings file settings.py

This file is where the major configuration of the project and application resides; without it, the project will be unable to run.
The URLS file urls.py

This file connects all the URL files in the application folder together. It provides different and unique major URL patterns (parameters) for the project.
The Web Server Gateway Interface (WSGI) file wsgi.py

It is a mediator responsible for communicating between a web server and a Python web application. It explains how the web server communicates with the app and how it can be chained to process a request.
The SQLite Database.

The database of the project. It is where the database schema is located.
The Manage file manage.py

This file powers and engineers Django-based commands when written from the command line(e.g., python manage.py runserver , python manage.py createsuperuser ).

I hope you were able to understand more about the Django project and application folder and file, structure and architecture.

[Continuation...]

How to Create a Django Project and Application.

chryzcode — Fri, 20 Oct 2023 23:29:53 +0000

Introduction.

This article is a guide on how to create your (first) Django project and application.

A Little about Django.

The web framework for perfectionists with deadlines.

Django is a high-level Python web framework that encourages rapid development and clean, pragmatic design. Built by experienced developers, it takes care of much of the hassle of web development, so you can focus on writing your app without needing to reinvent the wheel. It’s free and open source.

Copied from the Django official website.

Interested in reading a little about Python, check out one of my articles The Language: Python.

Get started

Prerequisites and Requirements

Here are the prerequisites and requirements to get started in creating your Django application :

Install Python.

Paste this command on your terminal. The command below is to verify if you have Python properly installed. It should provide the version of Python you installed.
```
python --version
```
Install Django.

The same thing applies to Django.
```
django-admin --version
```

Create a Django Project.

I believe you have Python and Django installed. Navigate to the directory of your choice on your terminal, desktop is my choice of directory for my new Django project

cd desktop

and paste the command below to create your project, my_project is the name of my Django project.

django-admin startproject my_project

To check the files or folders in your project

#for MacOS
ls my_project

#for Windows
dir my_project

Create a Django Application

To create your Django application copy and paste the command below on your terminal to change your directory to the newly created project.

#to change directory
cd my_project

#to create a django app
django-admin startapp my_app

Note: It is possible to have multiple applications based on the scope of the project.

Run and Test the Django Project

To check if our Django project is up and running, paste this code below on your terminal. The command is to run your server(app/project) locally.

python manage.py runserver

If the image below is the same as yours, then you have successfully created your Django project/ application.

The warning about unapplied migrations is related to the database which will not be covered.

To view the locally hosted project on your browser, click this link http://127.0.0.1:8000/. This is the local server link to your Django application and this should be the view of your project on the browser.

Congratulations on creating your (first) Django application.

Continuation...

Machine Learning Pipeline Architecture

chryzcode — Mon, 24 Apr 2023 08:24:43 +0000

Introduction.

In this article, a lot will be shared in the context and regards to machine learning pipeline architecture.

In our world, today, with how data is important and accessible, training a machine-learning model is highly possible. The presence of machine learning cannot be undermined especially in the field of automation, detection, prediction and technological assistance.

The creation and application of machine learning and models vary depending on the need and intended solutions. A view to machine learning (Models on Production) i.e(models on production environment) is administered through a certain infrastructure, Machine Learning Pipelines which we are going to explore today.

What is Machine Learning?

Machine learning is a subdivision/ subset of data science, a field of knowledge studying the extraction of value and meaningful insight from data. Meanwhile, machine learning suggests techniques and systems that train algorithms and programs on data to solve problems and make decisions with no or minimal rules, programming patterns and human intervention.

What is a Machine Learning Pipeline?

A machine learning pipeline is a technical infrastructure used to administer and automate machine learning processes and workflow, formatting raw data to intended output and valuable information.

Benefits of Machine Learning Pipeline Architecture.

Here are some of the benefits of machine learning pipeline architecture:

Flexibility

It is possible to make over workflows without changing the rest and other parts of the system (computation units and components) for better implementation.

Extensibility

It is easy and intuitive to create new functionalities, processes and components when the system is segregated into pieces.

Scalability

The availability of component/ computation segregation and separation provides the ability to scale if there is an issue. Each part of the computation is presented through a standard interface.

Bugs Prevention

Automated pipelines can prevent bugs. Manual machine learning workflow bugs might be really difficult to debug since inference of the model is still possible, but simply incorrect. With automated workflows, these errors can be prevented.

Less Cost

It reduces the expenditure and costs of data science projects and products.

Less Consumption of Time

It frees up development time for data scientists and increases their job satisfaction and experience. This improves efficiency and reduces the time spent getting set up on a new project and processes to update existing models.

Why does Machine Learning Pipeline Architecture Matter?

As algorithms start and begin to aid and enable machines to learn through data, it tends to be beneficial to both individuals and organisations in various aspects.

Below are a few reasons why machine learning pipeline architecture matter:

Timely Analysis And Assessment

It helps to understand and come up with strategic options and alternatives by analysing and assessing real-time data of the same or related environment.

Real-Time Predictions

Machine learning algorithms have been so beneficial to businesses by the provision of real-time predictions which tends to be closely accurate if not aiding in decisions making, implementation/ administration etc.

Transformation of Industries

Machine learning has led to the transformation of industries with its ability and expertise to provide valuable insights in Real-Time environments and situations.

Adoption and When to Use Machine Learning Pipeline Architecture.

There are a lot of advantages provided by the machine learning pipeline but it is not to be used in every data science product or project, it depends on the intended purpose and how vast it is.

However, situations or circumstances whereby continuous updating of models requires fine-tuning, e.g.(models with real-time data, especially users or been used in software or an application).

Pipelines have also become very much essential as machine learning projects and products grow. If the dataset or resource requirements are large, it allows for easy infrastructure scaling. If repeatability is important, this is provided through the automation and the audit trail of machine learning pipelines.

Adoption of Machine Learning Pipeline.

As aforementioned, industries, companies and products/ projects with massive amounts of data tend to use the machine learning pipeline as it helps in the fast, easy and efficient implementation of tasks.

Here are a few industries that have adopted the technologies of the machine learning pipeline:

Financial services

Businesses and financial industries and companies use machine learning technology to discover important insights into raw data and information. It is also used to prevent cyber attacks and fraudulent activities through detection, alert and cyber surveillance.

Government

Collecting/creating, processing, storage, transmission and control of national data is a huge task and especially protection and public safety are of importance. Machine learning also helps in the efficiency of the various sector by mining multiple data sources for insights.

Healthcare

In the healthcare field, machine learning technologies have helped in the analysation of data and information to diagnose medical illnesses and improve scientific treatment patterns.

Mining Industry

Machine learning technologies have helped the mining industry by sourcing and analyzing resources (energy sources, minerals etc.). It has also made the process more efficient, cost-effective and less time-consuming.

Machine Learning Pipeline Architecture and Stages.

A machine learning pipeline comprises several stages. Data is processed in all stages for the cycle to run, and it is transmitted from one stage to the other. i.e., the output of a processing unit supplied as an input to the next step. There are different stages but we are checking out the four main and major stages Pre-processing, Learning, Evaluation, and Prediction.

Pre-processing

Data processing is a process of basic transformation of data. Transforming raw data collected from users into an understandable and consumable format for the model. The outcome product of data pre-processing is the final dataset used for training the model and testing purposes.

Learning

This process involves the extraction of the pre-processing output result(model understandable format) for the appropriate application in a new setting or circumstances. The aim is to utilize a system for a specific input-output transformation task.

Evaluation

This involves assessing the performance of the model using the test subset of data to understand prediction accuracy. The predictive implementation of a model is evaluated by comparing predictions on the evaluation dataset with true values using a variety of metrics.

Prediction

The model's performance to determine the outcomes of the test data set was not used for any training or cross-validation activities. The best model on the evaluation subset is selected to make predictions on future/new instances.

Machine Learning Pipeline Infrastructure/ Model Preparation Process.

Machine learning Infrastructure consists of the resources, processes, and tooling essential to the operation, training, development and deployment of machine learning models. Every stage of its workflow is supported by machine learning infrastructure and is the base of its model. There is no specific infrastructure because it depends on the model available in a product or project.

Here are some of the major components of machine learning pipeline architecture:

Model Selection

Model selection refers to the process of choosing the model that best generalizes for a specific task or different data. It includes accuracy, interpretability, complexity, training time, scalability, and trade-offs.

Data Ingestion

This refers to the process of extracting and transferring large data in an automated way from multiple sources.

Model Testing

Model testing refers to the process where the performance of a fully trained model is evaluated on a testing set. It involves explicit checks for behaviours that are expected of the model.

Model Training

It is the process of feeding a machine learning algorithm with data to help identify and learn good values for all attributes involved.

Visualisation and Monitoring

It refers to the process of tracking and understanding the behaviour of a deployed model to analyze performance.

Machine Learning Inference

Machine learning inference is the process of running live data into a machine learning algorithm to calculate output such as a single numerical score.

Model Deployment

Model deployment is the process of implementing a fully functioning machine learning model into production where it can make predictions based on data.

Software/ Applications for Building Machine Learning Pipelines.

Azure Machine Learning Pipelines

Azure ML pipeline helps to build, manage, and optimize its workflows. It is an independently deployable workflow of a complete ML task.

Google ML Kit.

Deploying models in the mobile application(Andriod and IOS) via API, there is the ability to use the Firebase platform to leverage ML pipelines and close integration with the Google AI platform.

Amazon SageMaker

It builds, trains, and deploys machine learning models for any use case with fully managed infrastructure, tools, and workflows. One of the key features is that you can automate the process of feedback about model prediction via Amazon Augmented AI.

Kubeflow Pipelines

Kubeflow Pipelines is a platform for building and deploying portable, scalable machine learning (ML) workflows based on Docker containers.

TensorFlow

TensorFlow is a free, open-source and end-to-end(E23) platform software library for machine learning developed by Google. It makes it easy for you to build and deploy ML models.

Machine Learning Pipeline Tools.

Data Obtainment

Database: PostgreSQL, DynamoDB.

Distributed Storage: Apache Spark/Apache Flink.

Data Scrubbing / Cleaning

Scripting Language: SAS, Python, and R.

Processing in a Distributed manner: MapReduce/ Spark, Hadoop.

Data Wrangling Tools: R, Python Pandas.

Data Exploration / Visualization

Python, R, Matlab, and Weka.

Data Predictions

Machine Learning algorithms: Supervised, Unsupervised, Reinforcement, Semi-Supervised, and Semi-unsupervised learning.

Important libraries: Python (Scikit learn) / R (CARET).

Result Interpretation

Data Visualization Tools: ggplot, Seaborn, D3.JS, Matplotlib, Tableau.

Message Broker: Memphis.Dev

chryzcode — Mon, 03 Apr 2023 16:20:38 +0000

Firstly, what is a message broker?

A message broker is an architecture pattern, software or server that facilitates, provides and enables the connection, communication and exchange of information between applications, systems and services for message validation, transformation, and routing. It translates information and messages between interdependent applications even if there is a contrast in the language or executed and administered platforms.

What is Memphis.Dev?

You might be hearing this for the first time but Memphis{dev} is a product and an example of a message broker.

Memphis is an open-source application that enables and promotes excellent engagement, processing and communication(end-to-end support) with data, data-driven applications and streamlined pipelines.

A cloud-native message broker with an ecosystem that enables less cost-effective and fast/ rapid development of modern queue-based use classes that require large volumes of streamed and enriched data, modern protocols and zero ops for data-oriented developers and data engineers.

Why Memphis.Dev?

Opensource
Memphis is an open-source product that enables the public(people) to contribute to its development. This serves as an advantage as end users can use their experience to bring about improvements, evolvement and enhancement for the software/ service. It also permits an active and strong community that will yield an intuitive process while using.
Support of Major Programming Languages
Memphis supports major programming languages which increases the privilege and opportunity for more usage among data engineers.
Memphis Software Development Kit (SDK) includes Python, Go, Node.js, Typescript, Nest.JS.
Easy and Fast to Use
Memphis provides a higher level of ease of use, while other message brokers like Kafka, RabbitMQ, NATS, and other MQs are HARD to deploy, manage, secure, update, onboard, and tune. Memphis has been designed to be deployed as production-ready in 3 minutes. It also has Graphic User Interface (GUI) and Command Line Interface (CLI) for the users to choose from based on their preferences.
Accurate Real Time Processing
Memphis processing engine provides true real-time processing with a full cover for all three needed layers -
Ingestion, transformation, enrichment.
Schema Support
Schemaverse is a Schema management with versioning, GitOps, validation, enforcement, and zero trust. Schema management with support for Avro, Protobuf, JSON, and GraphQL.
Less Expensive/ Cost
Cost optimization and efficiency are one of Memphis' main building blocks. Implementation costs are close to zero due to the self-optimization that takes place when Memphis gets deployed, and SDKs and client connectivity are built in a low-code manner.

Get started with using Memphis

Introducing CVBuild 🎉

chryzcode — Sat, 11 Mar 2023 12:16:01 +0000

CVBuild is a platform where you can build your resume and online portfolio for free.

Check out for more information here https://cvbuild.onrender.com/.

3 Simple Methods To Do Reverse Email Lookup

chryzcode — Thu, 15 Dec 2022 15:00:05 +0000

We are in a world where data is vital, in one way or another other you consumed/used, created, or accessed some form of data. It is very pertinent any data accessed needs to be accurate, accessible quickly and straightforward.

In this article, we'll learn one of the best ways to source accurate data using emails. Emails are very unique and highly difficult to manipulate, imagine getting desired information about a company through its official email, how secure and accurate can that be?

This is where reverse email lookup comes in, proceeding we will know what it is all about, how it can be used and a detailed guide on starting.

What is a reverse email lookup?

A reverse email lookup is a data enrichment process used to uncover more information relating to an email address. A process that allows getting a lead's email address based on their data

Examples:

Finding the owner of the email address
The country they are based in
Job information and social media accounts associated with the email
Phone number, physical address, profile pictures and many more
The staff associated with the email

What is Reverse Email Lookup used for?

There is no limitation to sourcing information from performing a reverse email lookup, a lot can be achieved. In this section, I'll break down the different use cases where you may require data and information about an individual or entity using their personal or work emails.

Reverse Lookup of Personal & Generic Emails

The information available through a reverse lookup of personal emails is usually based on whatever platforms, social media accounts, and so on that the email is used on, be it their names, location, date of birth, educational background, connections and more. Below are some popular cases of reverse email lookup using personal or generic emails.

Scams and Fraud Prevention

Reverse Email Lookup prevents scams and fraud due to verification of whether the email sender is legit or not. It brings about security.

Know One Personality Better

An email search can provide more information about a person, like interests, and social media handles. It provides information that makes one prepared, influencing the rate of confidence, conversations and bonding.

Verification of Economics Transactions

In a world where online transaction (cashless policy) and E-commerce shopping is on the high, verifying the record of customers, buyers, and transactions goes a long to curbing scams and loss to the minimum.

Reverse Lookup of Work and Professional Emails

Another valuable use case where a reverse lookup can be beneficial is verifying work and professional emails. In the same way that personal emails can tell you a lot about a person's professional data like past and present jobs, skills and capabilities, location and more. These details can be used to determine if the person is a good fit for a particular job

HR Talent Identification

The hiring process will be less stressful, faster and more effective for HR by screening candidates through personal information using a simple email search.

Lead Generation

Another great use case is in sales and lead generation. Using a reverse email lookup, a sales team can qualify leads accurately, thus cutting down time wasted work on leads and instead, focusing on getting sales quicker and more efficiently.

Credit Risks Analysis

Obtaining the record of a customer using reverse email lookup before any online transaction can help identify the risk level for companies investing or loaning.

How Can I Do a Reverse Email Lookup? - 3 Simple Methods

To understand what Reverse Email Lookup is all about, let's look into diverse ways in which this can be processed. There are major ways to be covered, ranging from free to paid to a highly-efficient, automated method. It is up to you to choose which one suits you well.

FREE Reverse Email Lookup (Google & LinkedIn Search)

The most basic and straightforward way to perform a reverse email lookup is to use Google. As you would do a normal Google search, you can do the same with email addresses. All you have to do is simply input your email in the search bar, and it will return search results with information related to that email.

However, this method is very manual and can be time-consuming if you happen to be dealing with a large number of email addresses. Most importantly, it's not guaranteed to return an accurate result as an email might not be crawled successfully by Google.

Another great free alternative to Google is LinkedIn. LinkedIn is a de-facto platform to get information about the professional profile of a person. However, you can‚Äôt perform a reverse email lookup directly on LinkedIn. But you can derive the person's name and company through its email prefix and domain, which you can then use to perform the search according to these two queries.

Paid Reverse Email Lookup Tools

If you want a more efficient and reliable solution to handle large datasets of email addresses then you may want to consider going for a paid reverse email lookup tool. With paid tools, you can quickly gather the data you need which are all scattered online and present them in a manner easy to use. In addition, you can do searches more quickly and accurately, with all the information properly presented in a dashboard or sheet, ready for you to use and act upon.

There are a lot of paid reverse email lookup tools and services available, so it's up to you to choose what works best for you. One great tool is Spokeo. Spokeo lets you make reverse lookups involving names, addresses and phone, thereby allowing you more ways to obtain data and more data points to utilize.

Automated Reverse Email Lookup Using An API

If you still want something better, something that can integrate seamlessly with your existing business system or application, you can consider going for a reverse email lookup API. With an API, a reverse email lookup process can practically run on its own within your application. The data obtained from the email search can then be fed back automatically into your application, allowing you to utilize the data points immediately for your business without worrying about the lookup process itself.

Proxycurl is an excellent tool for obtaining and enriching data about people and companies. One of its many services includes a Reverse Work Email Lookup API which you can use to programmatically pull data into your applications. They excel in data relating to the professional world, a more valuable type of data widely sought after. Another attractive feature of Proxycurl is its agile, lightweight pricing structure, at a cost of as low as $0.03 per search. No bulky monthly subscription that ties you down every month so you only get to pay as you use (or pay as you go if you'd like).

Conclusion

Now you know all about reverse email lookup, what it is used for, its capabilities and the different tools available out there depending on your use case. Will you now choose to integrate it into your current business workflow, or start using it personally to protect yourself against certain risks or fraud?

Give Proxycurl’s Reverse Work Email Lookup API a try today, send an email to find out more.

Credit to Nubela

DEV Community: chryzcode

Google Authentication in Nodejs using Passport and Google Oauth

Create Your NodeJS Application

Test your Node.JS Application

Conclusion

JWT Custom Authentication for Django Application

Introduction

Let's start building

Create Django Application

Create a Virtual Environment(optional)

Install Necessary Dependencies

Application Configuration

Create Super User

Conclusion

What is SQL and SQLite?

Why this title/ subject matter?

What is SQL?

What is SQLite?

Differences between SQL and SQLite

Use Cases of SQL and SQLite

SQL

SQLite

Conclusion

Best Practices For Deploying Django Applications(Production Environment and Hosting)

Introduction

Why PostgreSQL Database and Heroku?

PostgreSQL Database

Heroku

Getting started

Prerequisites and Requirements

Configuring settings for static files

Setting Heroku and project up for deployment

Build a Custom Functional Django Project

Introduction

Create a virtual environment

Installing Django

Create note models

Add Django auth URLs

Create migration

Create database

Create a superuser

Add notes model to Django admin

Create custom application interfaces and functions

Add the Login and Logout Redirect URL

Create template and static files(.html and .css)

Building a Custom Django Project/ Application(Folders and Files 2)

Introduction

Building Custom Django Application

Create an application URL file

Connect the application URL file to the project URL file.

Connect the app to the project.

Create our first Django web page.

Django custom view using HttpResponse

Django custom view using Template

The Django Project Folder and Files Structure and Architecture 1

Introduction

Overview of the Django Project

The Application Folder(my_app)

The Project Folder(my_project)

How to Create a Django Project and Application.

Introduction.

A Little about Django.

Get started

Prerequisites and Requirements

Create a Django Project.

Create a Django Application

Run and Test the Django Project

Machine Learning Pipeline Architecture

Introduction.

What is Machine Learning?

What is a Machine Learning Pipeline?

Benefits of Machine Learning Pipeline Architecture.

Why does Machine Learning Pipeline Architecture Matter?

Adoption and When to Use Machine Learning Pipeline Architecture.

Adoption of Machine Learning Pipeline.

Machine Learning Pipeline Architecture and Stages.

Machine Learning Pipeline Infrastructure/ Model Preparation Process.

Software/ Applications for Building Machine Learning Pipelines.

Machine Learning Pipeline Tools.

Message Broker: Memphis.Dev

Create template and static files(`.html` and `.css`)

The Application Folder(`my_app`)

The Project Folder(`my_project`)