DEV Community: Siddharth Shanker

Documentation for Deployment of Applications and Managing with Supervisor

Siddharth Shanker — Wed, 24 Jan 2024 11:35:08 +0000

This involves deployment and integration of Django app, Postgres, Gunicorn, Nginx and Supervisor(used for managing gunicorn instances)

Copy the Django application from LOCAL to REMOTE server

Run this command on local system

scp -r <path-to-django-app-on-local>  ubuntu@<ip address>:<path-where-you-want-to-copy-to-on-remote>

Log into Remote Server and install libraries from UBUNTU Repositories

sudo apt update
sudo apt install python3-venv python3-dev libpq-dev postgresql postgresql-contrib nginx curl

Setting up PostgreSQL DB and User

Log into postgres session

sudo -u postgres psql

Inside Postgres Prompt setting up DB and user

CREATE DATABASE <db-name>;

CREATE USER <username> WITH PASSWORD '<password>';

ALTER ROLE <username> SET client_encoding TO 'utf8';

ALTER ROLE <username> SET default_transaction_isolation TO 'read committed';

ALTER ROLE <username> SET timezone TO 'UTC';

GRANT ALL PRIVILEGES ON DATABASE <db_name> TO <username>;

Setting up Django-Project and Virtual Environment

Create Virtual Environment for Django Project and activate

python3 -m venv <environment-name>

source <environment-name>/bin/activate

Install psycopg for postgres support

 pip install psycopg2-binary

Install gunicorn

pip install gunicorn

Now Integrating PostgreSQL

Go to Configs and change the database values to following

DATABASES = {
                    'default': {
                        'ENGINE': 'django.db.backends.postgresql_psycopg2',
                        'NAME': 'db-name',
                        'USER': 'username',
                        'PASSWORD': 'password',
                        'HOST': 'localhost',
                        'PORT': '',
                    }
            }

Making exception for port 8000

sudo ufw allow 8000

Run following commands to check integrity and collect static

python manage.py makemigrations

python manage.py migrate

python manage.py runserver

python manage.py collectstatic

SETTING UP GUNICORN FOR SUPERVISOR (IF NOT SETUP)

Create a gunicorn file managed by supervisor inside bin folder of virtual env

cd <path to env of django app>/bin

sudo nano gunicorn_supervisor

PASTE the following bash script configuration to run Gunicorn and make changes for individual sock files, path to project, user/group etc accordingly.

#!/bin/bash


DIR=<path to django app directory where manage.py is present>  #Directory where project is located
USER=ubuntu   #User to run this script as
GROUP=ubuntu  #Group to run this script as
WORKERS=3     #Number of workers that Gunicorn should spawn
SOCKFILE=unix:/run/<project-name>.sock  #This socket file will communicate with Nginx 
DJANGO_SETTINGS_MODULE=<project-dir-name-which-contains-settings.py>.settings   #Which Django setting file should use
DJANGO_WSGI_MODULE=<project-dir-name-which-contains-wsgi.py>.wsgi           #Which WSGI file should use
LOG_LEVEL=debug
cd $DIR
source <path to venv>/bin/activate  #Activate the virtual environment
export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
export PYTHONPATH=$DIR:$PYTHONPATH


#Command to run the progam under supervisor
exec <path-to-venv>/bin/gunicorn ${DJANGO_WSGI_MODULE}:application \
--workers $WORKERS \
--user=$USER \
--group=$GROUP \
--bind=$SOCKFILE \
--pythonpath <path-to-venv>/bin/python \
--log-level=$LOG_LEVEL \
--access-logfile=/var/log/supervisor/<name-the-file>.log \
--error-logfile=/var/log/supervisor/<name-the-file>.log

Give Permissions for this file to read-write-execute

sudo chmod 755 gunicorn_supervisor

Creating Supervisor program for managing the Gunicorn Instances

sudo apt-get install supervisor

Creating a conf file for supervisor to watch for Gunicorn Instance of the django app

cd /etc/supervisor/conf.d/

sudo nano <app-name>.conf

Paste the following and make changes accordingly for user, logs and program name

[program:<program-name>]
command=<path-to-app-venv>/bin/gunicorn_supervisor
user=root
autostart=true
autorestart=true
stderr_logfile=/var/log/supervisor/<name-of-error-log-file>.log
stdout_logfile=/var/log/supervisor/<name-of-output-log-file>.log

Run the following commands

sudo supervisorctl reread
sudo supervisorctl update

You can also check the status of your app or start, stop or restart it using supervisor.

sudo supervisorctl status <program-name>
sudo supervisorctl start <program-name>
sudo supervisorctl stop <program-name>
sudo supervisorctl restart <program-name>

NGINX Installation and Setup

Creating configuration for Nginx to proxy pass Gunicorn

sudo nano /etc/nginx/sites-available/<project-name>

Now Paste this and make changes accordingly linking correct sock file

server {
    listen 80;
    server_name server_domain_or_IP;

    location = /favicon.ico { access_log off; log_not_found off; }
    location /static/ {
        root <path-to-dir-which-contains-static-dir>;
    }

    location / {
        include proxy_params;
        proxy_pass http://unix:/run/<name-of-sock-file>;
    }
}

If there are multiple applications being served from same domain/IP then:-

1. Make individual app files within nginx
2. Change listening port of server i.e every app will listen on individual ports

    eg:- app1                               app2

        server {                            server {
            listen 80;                          listen 81;
            ....                                ....
        }                                   }

sudo ln -s /etc/nginx/sites-available/<project-name> /etc/nginx/sites-enabled

To save the configs and check for any errors

sudo nginx -t

Restart Nginx

sudo systemctl restart nginx

Finally, you need to open up your firewall to normal traffic on port 80. Since you no longer need access to the development server, you can remove the rule to open port 8000

sudo ufw delete allow 8000
sudo ufw allow 'Nginx Full'

Troubleshooting Nginx, Gunicorn and Supervisor

Django Technical Paper

Siddharth Shanker — Fri, 16 Jun 2023 07:35:00 +0000

Django is a high level Python web framework that enables rapid development of secure and maintainable websites.

Settings file

What is secret key ?

Django secret key is used to provide cryptographic signing. This key is mostly used to sign session cookies, create hashes and tokens for sensitive information like csrf tokens or password reset tokens. ### What are default Django apps inside it ? Are there more ?
django.contrib.admin - Provides Django administration interface, which allows users to manage and update application's data.
django.contrib.auth - Provides the authentication system, including user modes, login/logout views and password management.
django.contrib.contenttypes - Provides the track of all the models along with a high-level, generic interface for working with our models.
django.contrib.sessions- This app lets us store and retrieve session data for each visitor to your site.
django.contrib.messages - This framework allows us to temporarily store messages in one request and retrieve them for display in subsequent request.
django.contrib.staticfiles - Helps in managing static files (eg- CSS, JavaScript) in Django project and serves them during development.

These are default installed apps, other apps that can be added to our Django project include:-

sites - Provides framework for managing multiple sites
sitemaps - Helps in generating XML sitemaps for Django site
humanize Provides a set of template filters that help in formatting data in a more human readable way.
redirects app allows to manage URL redirects
gis- Provides support for geographic data and includes spatial fields, spatial indexes. ### What is middleware ? What are different kinds of middleware ?
Defined as classes with methods that handle the processing of requests and responses.
Middlewares can perform authentication, session management, content encoding, URL redirection and more. List of middlewares :-
django.middleware.security.SecurityMiddleware: This middleware adds several security-related HTTP headers to the responses, such as enforcing HTTPS, preventing click-jacking attacks, and setting content type options.

django.middleware.csrf.CsrfViewMiddleware: This middleware provides protection against Cross-Site Request Forgery (CSRF) attacks by adding and verifying CSRF tokens in POST requests.
django.contrib.sessions.middleware.SessionMiddleware: This middleware enables session management by storing session data in cookies or a database, and it makes the session data available in request objects.
django.contrib.auth.middleware.AuthenticationMiddleware: This middleware associates authenticated users with requests by adding the User object to the request.
django.contrib.messages.middleware.MessageMiddleware: This middleware enables the messaging framework, which allows the temporary storage of messages that can be displayed to the user on subsequent requests.
django.middleware.locale.LocaleMiddleware: This middleware handles language selection and translation by detecting the user's preferred language and activating the corresponding translation.
django.middleware.common.CommonMiddleware: This middleware performs common operations, such as URL rewriting, response gzip compression, and appending a slash to the end of URLs.

CSRF

Stands for Cross site request forgery.
CSRF attacks allow a malicious user to execute actions using the credentials of another user without that user's knowledge or consent.
Django provides CSRF tokens for security.
When a user visits a Django site, a CSRF token is generated ands stored in their sessions or cookies.
When the server receives POST request it verifies the CSRD token, if they match the POST request is determined as valid otherwise rejected.
CsrfViewMiddleware is responsible for generating and validating tokens.
In HTML templates we use { % csrf_token %}

XSS

Stands for Cross-Site Scripting
Type of security vulnerability that allows attackers to inject malicious scripts into web pages viewed by other users.
Occurs when web application doesn't verify user input.
Code is executed by the victims and lets attackers bypass access controls and impersonate users. ### Click Jacking
Here the attacker cons the victim by making them perform clicking action on a seemingly harmless element or perform any action on a web page without their consent.
This is done by deceiving the victim with the help of interface.
Eg:- Victim accesses a decoy website and clicks on a button to win a prize. Unknowingly, they have been deceived by an attacker into pressing an alternative hidden button and this results in the payment of an account on another site.

WSGI

Pronounced as "whiskey" :P
It is a specification that defines a simple and universal interface between web servers and web applications or frameworks.
Provides a set of rules and guidelines for how a web server should communicate with a web application or framework.
The communication between the server and the application/framework happens through a callable object called the "application" or "app" object. This object is responsible for handling incoming requests and returning appropriate responses.

Models

Models.py

File contains the definition of the application's data models using Django's Object-Relational Mapping (ORM) system. Models define the structure and behavior of database tables, allowing you to interact with data in a Pythonic way.

Field Options

null - If True, Django will store empty values as NULL in the database. Default is False.
blank - If True, the field is allowed to be blank. Default is False.
choices - A tuple of tuples or a list of tuples, where each tuple represents a choice option. The first element of each tuple is the value stored in the database, and the second element is the human-readable label displayed to the user.
default -
The default value for the field. This can be a value or a callable object. If callable it will be called every time a new object is created.
error_messages -
The error_messages argument lets you override the default messages that the field will raise. Pass in a dictionary with keys matching the error messages you want to override.
primary_key -
If True, this field is the primary key for the model.
If you don’t specify primary_key=True for any field in your model, Django will automatically add a field to hold the primary key
unique -
If True, this field must be unique throughout the table

Field Types

AutoField
- An IntegerField that automatically increments according to available IDs.
BigIntegerField
- A 64-bit integer, much like an IntegerField except that it is guaranteed to fit numbers from -9223372036854775808 to 9223372036854775807.
BinaryField
- A field to store raw binary data. It can be assigned bytes bytearray, or memoryview. By default, BinaryField sets editable to False, in which case it can’t be included in a ModelForm.
BooleanField
- A true/false field.
CharField
- A string field, for small- to large-sized strings.
- has extra arguments:
- .max_length - The maximum length (in characters) of the field. The max_length is enforced at the database level
DateField
- Field type used to store and handle date values. It typically provides functionalities to parse, format, and validate dates.
DecimalField
- Field type used to store decimal numbers with a fixed precision and scale. It is often used when precise decimal calculations are required, such as in financial applications.
FileField
Field type used to handle file uploads. It allows users to upload files, and the FileField stores the file on the server and keeps track of its location or other relevant information.
EmailField
Field type used for email address input and validation. It typically ensures that the entered value follows the format of an email address.
JSONField
Field type used to store JSON (JavaScript Object Notation) data. It allows you to store JSON objects or arrays in a database field and provides functionalities to access and manipulate the JSON data.
TextField
Field type used to store large amounts of text data. It is suitable for storing long-form textual content, such as blog posts or user comments, where a predefined limit on the text length is not necessary.
UUIDField
Field type used to store universally unique identifiers (UUIDs). UUIDs are 128-bit identifiers that are unique across all devices and systems, making them suitable for various identification purposes.
SlugField

  Field type used to generate and store a URL-friendly version of a string. It is commonly used in web development frameworks to create human-readable and search engine-friendly URLs based on other field values.

## Django ORM

>>> from iplapp.models import *
>>> Matches.objects.all()
<QuerySet [<Matches: Matches object (1)>, <Matches: Matches object (2)>, <Matches: Matches object (3)>, <Matches: Matches object (4)>, <Matches: Matches object (5)>, <Matches: Matches object (6)>, <Matches: Matches object (7)>, <Matches: Matches object (8)>, <Matches: Matches object (9)>, <Matches: Matches object (10)>, <Matches: Matches object (11)>, <Matches: Matches object (12)>, <Matches: Matches object (13)>, <Matches: Matches object (14)>, <Matches: Matches object (15)>, <Matches: Matches object (16)>, <Matches: Matches object (17)>, <Matches: Matches object (18)>, <Matches: Matches object (19)>, <Matches: Matches object (20)>, '...(remaining elements truncated)...']>
>>>
>>>
>>>
>>> Matches.objects.aggregate(count=Count('id'))
Traceback (most recent call last):
  File "<console>", line 1, in <module>
NameError: name 'Count' is not defined
>>> from django.models import Count
Traceback (most recent call last):
  File "<console>", line 1, in <module>
ModuleNotFoundError: No module named 'django.models'
>>> from django.db.models import Count
>>> Matches.objects.aggregate(count=Count('id'))
{'count': 636}
>>>

Turning ORM to SQL in Django Shell

Run the python shell

python manage.py shell

Import models and perform ORM query


>>> from django.db.models import *
>>>
>>> query = Matches.objects.all()

>>> print(query.query)

SELECT "iplapp_matches"."id", "iplapp_matches"."season", "iplapp_matches"."city", "iplapp_matches"."date", "iplapp_matches"."team1", "iplapp_matches"."team2", "iplapp_matches"."toss_winner", "iplapp_matches"."toss_decision", "iplapp_matches"."result", "iplapp_matches"."dl_applied", "iplapp_matches"."winner", "iplapp_matches"."win_by_runs", "iplapp_matches"."win_by_wickets", "iplapp_matches"."player_of_match", "iplapp_matches"."venue", "iplapp_matches"."umpire1", "iplapp_matches"."umpire2", "iplapp_matches"."umpire3" FROM "iplapp_matches"

AGGRGATIONS:

These are special mathematical operations that can be performed on Fields(vertically)
The common aggrigate functions are SUM, COUNT, MIN, MAX, AVG etc
Aggrigate functions should be imported from django models class Example
Move to shell

>>> from app.models import ModelName
>>> from django.db.models import Count,Sum,Min,Max,Avg
>>> ModelName.objects.aggregate(Count("id"))
>>> ModelName.objects.aggregate(Sum("Attribute"))
>>> ModelName.objects.aggregate(Avg("Attribute"))
>>> ModelName.objects.aggregate(Min("Attribute"))

ANNOTATIONS

Annotations are used to add an extra field to the query set object

Example:

>>> from app.models import ModelName
>>> from django.db.models import Count,Sum,Min,Max,Avg
>>> queryset = ModelName.objects.annotate(count = Count("id"))

Here each queryset object will have attribute 'count'

What is a migration file? Why is it needed?

Migrations files are generated by django when we hit command

python manage.py makemigrations  # It will show the changes that it's going to make

python manage.py migrate # This will change the state of database

django will keep track of all the changes that we do the models as a files with dependencies on each other in a migrations folder
ORM will check for the state of database when we hit migrate .if it come accross any changes it will try to do that in the process if any interruption happens because of some migrations file missing etc, it will collapse the database.

What are SQL Transactions

Transactions in SQL are the bundle or set of operations that are performed for one functionality.
Transaction involves one or more database operations
These will make sure that either all the operations are successful or non of them exicutes, so that consistency has maintained in the database

ATOMIC TRANSACTIONS

These are the transactions that make sure the operations should be performed as a whole not individually
Since django uses the auto save, transactions are default not atomic
So, to make them atomic, we can use transactions from django.db


from django.db import transaction

@transaction.atomic
def viewfunc(request):
    # This code executes inside a transaction.
    do_stuff()

Using Context manager


def  view(self,  *args  ,  **options):

    with transaction.atomic():
        # db operations

References

SOLID Principles

Siddharth Shanker — Sun, 14 May 2023 17:12:09 +0000

The acronym originated by Robert C. Martin, and it stands for five principles of Object-Oriented Programming. These are a set of rules for certain design patterns and software architecture.

The SOLID principles are:

The Single-Responsibility Principle (SRP)
The Open-Closed Principle (OCP)
The Liskov Substitution Principle (LSP)
The Interface Segregation Principle (ISP)
The Dependency inversion Principle (DIP)

The Single-responsibility principle (SRP)

“A class should have one, and only one, reason to change”

Every component of our code should have only one responsibility.


class User:

    def __init__(self, name, email, password):

        self.name = name

        self.email = email

        self.password = password



    def login(self):

        # Some code to authenticate user

        return True



    def change_password(self, new_password):

        # Some code to change user's password

        return True

Here, User class is responsible for authentication from login and password management through change_password method.
This violates SRP because the class is responsible for two distinct tasks.


class UserAuthenticator:

    def __init__(self, user):

        self.user = user



    def login(self):

        # Some code to authenticate user

        return True





class PasswordManager:

    def __init__(self, user):

        self.user = user



    def change_password(self, new_password):

        # Some code to change user's password

        return True

Now we have separated the responsibilities, each class is now focused on a single task.
This makes code easier to understand and maintain.

The Open–closed principle (OCP)

“Software entities … should be open for extension but closed for modification”

If we need to add a new functionality or method, we don't need to modify the class, rather we should be able to extend it.


class checkShape:

    def handle_shape(self, shape):

        if request.type == 'Circle':

            # Handle request type Circle

        elif request.type == 'Rectangle':

            # Handle request type Rectangle

        elif request.type == 'Parallelogram':

            # Handle request type Parallelogram

        else:

            # Handle other request types

Now, suppose we want to check for Triangle, we would need to modify if else ladder by adding another elif statement for Triangle.
This violates OCP.
Better solution would be to use polymorphism and create separate classes for each request type that inherit from a common abstract class or interface.


from abc import ABC, abstractmethod



class Shape(ABC):

    @abstractmethod

    def process_shape(self):

        pass



class RequestCircle(Shape):

    def process_shape(self):

        # Handle request type Circle



class RequestRectangle(Shape):

    def process_shape(self):

        # Handle request type Rectangle



class RequestParallelogram(Shape):

    def process_shape(self):

        # Handle request type Parallelogram



class checkShape:

    def __init__(self, request):

        self.request = request



    def handle_shape(self):

        self.request.process_request()

The Liskov substitution principle (LSP)

“Functions that use pointers or references to base classes must be able to use objects of derived classes without knowing it”

“Derived classes must be substitutable for their base classes”.
If a subclass redefines a function also present in the parent class, a client-user should not be noticing any difference in behaviour, and it is a substitute for the base class.

If a subclass violates the structure of Parent class, this would result in violation of LSP.


class Rectangle:

    def __init__(self, width, height):

        self.width = width

        self.height = height



    def area(self):

        return self.width * self.height





class Square(Rectangle):

    def __init__(self, side):

        self.width = self.height = side

The Square class sets both the width and height to the same value, which means that the area method will return an incorrect value for rectangles.
We can change the design to ensure that Square and Rectangle have a common interface but different implementations.

from abc import ABC, abstractmethod

class Shape(ABC):
    @abstractmethod
    def area(self):
        pass

class Rectangle(Shape):
    def __init__(self, width, height):
        self.width = width
        self.height = height

    def area(self):
        return self.width * self.height

class Square(Shape):
    def __init__(self, side):
        self.side = side

    def area(self):
        return self.side ** 2

Rectangle and Square can both inherit from the common abstract Shape class and implement the area method in a way that is consistent with their behaviour.

The Interface Segregation Principle (ISP)

“Many client-specific interfaces are better than one general-purpose interface”

A class should only have the interface needed (SRP) and avoid methods that won’t work or that have no reason to be part of that class.
This problem arises, primarily, when, a subclass inherits methods from a base class that it does not need.


class Vehicle:

    def flight_speed(self):

        pass

    def sail_speed(self):

        pass

    def speed(self):
        pass

    def acceleration(self):
        pass

    def brake(self):

        pass



class Car(Vehicle):

    # class would inherit all methods which might not be necessary

Vehicle interface defines 3 methods and forces Car class to use all 3 methods.
Car class might not require other methods defined in the Vehicle, which could be used by other classes like Ship and Aeroplanes.
Splitting the Vehicle interface into smaller interfaces that group together related methods. Then, the Car class can inherit from only the interfaces that it needs to implement.


class LandVehicle:

    def speed(self):

        pass

class AirVehicle:

    def flightspeed(self):

        pass 

class WaterVehicle:

    def sailspeed(self):
        pass

class Acceleration(self):

    def acceleration(self):
        pass

    def brake(self):

        pass 

class Car(LandVehicle, Acceleration):

    def speed(self):

        # implementation of speed method

        pass

    def accelerate(self):

        # implementation of accelerate method

        pass

    def brake(self):

        # implementation of brake method

        pass

class Aeroplane(AirVehicle, Acceleration):
        pass
        # define the methods from both classes

This way class Car would inherit only those interfaces it needs to implement.

# The Dependency Inversion Principle (DIP)

“Abstractions should not depend on details. Details should depend on abstraction. High-level modules should not depend on low-level modules. Both should depend on abstractions”


class Engine:

    def start(self):

        print("Engine started")



class Vehicle:

    def __init__(self):

        self.engine = Engine()



    def start(self):

        self.engine.start()

The Vehicle class depends directly on the low-level Engine class to start the vehicle's engine. This violates the Dependency Inversion Principle because the Vehicle class depends directly on a low-level module.
Dependency injection can be used to invert the dependencies and decouple the high-level Vehicle class from the low-level Engine class.


class Engine:

    def start(self):

        print("Engine started")



class Vehicle:

    def __init__(self, engine: Engine):
        self.engine = engine

    def start(self):

        self.engine.start()

We inject the engine object into the Vehicle constructor as a dependency.
This way, the Vehicle class is decoupled from the specific implementation of the engine and can work with any implementation that conforms to a common interface. This meets the Dependency Inversion Principle.


class ElectricEngine(Engine):
    def start(self):
        # implementation of start method for electric engine
        print("Electric Engine")
class FuelEngine(Engine):
    def start(self):
        print("Fuel Engine")

electric_engine = ElectricEngine()
fuel_engine = FuelEngine()
vehicle = Vehicle(electric_engine)
vehicle = Vehicle(fuel_engine)
vehicle.start()

We can now create an ElectricEngine or GasEngine class that conforms to this interface and pass it to the Vehicle constructor.
The Vehicle class is decoupled from the specific implementation of the engine and can work with any implementation that conforms to the engine interface.

References

Scaling and Load Balancer

Siddharth Shanker — Sun, 14 May 2023 12:34:19 +0000

Scaling

Process of increasing capacity of a system or application in order to handle more traffic or data
Either add more resources, such as servers or storage, or optimize the existing resources.

Why we need scaling ?

Increase in traffic or data as they grow can lead the system to become overloaded, have slow performance, downtime or even crash.
Scaling can improve performance by distributing the work load across multiple servers.
Faster response times and reduced downtime would result in better user experience.
System would become future-proof. As the traffic/workload increases, a highly scaled system would still continue to meet the demands of customers and users efficiently.

Types of Scaling

Horizontal Scaling:

Increasing or decreasing number of nodes in a system.

One can say, increasing/decreasing number of machines to handle the work load.

Also known as, scaling out.

Vertical Scaling:

Increasing or decreasing the power of the existing system.

Increasing/ decreasing CPU and Storages of existing system.

Also known as, Scaling Up.

Horizontal vs Vertical

	Horizontal Scaling	Vertical Scaling
Workload Distribution	Workload is distributed across multiple machines. Every machine has some part of data.	A single machine handles the entire workload.
Concurrency	Distributes multiple jobs across multiple machines. Reduces the workload on each machine.	Relies on multi-threading on the existing machine to handle multiple requests.
Complexity and maintenance	Higher, since one would need to maintain multiple machines.	Lower
Load Balancing	Necessary to actively distribute workload across the multiple nodes	Not required in the single node
Failure resilience	Low because other machines can offer backup	High since it’s a single source of failure
Communication	Network calls (Remote Procedural calls) which is slow	Inter process communication which is fast.
Data maintenance	Data inconsistency	Consistent
Limitation	Add as many machines as you can	There is hardware limitation

Which should you choose?

Both horizontal and vertical scaling have their own benefits and limitations. Since there isn’t a one-size-fits-all solution for organizations, you need to scale according to your needs and resources.

Cost - Initial hardware costs for horizontal upgrades are higher. If you are working on a tight budget and need to add more resources to your infrastructure quickly and cheaply, then vertical scaling may be the best option for you.
Future-proofing - Adding additional updated machines through horizontal scaling will increase the overall performance threshold of your organization. There is a limit to how much you can vertically scale a single node, and it may not be able to handle the demands of the future.
Reliability - Horizontal scaling may offer you a more reliable system. It increases redundancy and ensures that you are not relying on a single machine. If one machine fails, another may be able to pick up the slack temporarily.
Performance and complexity - Performance will depend on how your services work and how they are interconnected. Simple, straightforward applications won’t benefit much from being run on multiple machines. In fact, it may degrade its quality. Sometimes it’s better to leave the application as is and upgrade the hardware to meet demand.

How will I investigate the possibility of using scaling?

Analyse the current system: Analyse the current system to understand its limitations and potential for scaling.
Review the system architecture, monitor system performance, and try to identify any bottleneck or issues present in the system.
Determine scaling requirements: Analysing expected traffic volume, peak usage times, and user behaviour can help us determine the system's needs for additional resources or servers.
Evaluate scaling solutions: Involves considering both vertical and horizontal scaling.
Plan and implement a scaling solution: Either add more resources or servers to an existing infrastructure, or optimize the system for better performance. Choose accordingly after identifying scaling solution.
Test and monitor the system: Test and monitor the system to ensure that it's performing as expected. Ensure there are no bottlenecks and system is operable at high traffic.

Load Balancers

An essential component in modern web architecture that help distribute traffic evenly across multiple servers. They are often used to improve the availability and performance of applications by spreading out the workload and reducing the risk of server overload.
Helps in improving the efficiency of servers by optimizing traffic distribution, monitoring server health, and reducing downtime.
It can be a physical device or a virtualized instance running as a software process.

Investigating whether a load balancer is required or not

Analyse traffic patterns: Analyse the traffic patterns for the project. Monitoring web server logs or use a traffic monitoring tool to identify any patterns in traffic volume or usage.
Identify bottlenecks: Bottlenecks can limit the performance of the system. Review system logs or use performance monitoring tools to identify any issues or inefficiencies.
Consider high availability: If a project requires availability to meet demands of clients at any instant, then a load balancer is a must. Distributing traffic across multiple servers can help the project.
Determine scalability requirements: If the project is expected to grow in traffic volume or usage over time, a load balancer can ensure that the system can handle increased load by distributing traffic across multiple servers.
Evaluate load balancing solutions: Considering both hardware and software load balancing solutions, as per the requirements.
Plan and implement load balancing solution: Configure the load balancer, optimize the system for better performance.
Test and monitor the system: Test and monitor the system to ensure that it's performing as expected. This may involve load testing to simulate high traffic or usage times and monitoring the system for any issues or bottlenecks.

References

Gaurav Sen's system design
Load Balance GeeksforGeeks
CloudZero Blog on Scaling

SQL Concepts

Siddharth Shanker — Fri, 12 May 2023 10:15:46 +0000

Transaction

A transaction is a single logical unit of work that comprises one or more database operations.
Transaction typically consists of several operations that read or modify data in a database.

ACID Properties

ACID which stands for Atomicity, Consistency, Isolation and Durability are the key attributes that define a transaction.

Atomicity(A):

Entire transaction takes place at once or doesn't happen at all.
Transaction do not occur partially, i.e each transaction is considered as single unit.

There are two operations involved:
Abort :- Aborting a transaction leads to no changes being visible which were made to a database.
Commit:- Changes would be visible after commiting a transaction.

Example

Consider a transaction T which consists of T1 and T2: Task is to transfer 100 from account X to account Y

T1: Deduct the amount from account X
T2: Credit the amount to account Y

If the transaction fails after the completion of T1 and before completion of T2 then the amount will be deducted from account X but it will not be added to the account Y which ultimately results in an inconsistent database state.

Consistency( C ) :-

This property ensures that a transaction brings the database from one valid state to another valid state. In other words, the database must be consistent both before and after a transaction is executed.

Isolation (I):-

This property ensures that concurrent transactions do not interfere with each other. Each transaction must execute in isolation from other transactions, and the results of one transaction must not affect the results of another transaction.

Durability( D ):-

This property ensures that once a transaction is committed, its changes are permanently stored in the database, even in the event of a system failure or power outage. The changes made by a committed transaction should be durable and survive any system crash or failure.

CAP Theorem

CAP theorem suggests that it is impossible for a distributed database system to simultaneously provide consistency, availability, and partition tolerance.
In a distributed database system, there is always a trade-off between consistency and availability when it comes to handling network partitions.
CAP Theorem is a concept that a distributed database system can only have 2 of the 3: Consistency, Availability and Partition Tolerance.

Consistency :- Every read operation on the system should return the most recent value or an error. In other words, all nodes in the system see the same data at the same time.

A system has consistency if a transaction starts with the system in a consistent state, and ends with the system in a consistent state.

Availability :- Every request sent to a non-failing node in the system should receive a response, without any guarantee about the freshness of the data. In other words, the system should always be responsive and accessible.

Achieving availability in a distributed system requires that the system remains operational 100% of the time.

Partition Tolerance :- The system should continue to function even if there are network failures that cause communication delays or data loss between nodes in the system.

A system that is partition-tolerant can sustain any amount of network failure that doesn’t result in a failure of the entire network.

Joins

This is a query operation that combines rows from two or more tables based on a related column or set of columns between them.

Inner Join :

This join returns only the matching rows from both tables based on a common column or set of columns.
It is like an intersection operation in Set Theory.

Left Join:

This join returns all the rows from the left table and matching rows from the right table. If there are no matching rows in the right table, then the result will contain NULL values.

Right Join:

Same as Left join but this join returns all rows from the right table and matching rows from the left table.

Full Outer Join:

This join returns all the rows from both tables and NULL values for the non-matching rows.

Cross Join:

This type of join returns the Cartesian product of both tables, which means every row from the left table is joined with every row from the right table.

Aggregations, Filters in queries:

Aggregations

Aggregation is a query operation
Computes a summary value or statistic for a group of records.
Used to generate reports from large datasets.

Types of aggregations:-

COUNT: Returns the number of rows in a table or a subset of a table that match a specific condition.
SUM: Returns the sum of a specific column in a table or a subset of a table that matches a specific condition.
AVG: Returns the average value of a specific column in a table or a subset of a table that matches a specific condition.
MAX: Returns the maximum value of a specific column in a table or a subset of a table that matches a specific condition.
MIN: Returns the minimum value of a specific column in a table or a subset of a table that matches a specific condition.

These are typically used with the GROUP BY clause.

Filters:

Used to retrieve relevant information from large datasets based on specific criteria.
Can be applied using the WHERE clause in queries.
Combined using logical operators such as AND, OR, and NOT to create more complex conditions.

SELECT * FROM orders
WHERE order_total > 100

Types of filters:

Comparison Filters: Compare a column or an expression with a specific value or another column.
Comparison operators include "=", "<", ">", "<=", ">=", and "<>".
Range Filters: Fetches records that fall within a specific range of values.
Range operators include "BETWEEN" and "NOT BETWEEN".
Pattern matching Filters: Matches a pattern or substring within a column.
Pattern matching operators include "LIKE", "NOT LIKE"
Null Filter: Retrieves records with NULL values in specific column.
IS NULL and IS NOT NULL are used.
Logical Filters : Combine multiple conditions using logical operators such as "AND", "OR", and "NOT".

Normalization

Process of organizing data in a database to reduce data redundancy and dependency.
Objective is to minimize data duplication and inconsistency.
Improves data integrity, reduces storage requirements, and improves database performance.

Normalization divides a database into two or more tables and defines relationships between them.

Each table represents a distinct entity or concept, and the relationships between the tables are based on the attributes that they share.

1NF: A relation is in 1NF if all its attributes have an atomic value.
2NF: A relation is in 2NF if it is in 1NF and all non-key attributes are fully functional dependent on the candidate key in DBMS.
3NF: A relation is in 3NF if it is in 2NF and there is no transitive dependency.
BCNF: A relation is in BCNF if it is in 3NF and for every Functional Dependency, LHS is the super key.

Indexes

Indexing makes columns faster to query by creating pointers to where data is stored within a database.
An index is a structure that holds the field the index is sorting and a pointer from each record to their corresponding record in the original table where the data is actually stored.

Transactions

Can be defined as a group of tasks.
A single task is the minimum processing unit which cannot be divided further.

States of Transactions

A transaction in a database can be in one of the following states −

Active − In this state, the transaction is being executed. This is the initial state of every transaction.
Partially Committed − When a transaction executes its final operation, it is said to be in a partially committed state.
Failed − A transaction is said to be in a failed state if any of the checks made by the database recovery system fails. A failed transaction can no longer proceed further.
Aborted − If any of the checks fails and the transaction has reached a failed state, then the recovery manager rolls back all its write operations on the database to bring the database back to its original state where it was prior to the execution of the transaction.
Committed − If a transaction executes all its operations successfully, it is said to be committed. All its effects are now permanently established on the database system.

Locking mechanism

Mechanism that ensures that the integrity of data is maintained. It locks the data while a transaction is running
Any transaction cannot read or write the data until it acquires the appropriate lock.

1. Shared Lock(S): Shared lock is placed when we are reading the data, multiple shared locks can be placed on the data but when a shared lock is placed no exclusive lock can be placed.

2. Exclusive Lock(X): Exclusive lock is placed when we want to read and write the data. This lock allows both the read and write operation, Once this lock is placed on the data no other lock (shared or Exclusive) can be placed on the data until Exclusive lock is released.

Database Isolation Levels

The SQL standard defines four isolation levels.

Read Uncommitted

The lowest isolation level allowed by SQL is read uncommitted. In this level, transactions can read data that is not committed by other transactions, permitting dirty read.

Read Committed

This isolation level allows for the reading of data after it is committed by a transaction. This means no dirty reads are possible. This isolation is unable to prevent non-repeatable reads.

Repeatable Read

This isolation tries to improve on the previous isolation level by preventing both dirty reads and non-repeatable reads. This prevention is done by applying locks on rows that are read and rows that perform write operations.

Serializable

The highest isolation level allowed is serializable. This isolation level looks like a serial execution, with almost no concurrency. All the three read phenomena are prevented in this isolation level, but it compromises concurrent execution.

Triggers

A trigger is a stored procedure in database which automatically invokes whenever a special event in the database occurs.
For example, a trigger can be invoked when a row is inserted into a specified table or when certain table columns are being updated. In Structured Query Language, triggers are called only either before or after the below events:

INSERT Event: This event is called when the new row is entered in the table.
UPDATE Event: This event is called when the existing record is changed or modified in the table.
DELETE Event: This event is called when the existing record is removed from the table.

Strings in Python

Siddharth Shanker — Thu, 11 May 2023 06:26:42 +0000

In Python, a string is a sequence of characters enclosed in quotes. Strings can be created using single quotes ('), double quotes ("), or triple quotes (''' or """) depending on the requirements.
Strings in Python are immutable, meaning once they are created, their contents cannot be changed.
However, you can create a new string by concatenating existing strings or by using string methods.

>>> string1 = 'Hello, World!'
>>> string2 = "This is a string."
>>> string3 = '''This is a
... multi-line string.'''
>>> string4 = """This is also
... a multi-line string."""

String Methods of Python

1. len() :-

This method returns the length of a string.

>>> len(string1)
13

2. .upper():-

This method returns a new string with all characters in uppercase.

>>> string2
'This is a string.'
>>> string2.upper()
'THIS IS A STRING.'

3. .lower() :-

This method returns a new string with all characters in lowercase.

>>> string2
'This is a string.'
>>> string2.lower()
'this is a string.'

4. .capitalize() :-

This method returns a new string with the first character capitalized.

>>> string4 = string2.lower()
>>> string4
'this is a string.'
>>> string4.capitalize()
'This is a string.'
>>> string4
'this is a string.'

5. .title():-

This method returns a new string with the first character of each word capitalized.

>>> string3 = "This is a small string"
>>> string3.title()
'This Is A Small String'

6. split(delimiter) :-

This method returns a list of strings by splitting the original string at each occurrence of a specified delimiter.
By default, delimiter is space
example:-

>>> string3
'This is a small string'
>>> string3.split()
['This', 'is', 'a', 'small', 'string']
>>> string3 = 'This_is_a_string'
>>> string3.split('_')
['This', 'is', 'a', 'string']

7. count(x) :-

The .count() method returns the number of times an item appears in the string.

>>> string1 = 'Lets count a many times aaaaa'
>>> string1.count('a')
7

8. .strip() :-

This method returns the string with both leading and trailing characters.

>>> string1 = '   this is   a string  '
>>> string1.strip()
'this is   a string'

9. .endswith() :-

This method returns true if the string ends with the specified value.

>>> string1.endswith('aaaaa')
True
>>> string1.endswith('aaaaaa')
False

10. .find() :-

This method searches the string for a specified value and returns the position of where it was found.

>>> string1
'Lets count a many times aaaaa'
>>> string1.find('m')
13
>>> string1.find('tim')
18

11. .startswith() :-

This method returns true if the string starts with the specified value

>>> string1
'Lets count a many times aaaaa'
>>> string1.startswith('L')
True
>>> string1.startswith('l')
False

12. .swapcase() :-

This method swaps cases, lower case becomes upper case and vice versa

>>> string1 = "LETS SEE swap of casing"
>>> string1.swapcase()
'lets see SWAP OF CASING'

13. .isalnum() :-

This method returns True if all characters in the string are alphanumeric

>>> string1 = "abc 89a"
>>> string1.isalnum()
False
>>> string1 = "abc89a"
>>> string1.isalnum()
True

14. .isalpha() :-

This method returns True if all characters in the string are in the alphabet

>>> string1
'abc89a'
>>> string1.isalpha()
False
>>> string1='abcd'
>>> string1.isalpha()
True

15. .isnumeric() :-

This method returns True if all characters in the string are numeric

>>> string1 = "abc89a"
>>> string1.isnumeric()
False
>>> string1 = "89.456"
>>> string1.isnumeric()
True

16. .isdigit() :-

This method returns true if all characters in the string are digits.

>>> string1 = "abc 89a"
>>> string1.isdigit()
False
>>> string1 = "89"
>>> string1.isalnum()
True

17. .islower() :-

This method returns True if all characters in the string are in lowercase.

>>> string1 = "abAde"
>>> string1.islower()
False
>>> string1 = "abcdae"
>>> string1.islower()
True

18. .isupper() :-

This method returns True if all characters in the string are alphanumeric

>>> string1 = "abac"
>>> string1.isupper()
False
>>> string1 = "ABCA"
>>> string1.isupper()
True

19. .replace() :-

This method returns a string where a specified value is replaced with a specified value

>>> string1='abcd'
>>> string2 = 'efg'
>>> string1.replace('d' , string2)
'abcefg'

20. .format() :-

This method formats specified values in a string

>>> txt = "For only {price:.2f} dollars!"
>>> txt.format(price = 49)
'For only 49.00 dollars!'

References

Python docs official
W3Schools

Lists

Siddharth Shanker — Thu, 11 May 2023 05:34:27 +0000

In Python, a list is a mutable sequence data type that can store a collection of values of any data type, including other lists.
Lists are ordered, meaning the order in which items are added to the list is preserved.
They are indexed, meaning each item can be accessed by its index in the list.

>>> my_list = [1,2,3, 'four', 'five']
>>> my_list
[1, 2, 3, 'four', 'five']

List Methods of Python

1. Inserting data using .append(x) :-

This method appends the given element x at the end of the list.

>>> my_list.append(6.75)
>>> my_list
[1, 2, 3, 'four', 'five', 6.75]

2. Inserting from an iterable using .extend(iterable_data_type) :-

.extend() accepts an iterable and adds all its elements at the end of the list.

>>> my_tuple = ('seven' , 8, 9, 10.11)
>>> my_list
[1, 2, 3, 'four', 'five', 6.75]
>>> my_list.extend(my_tuple)
>>> my_list
[1, 2, 3, 'four', 'five', 6.75, 'seven', 8, 9, 10.11]

3. .insert(i, x) :-

This method inserts an item at a specific position in the list.

>>> my_list
[1, 2, 3, 'four', 'five', 6.75, 'seven', 8, 9, 10.11]
>>> my_list.insert(0, 0)
>>> my_list
[0, 1, 2, 3, 'four', 'five', 6.75, 'seven', 8, 9, 10.11]

4. .remove(x) :-

This method removes the first occurrence of an item x from the list.
If not present in the list it throws an error.

>>> my_list
[0, 1, 2, 3, 'four', 'five', 6.75, 'seven', 8, 9, 10.11]
>>> my_list.remove('five')
>>> my_list
[0, 1, 2, 3, 'four', 6.75, 'seven', 8, 9, 10.11]
>>> my_list.remove('five')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: list.remove(x): x not in list

5. .pop(i) :-

The .pop() method removes and returns the item at a specific position in the list. If no index is specified, it removes and returns the last item.
example:-

>>> my_list
[0, 1, 2, 3, 'four', 6.75, 'seven', 8, 9, 10.11]
>>> my_list.pop()
10.11
>>> my_list.pop()
9
>>> my_list
[0, 1, 2, 3, 'four', 6.75, 'seven', 8]
>>> my_list.pop(0)
0
>>> my_list
[1, 2, 3, 'four', 6.75, 'seven', 8]

6. index(x) :-

This method returns the index of the first occurrence of an item in the list.
You can specify a starting and/or ending index for the search.
example:-

>>> my_list = [1, 2, 3, 'four', 6.75, 'seven', 8]
>>> my_list.index(3)

7. count(x) :-

The .count() method returns the number of times an item appears in the list.

>>> my_list = [1, 2, 3, 'four', 6.75, 'seven', 8,3,3]
>>> my_list.count(3)
3

8. .sort() :-

The .sort() method sorts the items in ascending order.

>>> my_list
[1, 2, 3, 8, 3, 3]
>>> my_list.sort()
>>> my_list
[1, 2, 3, 3, 3, 8]

9. .reverse() :-

The .reverse() method is used to reverses the order of items in the list.

>>> my_list
[1, 2, 3, 3, 3, 8]
>>> my_list.reverse()
>>> my_list
[8, 3, 3, 3, 2, 1]

10. .copy() :-

This method will returns a copy of the list.

>>> my_list
[8, 3, 3, 3, 2, 1]
>>> my_copied_list = my_list.copy()
>>> my_copied_list
[8, 3, 3, 3, 2, 1]

Using Lists as Stacks

In Python, list can be directly usable as a stack(“last-in, first-out”).

to add an element to the top of the stack use append() method

and remove element from the top of the stack use pop() method.

Example:

>>> stack = [1,2,3,4]
>>> stack.append(5)
>>> stack.append(6)
>>> stack
[1, 2, 3, 4, 5, 6]
>>> 
>>> 
>>> stack.pop()
6
>>> stack
[1, 2, 3, 4, 5]

Using Lists as Queues

In Python, it is also possible to use list as a queue (“first-in, first-out”) ,for that we need to use append()method for inserting element and pop(0) for deleting element in the queue .However it is not the efficient way beacuse every time we delete a element from first position ,means we need to left shift by 1 for all the remaining elements.

To implement a queue, use collections.deque which was designed to have fast appends and pops from both ends.

Example:

>>> from collections import deque
>>> 
>>> queue = deque([1,2,3,4])
>>> queue
deque([1, 2, 3, 4])
>>> 
>>> 
>>> queue.append(5)
>>> queue
deque([1, 2, 3, 4, 5])
>>> 
>>> queue.popleft()
1
>>> queue
deque([2, 3, 4, 5])

References

Python docs official
W3Schools

Python Dictionary Data Structure and it's methods

Siddharth Shanker — Wed, 10 May 2023 18:51:38 +0000

Dictionary

In Python, a dictionary is a built-in data structure that is used to store a collection of key-value pairs.
Each key in a dictionary maps to a corresponding value, similar to a real-world dictionary where each word maps to a definition.
Dictionary is a ordered data structure i.e it will remember the order of insertion.
Python dictionaries are mutable, we can change the value of a given key at any point of time.
Dictionaries do not allow duplicate keys, since it uses hashing concept to store keys.

my_dict = {
...     "name": "John",
...     "age": 30,
...     "city": "New York"
... }
>>> my_dict
{'name': 'John', 'age': 30, 'city': 'New York'}

Dictionary Methods of Python

1. Inserting keys using dicitonary_name["key_name] = value :-

my_dict = {
... 'name' : 'John',
... 'age': 30,
... 'city': 'New York'
... }
>>> my_dict
{'name': 'John', 'age': 30, 'city': 'New York'}
>>> my_dict['surname'] = 'Wick'
>>> my_dict
{'name': 'John', 'age': 30, 'city': 'New York', 'surname': 'Wick'}

2. Accessing a Value using Key :-

There are two ways you can access a value using key
In python, there is a method .get() which takes a key as a argument, used to get a value as shown
.get() will return None , if key is not found in a dictionary

>>> my_dict.get("name")
'John'
>>> my_dict.get('occupation')
>>>

The other way to access a value is using square brackets, the major difference between two methods is that it will through error if key is not found in dictionary.

>>> my_dict['occupation']
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 'occupation'

Methods used while iterating over a Dictionary

1) .items() :-

This method returns an object which is of tuple form which includes key and value. Tuple - (key, value) object.
We will have to use an iterating loop to extract these tuples
example:-

>>> my_dict = {
...     "name": "John",
...     "age": 30,
...     "city": "New York",
...     "country": "USA",
... }
>>> for key,value in my_dict.items():
...     print(key,value)
... 
name John
age 30
city New York
country USA

2) .keys() :-

The .keys() method is used to return a view object that contains the keys of a dictionary. Each key in the view object is a string that corresponds to a key in the original dictionary.

>>> for keys in my_dict.keys():
...     print(keys)
... 
name
age
city
country

3) .values() :-

The .values() method is used to return a object that contains the values for respective keys of a dictionary.
example:-

>>> for values in my_dict.values():
...     print(values)
... 
John
30
New York
USA

4) in :-

Returns a boolean value for whether a key is present in dictionary or not.
example:-

>>> print('occupation' in my_dict)
False
>>> print('name' in my_dict)
True

5) update() :-

The .update() method is used to update a dictionary with new key-value pairs from another dictionary or an iterable of key-value pairs. If a key in the new dictionary already exists in the original dictionary, its corresponding value is updated with the new value.

>>> my_dict = {'name': 'John', 'age': 30, 'city': 'New York', 'country': 'USA'}
>>> new_dict = {'city': 'Los Angeles', 'phone': '555-1234'}
>>> my_dict.update(new_dict)
>>> my_dict
{'name': 'John', 'age': 30, 'city': 'Los Angeles', 'country': 'USA', 'phone': '555-1234'}

6) .copy() :-

The .copy() method is used to create a shallow copy of a dictionary. It returns a new dictionary with the same key-value pairs as the original dictionary.
Any changes made to the new dictionary isn't reflected in original dictionary.
example:-

>>> my_dict = {'name': 'John', 'age': 30, 'city': 'New York', 'country': 'USA'}
>>> new_dict = my_dict.copy()
>>> 
>>> new_dict['city'] = 'Los Angeles'
>>> my_dict
{'name': 'John', 'age': 30, 'city': 'New York', 'country': 'USA'}
>>> new_dict
{'name': 'John', 'age': 30, 'city': 'Los Angeles', 'country': 'USA'}

7) .pop(key) :-

The .pop() method is used to remove and return the value for a given key in a dictionary.
Key-Value pair is removed from dictionary.

my_dict
{'name': 'John', 'age': 30, 'city': 'New York', 'country': 'USA'}
>>> my_dict.pop('age')
30
>>> my_dict
{'name': 'John', 'city': 'New York', 'country': 'USA'}
>>> my_dict.pop('work')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 'work'

8) .popitem() :-

This method will pop the last item that was inserted and will return a tuple of key and value
This will return KeyError if dictionary is empty

>>> my_dict['age']=35
>>> my_dict
{'name': 'John', 'city': 'New York', 'country': 'USA', 'age': 35}
>>> my_dict.popitem()
('age', 35)
>>> my_dict
{'name': 'John', 'city': 'New York', 'country': 'USA'}

9) .clear() :-

Returns NONE after clearing the dictionary
example:-

>>> my_dict
{'name': 'John', 'city': 'New York', 'country': 'USA'}
>>> my_dict.clear()
>>> my_dict
{}

10) .setdefault()

Returns the value of the item with the specified key

example:-

>>> my_dict
{}
>>> my_dict.setdefault('name' , 'John')
'John'
>>> my_dict
{'name': 'John'}

References

Python docs official
W3Schools

Objects and Object Oriented Programming

Siddharth Shanker — Wed, 10 May 2023 17:45:16 +0000

Introduction

Object-oriented programming is a widely-used paradigm in software development. Python's object-oriented features include classes, objects, inheritance and polymorphism which allow for efficient and modular coding.

Everything in Python is an object. Using Pythom we can create classes and objects.

Classes

Class is like a blueprint that helps to create an object. Classes provide a means of bundling data and functionality together. It's a collection of objects.
Class comprises of properties and behaviour.

For eg:- Consider a class Vehicle, it might have properties or behaviour of Vehicles. Those properties might be size, fuel type, number of wheels, body type, speed etc.
How to make a Class in Python. We use Class keyword.

class Vehicle:
    def __init__(self):
            #constructor
    def method1(self, argument1, arg2 ....):
            #method code
    def method1(self, argument1, arg2 ....):
            #method code

Objects

Objects are an instance of a class. With the help of objects we can access the method and functions of a class.

This may be any real-world object.
For eg:- In case of a class Vehicle, we can have real world objects like Bicycle, Motor-Cycle, Cars, Trucks etc.

Creating an object Car of Vehicle class.

    #Case 1
    car = Vehicle()
    #Case 2 
    cycle = Vehicle
    cycle.method1()

This creates a new instance of class Vehicle and assigns this object to local variable car.

Objects consists of attributes/properties.
It consists of behaviour represented by the methods of an object.
Identity gives a unique name to an object.

Eg:- Car is an identity which can have properties like bodyType (like SUV, Sedan), color, top speed.
Behaviour could be acceleration, vehicle Engine, vehicle transmission type.

Inheritance

Inheritance is the capability of one class to derive or inherit the properties from another class.
Inheritance is like a family tree where children inherit traits and characteristics from their parents. Similarly, in object-oriented programming, a new class can be created by inheriting the properties and methods of an existing class.

    class Vehicle:
        def __init__():
            #constructor
        def method1():
            #method

    class Car(Vehicle):
        #child class of Vehicle
        #This derives all the attributes and methods of parent class
        def __init__(self):
            #constructor

Types of Inheritance:-

Single Inheritance:

Single-level inheritance enables a derived class to inherit characteristics from a single-parent class.

# Base class
class  Animal: 
    def  __init__(self, name, species):
         self.name = name self.species = species 
class  Cat(Animal): 
    def  __init__(self, name): 
        super().__init__(name, species="Cat") 
        self.sound = "Meow"

cat1 = Cat("Luna")
print(cat1.name, cat1.species, cat1.sound) 
# output: Luna Cat Meow

Multilevel Inheritance:

Multi-level inheritance enables a derived class to inherit properties from an immediate parent class which in turn inherits properties from his parent class.

class  Animal: 
    def  __init__(self, name, species): 
        self.name = name self.species = species 

class  Mammal(Animal): 
    def  __init__(self, name, species): 
        super().__init__(name, species) 
        self.is_mammal = True  

class  Cat(Mammal): 
    def  __init__(self, name): 
        super().__init__(name, species="Cat") 
        self.sound = "Meow"

cat1 = Cat("Luna")
print(cat1.name, cat1.species, cat1.sound, cat1.is_mammal) 
# output: Luna Cat Meow True

Hierarchical Inheritance:

Hierarchical level inheritance enables more than one derived class to inherit properties from a parent class.

# Hierarchical inheritance
# Base class
class Animal:
    def __init__(self, name, species):
        self.name = name
        self.species = species

#Derived class 1
class Cat(Animal):
    def __init__(self, name):
        super().__init__(name, species="Cat")
        self.sound = "Meow"

# Derived class 2
class Dog(Animal):
    def __init__(self, name):
        super().__init__(name, species="Dog")
        self.sound = "Woof"

class Bird(Animal):
    def __init__(self, name):
        super().__init__(name, species="Bird")
        self.sound = "Tweet"

# Derived class 3
cat1 = Cat("Luna")
dog1 = Dog("Max")
bird1 = Bird("Tweety")

print(cat1.name, cat1.species, cat1.sound)  # output: Luna Cat Meow
print(dog1.name, dog1.species, dog1.sound)  # output: Max Dog Woof
print(bird1.name, bird1.species, bird1.sound)  # output: Tweety Bird Tweet

Multiple Inheritance:

Multiple level inheritance enables one derived class to inherit properties from more than one base class.

# multilevel inheritance

class  Animal: 
    def  __init__(self, name, species): 
        self.name = name 
        self.species = species 
class  Carnivore: 
    def  __init__(self): 
        self.is_carnivore = True  
    def  eat(self): 
        print("I eat meat.")

class  Cat(Animal, Carnivore): 
    def  __init__(self, name): 
        Animal.__init__(self, name, species="Cat") 
        Carnivore.__init__(self)  
        self.sound = "Meow"

cat1 = Cat("Luna")
print(cat1.name, cat1.species, cat1.sound, cat1.is_carnivore)
# output: Luna Cat Meow True

Polymorphism

The word "polymorphism" means "many forms", and in programming it refers to methods/functions/operators with the same name that can be executed on many objects or classes.

class Bird:
    def intro(self):
        print("There are many types of birds.")

    def flight(self):
        print("Most of the birds can fly but some cannot.")

class sparrow(Bird):
    def flight(self):
        print("Sparrows can fly.")

class ostrich(Bird):
    def flight(self):
        print("Ostriches cannot fly.")

obj_bird = Bird()
obj_spr = sparrow()
obj_ost = ostrich()

obj_bird.flight()
obj_spr.flight()
obj_ost.flight()

Output:- 
Most of the birds can fly but some cannot.
Sparrows can fly.
Ostriches cannot fly.

Encapsulation

Encapsulation is the practice of hiding the internal details of an object and only exposing a public interface that can be used to interact with the object.
This puts restrictions on accessing variables and methods directly and can prevent the accidental modification of data.

class Animal:
    def __init__(self, name, species):
        self.__name = name
        self.__species = species

    def get_name(self):
        return self.__name

    def set_name(self, name):
        self.__name = name

cat1 = Animal("Luna", "Cat")
print(cat1.get_name())  # output: Luna
cat1.set_name("Mittens")
print(cat1.get_name())  # output: Mittens

We are using setters and getters to access variables. Thus, preventing accidental change.

Abstraction

Abstraction is the practice of focusing on the essential features of an object and ignoring the irrelevant details. It involves creating abstract classes and interfaces that define a set of methods or properties that a class must implement, without specifying how they are implemented.

from abc import ABC, abstractmethod

class Animal(ABC):
    def __init__(self, name, species):
        self.name = name
        self.species = species

    @abstractmethod
    def make_sound(self):
        pass

class Cat(Animal):
    def make_sound(self):
        print("Meow")

class Dog(Animal):
    def make_sound(self):
        print("Woof")

cat1 = Cat("Luna", "Cat")
dog1 = Dog("Max", "Dog")

cat1.make_sound()  # output: Meow
dog1.make_sound()  # output: Woof

References

GeeksforGeeks
W3Schools
freecodecamp