DEV Community: Marriane Akeyo

Docker from zero to pro

Marriane Akeyo — Tue, 01 Aug 2023 19:30:51 +0000

In this article, we embark on an insightful journey into the realm of Docker essentials, unraveling all the knowledge needed to grasp this groundbreaking technology. This involves delving deep into docker commands with examples and docker compose as well. Lets dig in!!

Docker definition

Docker can be defined as an application used to automate the deployment, management and scaling of applications. Its runs applications in an isolated environment very similar to virtual machines but slightly better. Docker works with "boxes" called containers which as the name suggests, holds everything that is required to run the application, from the code to its dependencies. Multiple containers can run on the same machine without affecting each other, since each container has its own dependencies. They all however share the same kernel(acts as an intermediary between the hardware and the software, providing essential services and functionalities to manage system resources, facilitate communication between hardware and software, and ensure overall system stability and security.)

Advantages of docker

It uses less memory space, since the application are stored in the docker container and not locally.
They are first to boot with just one command once the container is set up.
They are easy to set up compared to locally running instances of the same application which are limited to the running computer's capabilities and easy to scale when needed.
Easy to share with others, deploy and test applications.

The installation of docker and docker-compose(discussed later in the article) are limited to the type of OS a user is running. You can visit this link for installation guidance.
To verify if docker is correctly installed use the following command:

docker --version

Output example depending on the version installed
Docker version 24.0.4

Docker Image

This is a package template used to create one or more containers as we shall see in the examples bellow.

Docker hub

There are some common tool used in day to day programming like languages eg python, web servers eg nginx and databases eg postgresql , which are used when creating various applications. These tools are stored in a registry called docker hub in form of images. You need to create an account in order to access dockerhub and utilize its images.
Once the account is created, we can easily pull images from docker hub.
The images you create can also be pushed into docker hub and made public or private.
The following command pulls a python version 3.10 image from docker hub

docker pull python:3.10-alpine3.18

The alpine3.18 keyword pulls the smallest in size of python3.10 available , hence saving on storage space. It is advisable to use "alpine" when pulling or defining images.

To view all the images we can run

docker images
Output:
REPOSITORY       TAG               IMAGE ID       CREATED       SIZE
python           3.10-alpine3.18   7d34b8225fda   7 weeks ago   49.7MB

Running a normal installation of python3 using docker pull python:3.10 then viewing installed images, displays the difference in size between the normal one and the alpine version as shown:

REPOSITORY       TAG               IMAGE ID       CREATED       SIZE
python           3.10              d9122363988f   6 weeks ago   1GB
python           3.10-alpine3.18   7d34b8225fda   7 weeks ago   49.7MB

To remove an image run

docker image rm <image_name/image_id>

Example:
docker image rm python:3.10

Output:
d9122363988f

Docker Container

Once the image is defined, it can be used to create a container, since a container runs an instance of an image.
For example using the python image above:

docker run --name pytry -it python:3.10-alpine3.18

The --name flag is used to define the name that will be used to reference the container, else a random name will be given to your container.
The -it command is used to run the command in an interactive mode.
The above command produces a python shell which we can use to run various commands as shown.

Python 3.10.12 (main, Jun 15 2023, 03:17:49) [GCC 12.2.1 20220924] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> print("I am a python container")
I am a python container
>>>

To view all the flags that can be used with the docker run command use:

docker run --help

Now that the container is defined, run the command bellow on another terminal window to view running containers

Command:
docker ps

Output:
CONTAINER ID   IMAGE                    COMMAND     CREATED          STATUS          PORTS     NAMES
5bcde232650b   python:3.10-alpine3.18   "python3"   32 seconds ago   Up 31 seconds             pytry

To view all the container which are present in docker use:

docker ps -a
Move to the terminal window running python container and run the command exit() to return to the command line.
Notice running docker ps to view running containers produces no results since our container is closed.

To make the code above more complex we can create a folder called hello.py in the current directory and write some code in it. Then run the command again but with a different container name to avoid conflicts.

# file in the current directory
hello.py
from datetime import datetime
time = datetime.now()
print(f'I am a python file in a docker container running at {time}')

Command:
docker run --name pytryFile -w /app -v "$(pwd):/app" python:3.10-alpine3.18 python hello.py

Output:
I am a python file in a docker container running at <a_timestamp_of _the_time_right_now>

Let's break down this command:

-v "$(pwd):/app" mounts the current directory (pwd) into the container at the /app directory, allowing access to your Python script.
-w /app sets the working directory inside the container to /app.
python:3.10-alpine3.18 is the image name
python hello.py is the command that runs your Python script inside the container.

Managing Containers

To stop a running docker container use:

docker stop <container_name/container_id>
example:
docker stop pytry

Output:
pytry

To start a stopped container:

docker start <container_name/container_id>
example:
docker start pytry
Output:
pytry

To remove a docker container :

docker rm <container_name/container_id>
example:
docker rm pytry

To list all container's id

docker ps -aq

To get rid of all of the containers:

docker rm $(docker ps -aq)

To view other interesting flags you can use to manipulate containers:
docker ps --help

Docker Volumes

They enable easy sharing of information like files and folders between host and container and between containers themselves, just as the hello.py example above demonstrated.
Another example, we create a new folder and inside it , a file called hello.html that displays:

<h2> Hello World </>

To display this content on a web browser using the nginx web server, run the nginx image as shown below:

docker run --name helloWorld -v $(pwd)/hello.html:/usr/share/nginx/html/hello.html:ro -p 8080:80 -d nginx

# View the content on the browser as shown:
http://localhost:8080/hello.html

/usr/share/nginx/html: is the nginx folder used to display content stored in nginx web server
ro: includes permissions on the shared content, in our case read and write permission.
-p: this flag illustrates the port mapping in which the contents will be viewed from. In our case all the contents on the container(80) will be viewed on host by accessing port(8080)

Note: we did not pull the nginx image before using it , but running "docker images" after the above command is successful returns an nginx image as one of the images present. This is because when executing docker run ..., docker checks for the presence of the image locally and once it can't find it, it pulls it from docker hub and runs it

docker images
Output:
REPOSITORY       TAG               IMAGE ID       CREATED       SIZE
python           3.10-alpine3.18   7d34b8225fda   7 weeks ago   49.7MB
nginx            latest            021283c8eb95   3 weeks ago   187MB

# running containers
docker ps

Output:
CONTAINER ID   IMAGE     COMMAND                  CREATED          STATUS          PORTS                                   NAMES
eefaa7cd973c   nginx     "/docker-entrypoint.…"   11 minutes ago   Up 11 minutes   0.0.0.0:8080->80/tcp, :::8080->80/tcp   helloWorld

Dockerfile

This is a file given the name "Dockerfile" in running applications and is usually used to build applications which contain multiple parts and share them.
In simple terms a docker file is used to make our own personal images.
It contains a list of steps used to bring the app to life.
A docker file has its own structure, always starting with a FROM clause and the rest of the flags can come interchangeably depending on the requirements of the application.
The FROM clause can also appear multiple times to create multiple images.
Example of commands that can come after the FROM clause include:

FROM - statement defining the image to pull
RUN - to execute commands
WORKDIR - define the directory to work from
COPY - create a duplicate o the files or folders present in the host to the container.
ADD - copies new files, folders to the remote directory.
RUN - execute a command
CMD - run a file

More commands like this one can be found in the docker reference

An example of a dockerfile includes:
Assuming we have a folder called Example,with a hello.js file
containing

console.log('hello, world');

Create another file and name it "dockerfile" with the content below:

FROM node:alpine
WORKDIR /app
COPY . .
CMD node hello.js

Docker Build

Once a docker file is defined, the "build" command is used to create the image. View docker build --help to view the flags that can be used with docker build.
The syntax is as shown:

docker build --tag <name_of_image>:<tag> <directory_with_dockerfile>

The tag represents the image versioning which aids in image naming. If none is given, then latest is assumed.It provides more control of the image you create.

Creating the nodejs image above

docker build --tag greetings:latest .
Viewing the images as shown above, we notice a new image with a tag greetings:latest

REPOSITORY       TAG               IMAGE ID       CREATED       SIZE
python           3.10-alpine3.18   7d34b8225fda   7 weeks ago   49.7MB
nginx            latest            021283c8eb95   3 weeks ago   187MB
greetings           latest            531620bb45c5   17 seconds ago   181MB

Running the image:

docker run greetings:latest

Output:
hello, world

.dockerignore

It is used to ignore files that the application does not require to run like node_modules, requirement.txt and .git.
It is created as file in the current directory and the files to be ignored placed inside it

.dockerignore
requirement.txt
node_modules

Docker Registry

As discussed before images are pulled and pushed from docker hub. These images are stored in a repository called docker registry.
That said it is possible to push personal images to docker hub.

First create an account in docker hub.
Run the following command and provide your Docker Hub credentials (username and password) when prompted: docker login
Tag the previously built Docker image with your Docker Hub username and the desired repository name. docker tag greetings your_username/greetings:latest
Finally, push the tagged image to Docker Hub using the following command: docker push your_username/greetings:latest This will push the Docker image to your Docker Hub repository named "greetings" with the "latest" tag.

Docker Inspect

This command allows you to retrieve detailed information about Docker objects, such as containers, images, volumes, and networks. It provides a JSON representation of the specified Docker object, which includes various metadata and configuration details.
The syntax is as shown:
docker inspect <container_name /id >
Pick a container and try it out.

Docker Logs

This command is used to monitor traffic of a running container.
The syntax is as shown:
docker logs <container_name /id >
To follow the logs of the container as they come in add the -f flag before the container name/id.
To add a timestamp to each log when following add the the flags -ft before the container name/id.
For more important flags view
docker logs --help

Docker Network

This command is used to place two or more networks on the same network hence making it easy for them to communicate with each other. For example a postgres container and a pgadmin container that provides a graphical user interface of the data in a postgres database can easily communicate if they are placed on the same network.
The syntax is as shown:
docker network create <network_name>

The two images will have to now contain the flag --network when run in order to communicate eg
docker -e POSTGRES_PASSWORD=pass -p 5432:5432 --network <network_name> -d postgres:latest

Docker Compose

It allows you to define and manage multiple docker containers as one instead of linking them using a network.
This command is used to bring multiple parts of an application,eg frontend, backend and storage to life with just one command.
This is especially useful for development and testing environments, as well as for deploying applications to production.
It is mainly done using a docker-compose.yaml file containing services, networks, and volumes. Each service represents a container, and you can specify their images, ports, volumes, environment variables, etc.
The YAML file should be placed in the root directory of your project.
Ensure you install docker-compose according to your os type before usage. The docker-compose version should also be compatible with your docker version.
An example of a docker-compose file includes:

services:
  db:
    image: postgres:15.3-alpine3.18
    environment:
      - POSTGRES_PASSWORD=<password>
      - POSTGRES_DB=<db_name>
    volumes:
      - "./green:/var/lib/postgresql/data:rw"
    ports:
      - "5432:5432"
  admin:
     image: dpage/pgadmin4
     environment:
        - PGADMIN_DEFAULT_EMAIL=<any@email.com>
        - PGADMIN_DEFAULT_PASSWORD=<pgadmin_password>
     ports:
        - "8080:80"
     volumes:
        - "./pgadmindata:/var/lib/pgadmin:rw"

You can include the version of docker compose you are using eg version:3.8, just above everything else but its not a requirement by default the installed version will be used.

Let's unpack the file above:

A service: contains various parts of your application. It also directs docker on how to build the image required by that service. Each service has to have a unique name that differentiates it from another. We use db as the service title for our postgres database image and admin to reffere to the pgadmin image and the flags it will use to run successfully.

To run the file :
docker-compose up

Use docker ps to view if the containers were created successfully and are up and running.
So generally our docker-compose file runs both postgres and pgadmin which is postgres gui in the same network.

Running https://localhost 8080 provide a pgadmin interface where you can log in using pgadmin credentials and create a server to view the postgres database with on the browser as shown.

Create a server name

Move to connection and use the name of the postgres service as the hostname and include the postgres user and password in the columns indicated

Now you can run

docker exec -it postgres:15.3-alpine3.18
psql -U <postgres_user>
\c <db_name>
CREATE TABLE hello_pdadmin;

Refresh pgadmin to see the changes.
To build the images and not run them, use
docker-compose build
To stop the running containers use
docker compose down

Conclusion

In conclusion, Docker is not just a technology but a mindset that encourages continuous integration and continuous deployment. Whether you're an individual developer, part of a team, or managing large-scale infrastructures, Docker has something to offer for everyone.Embracing Docker's capabilities will undoubtedly enhance your development workflow and bring your projects to new heights of efficiency and reliability. Thank you for staying curious and happy hacking!!

Functions for manipulating data in PostgreSQL

Marriane Akeyo — Thu, 06 Jul 2023 11:58:15 +0000

PostgreSQL is an open source, object-relational database management system (DBMS) that provides robust features and scalability for handling large and complex datasets. It follows the SQL (Structured Query Language) standard and extends it with additional capabilities.Some common data types it supports includes:

Text eg CHAR, VARCHAR and TEXT
Numeric eg INT and DECIMAL
Date/time eg DATE, TIME, TIMESTAMP, INTERVAL
Arrays It also supports JSON, XML, hstore (key-value store), and other specialized data types to handle diverse data requirements. To determine dataypes from existing tables you can query the INFORMATION SCHEMA.

SELECT
     column_name,
     data_type,
FROM INFORMATION_SCHEMA.COLUMNS
WHERE column_name IN ('title', 'description',                                                     'special_features)
AND table_name = 'name_of_your_table';

Array manipulation

It allows you to store and manipulate arrays of values.
Arrays can contain elements of any data type, including built-in types, user-defined types, or even arrays of arrays.

Array Declaration Syntax:
To declare a variable of array type in PL/pgSQL (PostgreSQL's procedural language), you use the same syntax. As shown,
array_name data_type [];

For example:

CREATE TABLE grades (
  email text[][],
  test_scores int[]
);

Array Initialization:
Array literals can be created using the curly braces "{ }" notation. For example, initializing the arrays declared above:

INSERT INTO grades (email, test_scores)
VALUES (
     {{"work","work@gmail.com},{"other","other@gmail.com"}},
     {92,85,56,88}
);

Accessing Array Elements:

SELECT 
     email [1][1] AS type,
     email [1][2] AS email,
     test_scores [1] AS scores
FROM grades;

Note : Postgres arrays start with 1 and not 0

The ANY() function

It is an array operator in PostgreSQL that allows you to compare a value with an array and returns true if the value matches any element of the array. It is commonly used in combination with other comparison operators, such as =, <>, <, >, <=, >= and many more.

The syntax is as follows:
ANY(array_expression)

We can use the ANY() function to manipulate the data selected from the grades table as shown:

SELECT 
     email [1][1] AS type,
     email [1][2] AS email,
     test_scores [1] AS scores
FROM grades
WHERE 'other' = ANY(email);

The array containment operator (@>)

It is used in PostgreSQL to check if an array contains all the elements of another array, verifying if an array is a subset of another array, or filtering rows based on array containment.It evaluates to true if the left-hand array contains all the values in the right-hand array.

The syntax is as shown:
array_expression @> array_to_check
It thus checks if the array represented by array_expression contains all the elements in the array_to_check.

Note: The @> operator checks for containment, meaning it requires the left-hand array to contain all the elements without any duplicates. If you need to check for equality (exact match), you can use the = operator instead.

We can thus obtain the same results above when we use the @> operator.

SELECT 
     email[1][1] AS type,
     email[1][2] AS email,
     test_scores[1] AS scores
FROM grades
WHERE email @> ARRAY['other'];

Both queries will return the same result set, consisting of rows where the "email" array contains the value 'other'. The choice between using ANY() or the @> operator depends on your preference and the specific requirements of your application.

Date/Time manipulation

The date/time data types are used to store and manipulate temporal data.They include:

date: Stores a calendar date (year, month, day). Example:'2023-07-06'
time: Stores the time of day (hour, minute, second). Example: '14:30:45'
time with time zone: Similar to time, but includes time zone information.Example: '14:30:45+05:30'
timestamp: Stores both date and time (year, month, day, hour, minute, second). Example: '2023-07-06 14:30:45'
timestamp with time zone: Similar to timestamp, but includes time zone information.Example: '2023-07-06 14:30:45+05:30'
interval: Represents a duration of time. Example: '1 day 3 hours'

PostgreSQL also provides a rich set of functions and operators for manipulating and performing operations on date/time data. These include functions to extract specific components (e.g., year, month, day) from a date/time value, perform arithmetic operations on dates and intervals, and format dates/times as strings. As we shall see bellow.

We can start with simple calculations such as :

When you subtract two date time values you get an integer datatype.
Subtracting two timestamps gives us an interval.
Adding an integer to a date we get a date value equivalent to the sum of the two. SELECT date '2020-09-11' + integer '3';
You can also manipulate date time data using other arithmetic operations. SELECT timestamp '2023-07-06 14:30:45' + 21 * interval '1 day 3 hours'

AGE()

This function allows us to calculate the difference between two timestamps.

-- table creation
CREATE TABLE sample (
     start_date timestamp,
     end_date timestamp
     );

-- insert into table
INSERT INTO sample (start_date, end_date)
VALUES ('2023-07-01 12:00:00', '2023-07-06 14:30:00');

-- timestamp difference
SELECT AGE(start_date, end_date) AS diff
FROM sample;

Retrieving the current timestamp

The CURRENT_TIMESTAMP and the NOW() functions are used to retrieve the current timestamp in the format 'YYYY-MM-DD HH:MI:SS+TZ'. Both functions provide the current date and time information. You can use these functions in SELECT queries, INSERT statements, or anywhere you need to retrieve the current timestamp in your SQL statements.

CURRENT_TIMESTAMP Function

SELECT CURRENT_TIMESTAMP;

Example Output: '2023-07-06 14:30:45+05:30'

NOW() Function

SELECT NOW();

Example Output: '2023-07-06 14:30:45+05:30'

Casting enables the conversion from one timestamp to another. We can thus get the now value without a timestamp as shown

SELECT NOW() :: timestamp;

You can also use the CAST() function to get the same results.

`SELECT CAST(NOW() as timestamp);

To make things much easier you can use CURRENT_DATE as shown:

SELECT CURRENT_DATE;

To select just the time without date value:

SELECT CURRENT_TIME;

Extracting and Transforming date and datetime data

The EXTRACT() function

Used to retrieve specific components or fields from date or datetime data.
The syntax is as follows:

EXTRACT(field FROM source)

Some examples include:

`
SELECT EXTRACT(MINUTE FROM INTERVAL '3 hours 30 minutes') AS extracted_minute;

-- example two
SELECT EXTRACT(YEAR FROM '2023-07-06'::DATE) AS extracted_year;

-- example three
SELECT EXTRACT(quarter FROM TIMESTAMP '2023-07-06 14:30:00') AS quarter;
`

The DATE_PART() function

It performs the same function as extract but their syntax differ a bit as shown.

DATE_PART(field,source)

Lets redefine the examples above

`
SELECT DATE_PART(MINUTE, INTERVAL '3 hours 30 minutes') AS extracted_minute;

-- example two
SELECT DATE_PART(YEAR, '2023-07-06'::DATE) AS extracted_year;

-- example three
SELECT DATE_PART(quarter, TIMESTAMP '2023-07-06 14:30:00') AS quarter;
`
DATE_ADD
Used to retrieve the next day's date.For example:
Given the table :
------------+-----------+-------------+
| product_id | new_price | date |
+------------+-----------+-------------+
| 1 | 20 | 2019-08-14 |
| 2 | 50 | 2019-08-12 |
| 3 | 35 | 2019-08-16 |
| 4 | 65 | 2019-08-17 |
| 5 | 20 | 2019-08-18 |
+------------+-----------+-------------+

`
SELECT
curr_date,
DATE_ADD(change_date, INTERVAL 1 DAY) AS day_after
FROM your_table_name;

The DATE_TRUNC function

It is used to round off timestamps or interval values to a specified precision. It allows you to remove the smaller units of time from a given date or timestamp while preserving the desired level of precision.
The syntax is as shown:

DATE_TRUNC(unit, source)

Lets look at some examples:

`
-- returns the year in the formart '2023-01-01'
SELECT DATE_TRUNC('year', DATE '2023-07-06') AS truncated_date;

-- returns '2023-07-06 14:00:00'
SELECT DATE_TRUNC('hour', TIMESTAMP '2023-07-06 14:30:45') AS truncated_timestamp;

-- returns '2005-05-01'
SELECT DATE_TRUNC( 'month', TIMESTAMP '2005-05-21 15:30:30');
`

Text manipulation

It is made up of VARCHAR, CHAR and Text data.
Text datatype is considered to be mostly used in any database system and can be manipulated in various ways.

String concatenation

This includes merging of two or more strings together or a non-string data with a string data.
It can be done using pipes(||) or the CONCAT() function.

SELECT first_name, last_name first_name || ' ' || last_name AS full_name FROM customers

The full name column will now contain the last_name and first_name of a customer separated by a space.
The same results can be achieved using the CONCAT() function as shown

SELECT first_name, last_name, CONCAT(first_name ' ' last_name) AS full_name FROM customers;

Concatenating multiple rows

GROUP_CONCAT() is used to concatenate values from multiple rows into a single string. It is often used in conjunction with the GROUP BY clause to aggregate data in a way that combines multiple rows into one, typically for reporting or display purposes.
Consider a simplified example with a table called orders:

sql

+---------+-----------+
| order_id| product |
+---------+-----------+
| 1 | Apple |
| 2 | Banana |
| 3 | Orange |
| 1 | Banana |
| 2 | Apple |
+---------+-----------+

If you want to concatenate the distinct product names for each order, sorted lexicographically and separated by a comma, you can use:

SELECT
order_id,
GROUP_CONCAT(DISTINCT product ORDER BY product ASC SEPARATOR ', ') AS products
FROM orders
GROUP BY order_id;

The result would be:

+---------+-----------------+
| order_id| products |
+---------+-----------------+
| 1 | Apple, Banana |
| 2 | Apple, Banana |
| 3 | Orange |
+---------+-----------------+

Changing the case of a string

Some commonly used functions to change the case of strings include:
-** UPPER()**: converts a string to uppercase. For example:

SELECT UPPER('Hello World') AS upper_case;

Output: 'HELLO WORLD'

LOWER(): converts a string to lowercase. For example:

SELECT LOWER('Hello World') AS lower_case;

Output: 'hello world'

INITCAP(): converts a string to camel case. For Example:

SELECT INITCAP('hello world') AS init_cap;

Output: 'Hello World'

Replacing a string

The REPLACE() function locates a string and replaces it with another string.
For example:

SELECT REPLACE (currency, 'dollars', 'shillings) AS new_currency FROM salary

The above query replaces all the dollar strings to shillings in the currency column and selects them as new_currency.

Parsing string and character data

The CHAR_LENGTH() function
This function is used to determine the number of characters in a string.
Syntax:

CHAR_LENGTH(string)

Example:
Assuming we have a table called film with film titles and descriptions, we can find the title length as shown:

SELECT title, CHAR_LENGTH(title) AS len FROM films;

The LENGTH() function can be used interchangeably with CHAR_LENGTH() function to produce similar results, and is mostly preferred.

SELECT title, LENGTH(title) AS len FROM films;

The POSITION() function
It is used to return the position of a string from left to right.
Syntax: POSITION('string' IN source)
Example:
Selecting from the table created in Array manipulation above

SELECT email POSITION('@' IN email) FROM grades;

The STRPOS() function is an alias of POSITION() with a slightly different syntax as shown:

STRPOS(source, 'string')

For example:

SELECT email STRPOS(email, '@') FROM grades;

The LEFT() function
Allows us to select a set of characters from the left of a string.
Syntax:

LEFT(column, number_limit)

Example:
Considering a film table with title and description columns

SELECT LEFT(description, 50) FROM film;

The RIGHT() function
Allows us to select a set of characters from the right side of a string.
Syntax:

RIGHT(column, number_limit)

Example:
Considering a film table with title and description columns

SELECT RIGHT(description, 50) FROM film;

The SUBSTRING() function
This function selects characters in between a string.
Syntax:

SUBSTRING(column, start, end)

Example:

SELECT SUBSTRING(description, 10, 50) FROM film;

The substring function can be combined with other functions to provide more context.For example selecting the name before @ in an email

`
-- before @
SELECT
SUBSTRING(email, 0, POSITION('@' IN email)) AS name
FROM grades;

-- after @
SELECT
SUBSTRING(email, POSITION('@' IN email)+1 FOR LENGTH (email) ) AS domain
FROM grades;
or
SELECT
SUBSTRING(email, FROM POSITION('@' IN email)+1) AS domain
FROM grades;
`

The SUBSTR() is another alternative of SUBSTRING but does not allow other functions in it or the FOR keyword.

SELECT SUBSTR(description, 10, 50) FROM film;

The TRIM() function
It is used to remove whitespaces from strings.
It follows the syntax

TRIM([leading|trailing|both] [characters] from string)

If the first parameter is not given, the default value is both.
If the second parameter is not given the default value is blank lines.
Most of the times only the string is passed.

SELECT TRIM(' john doe ')

Output: 'john doe'

Other trim like functions include:
LTRIM() - which removes whitespaces only from the beginning of the string.

SELECT LTRIM(' john doe ')

Output: 'john doe '

RTRIM() - which removes whitespaces at the end of the string

SELECT TRIM(' john doe ')

Output: ' john doe'

The LPAD and RPAD functions
They append a spesified number of characters to a string with LPAD() appending to the left of a string and RPAD() to the right of a string.
Syntax: LPAD/RPAD(string, no_of_times, string_to_be_added)
Example:

LPAD('food', 10, '*');

If no 'string_to_be_added' is provided then the string is padded by white spaces.

Text searching
The LIKE character as learnt in introduction to sql is used to match characters and strings.
We can use _ to match exactly one character or % to match more than one character as shown.

SELECT title FROM film WHERE title LIKE 'Queen%'; or WHERE title LIKE '%Charlotte';

Either of the where clauses will return the title 'Queen Charlotte'
However, LIKE is case sensitive and might not give us accurate results if we are performing a full text search and is also quite expensive in terms of performance.
We can therefore utilize the to_tsvector() and to_tsquery() in our where clause which perform natural language queries of text and data in the database, as shown:

SELECT title FROM film WHERE to_tsvector(title)@@to_tsquery('Queen');

tsvector vectorises the title and tsquery searches for the subquery within the vectors.

Extending the Capabilities of PostgreSQL

The CREATE_TYPE command
It is used to create user defined datatypes with specific attributes and behaviors.
It registers the type in a postgres db and can be used where postgres expects a type name.

Creating a Composite Type They are user-defined types that can contain multiple fields or attributes. Syntax:

CREATE TYPE type_name AS ( attribute1 datatype1, attribute2 datatype2, ... );

Example:

CREATE TYPE person_type AS ( name text, age integer, email text );

Usage:

`
-- create a table using the composite type
CREATE TABLE persons (
id serial primary key,
person person_type
);

-- insert into the table
INSERT INTO persons (person)
VALUES (('John Doe', 30, 'john@example.com'));

-- query the table
SELECT person.name, person.age, person.email
FROM persons;
`

Creating an Enumerated Type: It allows you to define a fixed set of values that a column or variable can take. Syntax:

CREATE TYPE type_name AS ENUM ('value1', 'value2', ...);

Example:

CREATE TYPE day_of_week AS ENUM ('Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday', 'Saturday', 'Sunday');

Once created, these custom types can be used in table definitions, column definitions, function parameters, and more.

User defined functions
They allow you to create custom functions tailored to your specific requirements. You can define functions that encapsulate complex logic, perform calculations, manipulate data, or provide any other functionality you need.

Syntax:

CREATE FUNCTION function_name (parameter1 data_type1, parameter2 data_type2, ...) RETURNS return_type AS $$ -- Function body goes here BEGIN -- SQL statements or other logic RETURN result_value; END; $$ LANGUAGE plpgsql; -- or other supported language like SQL, Python, etc.

Example:

`
CREATE FUNCTION square (i integer)
RETURNS integer
AS $$
BEGIN
RETURN i * i;
END;
$$
LANGUAGE plpgsql;

-- you then can use it in SQL queries just like any built-in function
SELECT square(5) AS square_result;
`

LEVEINSTEIN() - claculates the distance required to make 2 strings simmilar in values between 0 and 1 , 1 being most accurate.

SIMILARIRY()- claclulates how simmilar or different two strings are from each other

Postgres Extensions
PostgreSQL provides a command called CREATE EXTENSION to install an extension.
Syntax:

CREATE EXTENSION extension_name;

Example:

CREATE EXTENSION hstore;

Extensions can also be installed using package managers, such as apt, yum, or brew, depending on the operating system.
Some popular extensions include:

PostGIS: Adds support for geographic objects and spatial indexing, enabling the use of PostgreSQL as a geospatial database.
pgcrypto: Provides cryptographic functions for secure data storage and retrieval.
pg_trgm: Offers fuzzy string matching and indexing capabilities.
hstore: Allows storing key-value pairs within a single column, providing a basic NoSQL-like functionality.
uuid-ossp: Generates universally unique identifiers (UUIDs) using various algorithms.
pg_stat_statements: Collects statistics about SQL statements executed in a database, aiding in query optimization.
PostPic: allows for image processing within the db

To view extensions available for install query

SELECT name FROM pg_available_extension;

To view already installed extensions

SELECT extname FROM pg_extension;
You can read more about the usage of postgres extensions in the postgreSQL documentation.

Thank you for reading, I hope this helps you dive deeper into postgres. Stay curious!!! and Happy hacking.

SQL 102:Intermediate SQL

Marriane Akeyo — Mon, 17 Apr 2023 08:50:37 +0000

This post is a continuation of the Introduction to SQL. However, if you are familiar with basic SQL concepts you can take a read through this article for more advance concepts.

CASE WHEN AND THEN

Evaluates a set of conditions just like an if else statement and return a corresponding result when the condition is true.
The corresponding syntax is as shown:

SELECT column1,
       column2,
       CASE WHEN condition1 THEN result1
            WHEN condition2 THEN result2
           ...(other when conditions)
           ELSE default_result
       END AS new_column_name
FROM table_name;

Lets look at a more concrete example. Assuming we have a matches table with different teams and how they played and we want to determine if the won when they played at home, we shall have the following query.

SELECT id,
       name,
       CASE WHEN home_goal > away_goal THEN 'Its a home win'
            WHEN home_goal < away_goal THEN 'Its a home loss'
            ELSE 'Its a tie'
       END AS home_scores
INTO home_analysis
FROM matches;

The output of the above code is three columns id, name and home_scores which are stored inside the home_analysis table.
We used the CASE statement to filter the results of the home_scores column.

You can also use a CASE statement to aggregate data based on the results of a WHERE clause, since aggregate functions cannot be used directly in a WHERE clause.

Let's say you have a table called "employees" with columns "name", "salary", and "department". You want to create a report that shows the total salary for each department, and categorize each department as either "High Paying" or "Low Paying" based on the total salary. The results should include only employees hired on or after January 1, 2023

SELECT department,
       SUM(salary) as total_salary
       CASE WHEN SUM(salary) >= 1000000 THEN 'High paying dpt'
            ELSE 'Low paying dpt'
       END AS dpt_salary
FROM employees
WHERE hire_date >= '2023-01-01'
GROUP BY department;

The sum of the salaries will only be placed in any category if they satisfy the where clause first.

SUBQUERIES

They are queries found inside other queries hence also known as nested queries.The subquery is executed first, and the results are then used in the main query. This allows you to perform more complex queries that involve multiple tables and conditions.
The syntax is as shown:

SELECT column
FROM (SELECT column
      FROM table) AS subquery

A subquery can be placed at any part of your query, such as the SELECT, FROM, WHERE or GROUP BY
The allow you to compare summarized values to detailed data and also reshape your data as desired.
A subquery is also used to combine data that cannot be joined.

A subquery in the where clause

It is used to filter data based on the results of another table.For example, if you have a table of customers and a table of orders, you can use a subquery to find all customers who have made at least one order.

SELECT *
FROM customers
WHERE id IN (SELECT c_id FROM orders);

A subquery in the FROM clause

They are used to restructure and transform the data to be selected. Ensure you alias the subquery for easy reference.
Lets use the customer and order table for reference and see the orders for each customer in the 2022/2023 season.

SELECT name,
       order
FROM (SELECT c.name AS name 
             o.id AS order
      FROM customers AS c
      INNER JOIN orders AS o
      ON c.order = o.id)
      AS customer_order
WHERE season = '2022/2023';

We can also create more than one subquery inside a from statement.

SELECT customers.customer_id,
      customers.first_name, '
      customers.last_name, 
      orders.total
FROM (
   SELECT customer_id, 
          SUM(price * quantity) AS total
   FROM order_items
   GROUP BY customer_id
   ) AS orders
INNER JOIN customers 
ON orders.customer_id = customers.customer_id
WHERE orders.total > (
   SELECT AVG(total)
       FROM (
          SELECT customer_id, 
                 SUM(price * quantity) AS total
          FROM order_items
          GROUP BY customer_id
         ) AS order_totals
)

In this example, we're using two subqueries in the FROM clause. The first subquery calculates the total amount spent by each customer on all of their orders. The second subquery calculates the average total amount spent by all customers.

The main query then joins the orders subquery with the customers table to retrieve the customer's ID, first name, and last name, as well as their total amount spent. Finally, the WHERE clause filters the results to only include customers whose total amount spent is greater than the average total amount spent by all customers.

Subquries in the select statement

They can be used to perform calculations based on data from another table. For example, if you have a table of orders and a table of products, you can use a subquery to calculate the total price of each order.

SELECT id, 
       (SELECT SUM(price * quantity)
        FROM products 
         WHERE order.id = products.order_id) AS total_price
FROM orders;

It can also be used to retrieve information from a table as we saw above we got the total_price of our commodities from products table.
The big question however is, when is it advisable to use subqueries considering that each subquery requires additional computing power. There are certain factors to consider when selecting a subquery:
1.Data complexity
Subqueries are useful when working with complex data queries that require multiple tables and conditions. If the query involves multiple subqueries or multiple nested subqueries, it can become very difficult to read and maintain. In this case, it may be better to break the query into smaller parts or consider using a different approach.

2.Data volume
Subqueries can be slow to execute when dealing with large datasets. If the query involves a large amount of data, it may be better to use a join or a different type of query.

3.Performance
The performance of a subquery can depend on the database engine being used. Some database engines can optimize subqueries for better performance. It's important to consider the performance implications when deciding whether to use a subquery.

4.Maintainability
Subqueries can make queries more difficult to read and maintain. It's important to consider whether the use of a subquery will make the query more or less maintainable in the long run.
5.Query reuse
If the same subquery is going to be used multiple times in different parts of a query or across multiple queries, it may be more efficient to create a view or a temporary table instead of using a subquery.

Corelated Subqueries

It is a type of subquery which uses the values from the outer query to generate results, by referencing one or more columns in the main query. It is always re-run for every new row generated.

SELECT *
FROM employees AS e
WHERE e.salary > (
     SELECT AVG(salary)
     FROM employees
     WHERE dpt_id = e.dpt_id
)

It is also very possible to have a nested subquery. Lets select an employee's name and the average salary he received in the first quarter of the year.

SELECT
     e.name AS employee,
     (SELECT AVG(jan_pay + feb_pay + mar_pay),
      FROM salary AS s
      WHERE s.empl_id = e.id
          AND id IN (
               SELECT id 
               FROM salary 
               WHERE year == 2023
               ) AS avg_salary
FROM employees AS e
GROUP BY employee;

The subquery in the SELECT clause is dependent on the equality of the salary and the employees id, which is also dependent on the year column in the salary table.

Common Table Expressions(CTEs)

Subquries are a good way of bringing multiple unrelated tables together and also simplifying calculations. However, having multiple nested subqueries can be hard to read and understand as discussed above. That is why a solution to this can be using CTEs.
CTEs are declared ahead of the main query using a WITH statement and referenced later in the FROM statement.
It can be referenced within a SELECT, INSERT, UPDATE, or DELETE statement in SQL.

WITH s AS (
   SELECT emp_id, id,
         AVG(jan_pay + feb_pay + mar_pay) AS q1
   FROM salary
   WHERE year == 2023
   GROUP BY id
)
SELECT e.name AS employee,
       s.q1
FROM employees AS e
INNER JOIN s
ON e.id = s.emp_id

CTEs are better than subqueries because they are run only once and then stored in memory, hence reducing the amount of time required to run the query.
You can also have multiple CTEs, separated by a comma but make sure you join them in their respective order.

WITH max_salary AS (
  SELECT department_id, MAX(salary) AS max_salary
  FROM salary
  GROUP BY department_id
), 
avg_salary AS (
  SELECT department_id, AVG(salary) AS avg_salary
  FROM salary
  GROUP BY department_id
)
SELECT employees.*, max_salary.max_salary, avg_salary.avg_salary
FROM employees
JOIN max_salary ON employees.department_id = max_salary.department_id
JOIN avg_salary ON employees.department_id = avg_salary.department_id
WHERE employees.salary = max_salary.max_salary

Window Functions

Perform a set of operations that are somehow related to the current row, similar to group by but rows are not groupd into a single row.
They are called "window" functions because they operate on a "window" of rows, defined by an OVER clause, rather than on individual rows.
Some use cases of window functions include:

Fetching values before and after the current row.
Ranking row values.
Calculating averages See the example below:

SELECT transaction_id, 
       sales_amount, 
       SUM(sales_amount) OVER() AS total_sales
FROM sales_transactions

This sums the sales_amount per amount and displays it as total_sales.The first row will contain its own value in the total sales column, the second row will contain a sum of the first and second row and so on.
At any given point we might want to give each of our rows a value in order to easily access it. This can be done using the
ROW_NUMBER() function.

SELECT transaction_id, 
       sales_amount, 
       ROW_NUMBER() OVER() AS row_n
FROM sales_transactions

The output of the above query will contain an extra row called row_n with row numbers from 1 and so on.
It is possible that we want the values in the row_n columns to be ordered in a certain way. We can thus use the ORDER BY clause in OVER() function as shown:

SELECT transaction_id, 
       sales_amount, 
       ROW_NUMBER() OVER(ORDER BY sales_amount DESC) AS row_n
FROM sales_transactions

The row_n values will now be sorted starting from the one with the highest sales order to the lowest.

We can generate ranks instead of ordering as well using the RANK() window function.

SELECT transaction_id, 
       sales_amount, 
       SUM(sales_amount) RANK() OVER() AS total_sales
FROM sales_transactions

Note that window functions are produced after other queries have been processed. It uses the generated table to find its results.
As stated earlier we can also use window functions to compare the current row and the one before it. This is made possible using the LAG() function. Example:

Assuming we have a games table with year, champion,medals,match and other columns in it. We can find out which champion won recurrently.

SELECT 
      year,match, champion
      LAG(champion) OVER(
      ORDER BY year ASC) AS last_champion
FROM games
ORDER BY year ASC;

We can now compare the champion and last_champion columns and find recurrent champions.
The above query will produce the values for last_champion regardless of weather the matches are similar or not. For example if we had matches for hockey and rollball within the same year it will not provide the last_champion for each separate match but just generalize them.
To solve this , we use a PARTITION BY clause as shown below.

SELECT 
      year,match, champion
      LAG(champion) OVER(
      PARTITION BY match
      ORDER BY year ASC) AS last_champion
FROM games
ORDER BY year ASC;

Window partitions also group the result set of a row into a smaller group so that window functions can be applied to those subsets separately.
Lets look at another example:

SELECT department_id, 
       employee_id, 
       salary, 
       AVG(salary) OVER (PARTITION BY department_id
                              ) AS avg_salary
FROM employees

The results will be first partitioned then average salary calculated.

Sliding Windows

They enable us to perform calculations relative to the current window of the dataset.
The window is defined by a rage of rows that "slide " or move through the partition based on a specified ordering.
The syntax is as shown:

ROWS BETWEEN <start> AND <finish>

The start and finish can be replaced with PRECEDING(comes before), FOLLOWING(comes after), UNBOUNDED PRECEDING(all rows in the partition up to and including the current row), UNBOUNDED FOLLOWING, CURRENT ROW

SELECT date,
       jan_pay,
       SUM (jan_pay)
       OVER(ORDER BY date ROWS BETWEEN
            UNBOUNDED PRECEDING AND CURRENT ROW)
       AS jan_total
FROM salary
WHERE year = 2023;

Fetching

They include functions that enable us to get certain values from different parts of the table or partition into one row.
The LAG() function discussed above is also one of them.
The LEAD() function is used return the values after the current row. Lets use the LAG() example shown above.

SELECT 
      year,match, champion
      LEAD(champion) OVER(
      ORDER BY year ASC) AS following_champion
FROM games
ORDER BY year ASC;

The above query compares the current champion and the champion after him.
FIRST_VALUE returns the first value in the table or partition.

SELECT 
      year,match, champion
      FIRST_VALUE(champion) OVER(
      ORDER BY year ASC) AS first
FROM games
ORDER BY year ASC;

LAST_VALUE returns the last value in the table or partition.

SELECT 
      year,match, champion
      LAST_VALUE(champion) OVER(
      ORDER BY year ASC) AS last
FROM games
ORDER BY year ASC;

NOTE

A sliding window can also be defined as a frame. Without frame the LAST_VALUE() function would return the current row. Efficient example of the LAST_VALUE() would be:

SELECT 
      year,match, champion
      LAST_VALUE(champion) OVER(
      ORDER BY year ASC
      ROWS BETWEEN UNBOUNDED PRECEDING
      AND CURRENT ROW)) AS last
FROM games
ORDER BY year ASC;

There are two types of frames, RANGE BETWEEN and ROWS BETWEEN. RANGE BETWEEN however lists duplicate values when used with the OVER's ORDER BY subclause as a single entity. Hence ROWS BETTWEEN is mostly used over RANGE BETWEEN.

Ranking

It involves arranging the data in a partition or table in a particular order.
The ranking functions include:
ROW_NUMBER(): It assigns each row a unique number even if the row values are the same. Check the window functions example for more clarification.

RANK(): It orders the values of a table or partition giving rows with identical values the same number and skipping one number over to the next value.

DENSE_RANK(): It ranks values in a particular table or partition giving the same number for identical values but does not skip a number to the next value, hence mostly preferred.

SELECT transaction_id, 
       sales_amount, 
       SUM(sales_amount) DENSE_RANK() OVER() AS rank_sales
FROM sales_transactions

This will rank the total sales giving identical values the same rank then move to the next total_sales value without skipping a number.

transaction_id	sales_amount	rank_sales
iutio	64	1
ghjkl	48	2
frtyu	48	2
lkjhg	36	3

Paging

This includes splitting data into approximately equal chunks which enables us to easily analyze data piece by piece.
Paging in sql is done using the NTILE(n) which splits data into approximately n equal pages.

SELECT transaction_id, 
       sales_amount, 
       NTILE(15) OVER() AS page1
FROM sales_transactions

In the above query the rows are divided into 15 buckets when called.

Pivoting

It makes it easier to understand and analyze data by converting columns into rows. It is very valuable when preparing data for visualization.

CREATE EXTENSION IF NOT EXISTS tablefunc;
SELECT * FROM CROSSTAB ( $$
     SELECT country,
            year,
            COUNT(*) :: INTEGER AS awards
     FROM medals
     WHERE year IN (2022, 2023)
     GROUP BY country, year
     ORDER BY country ASC;
      $$) AS ct ( country VARCHAR, 
                 "2022" INTEGER,
                  "2023" INTEGER)
 ORDER BY country ASC;

The above query converts the country and year column in the medal table into rows and splits the year into 2022 and 2023.
Each row is also given their respective datatype.

Conclusion

In conclusion, this article has covered several intermediate SQL topics that can help you gain a deeper understanding of SQL and data manipulation. By mastering these concepts, you will be able to write more complex SQL queries and work with large datasets more efficiently. I hope you found this article helpful in your SQL journey and that it has provided you with valuable insights and knowledge. Keep practicing and exploring the world of SQL to become a proficient data analyst or developer. Stay curious and stay golden!

SQL101: Introduction to SQL

Marriane Akeyo — Fri, 17 Feb 2023 06:27:12 +0000

Structured Query Language(SQL) is a database language that is used to manage and retrieve data stored in a database, such that it can be easily accessed and manipulated. It is a common language recognized by Database Management Systems(DBMS) like MYSQL. We are going to make reference to relational databases since they are mostly used.

Relational database:Store data in form of tables. As the name suggests, a relation ties two records or tables together.
Table: made up of rows and columns.
Rows: A single record in a table.eg one person's details in a User's table.
Column: value of a particular type eg First_Name column in table person.
Statement: A text that is uniformly recognized as a valid command by the database.They usually end with a semicolon.

CREATE TABLE users(
id INTEGER,
first_name VARCHAR(60),
last_name VARCHAR(60)
);

Note:SQL clauses like CREATE and TABLE are usually written in capital letters.
SQL syntax are divided into data definition language and data manipulation language.

Data Definition Language

They are used to build and modify the structure of your tables,views, indexes and other objects in the database. Examples include:

CREATE TABLE statement: creates a new table with the specified columns and data types.
CREATE DATABASE statement: creates a new database.
ALTER TABLE statement: used to add, modify or delete columns in an existing table.
ALTER DATABASE statement: modifies an existing database.
DROP TABLE statement: used to delete an existing table and all of its data.
CREATE INDEX statement: creates an index (search key)

Data Manipulation Language

They include statements used to work with data in the database.Some examples include:

INSERT Statement:used to insert new data into a table.
UPDATE Statement:used to modify the existing data in a table.
DELETE Statement:used to delete data from a table.
SELECT Statement:used to retrieve data from one or more tables.

We shall look at examples of these statements in details.

Creating a database

Lets now create a database named customers that we shall be working from.The syntax is as shown:

CREATE DATABASE database_name;

Now to our database
CREATE DATABASE customers;

Some databases might require you to use (``) when defining database schemas eg creating tables and databases.As shown below:

` CREATE DATABASE `customers`; `

We can also be able to view all the databases we have created using the following command:

SHOW DATABASES;

Assign a user to our database

Once you are able to access your database either through MySQL prompt or any other way, it is advisable to create a new user in order to avoid using the root user and causing damage to the whole database in general.
Syntax:

CREATE USER 'username'@'host' IDENTIFIED BY 'password';

Lets create a user named marrie, at localhost, with a password pass_123.

CREATE USER 'marrie'@'localhost' IDENTIFIED BY 'Pass_123';

We can now grant our new user some privileges. The syntax is as shown:

GRANT PRIVILEGE ON database.table TO 'username'@'host';

Lets grant our user access to the customers database.

GRANT PRIVILEGE ON customers.* TO 'marrie'@'localhost';

It’s good practice to run the FLUSH PRIVILEGES command. This will free up any memory that the server cached as a result of the preceding CREATE USER and GRANT statements:

FLUSH PRIVILEGES;

Working in our current database

Now that we have our database assigned to a user, we can start issuing commands to create tables and insert our data in the tables. First we move to our database.
Syntax:
USE database;

Example:
USE customers;

Creating a table

Once inside our customers database , we can now be able to create various tables as discussed above.

` CREATE TABLE table_name ( column1 datatype constraint, column2 datatype constraint, column3 datatype constraint ); `

Lets create a table users as an example.

` CREATE TABLE users ( id INT PRIMARY KEY, first_name VARCHAR(50) NOT NULL, last_name VARCHAR(50) NOT NULL, email VARCHAR(100) UNIQUE, birth_day INT, born_day DATE() ); `
You can also confirm if the table is created using the following command

SHOW TABELS;

Note that the exact syntax and behavior of the SHOW TABLES statement may vary slightly depending on the specific SQL database management system being used.

Alter table

We can also change the outlook of the table in case we want to add, delete, or modify columns, constraints, indexes, and other attributes of a table.
Syntax:

ALTER TABLE table_name action;

Where table_name is the name of the table you want to modify, and action is the specific alteration you want to make to the table.
Example:

Add a new column to an existing table:

` ALTER TABLE users ADD COLUMN age INT; `
Modify an existing column in a table:

` ALTER TABLE users MODIFY birth_day DATE(); `

Delete a column from an existing table:

` ALTER TABLE user DROP COLUMN age; `

Add a primary key to a table:

` ALTER TABLE user ADD CONSTRAINT PRIMARY KEY (id) `

Note that the exact syntax and behavior of the ALTER TABLE statement may vary slightly depending on the specific SQL database management system being used. Also, it's important to be careful when using ALTER TABLE, as modifying the structure of a table can affect the data that is stored in the table.

Inserting data in a table

We have our table, we can now feed it with some data.
Syntax:

INSERT INTO table_name (column_name_1, column_name_2, column_name_3)VALUES (value_1, value_2, value_3);

Note that the order of the values in the "VALUES" clause must match the order of the columns in the "INSERT INTO" statement.
Example:

INSERT INTO users (id, first_name, last_name, email, birth_day, born_day) VALUES (1, 'John', 'Doe', 'johndoe@example.com', '1980-01-01', '2000-01-01');

You can also insert multiple rows at once by separating each set of values with a comma, like this:

` INSERT INTO users (id, first_name, last_name, email, birth_day, born_day) VALUES (1, 'John', 'Doe', 'johndoe@example.com', '1980-01-01', '2000-01-01'), (2, 'Jane', 'Kim', 'janekim@example.com', '1985-02-02', '2005-02-02'), (3, 'Bob', 'Smith', 'bobsmith@example.com', '1990-03-03', '2010-03-03'); `
This would insert three new rows into the "users" table, one for each set of values.

Viewing the data in the table.

Now lets verify that we added something to our table.

SELECT * FROM users;

You will notice that the above displays all the data in the users table.
We can also select a specific column from the table.

SELECT first_name, last_name FROM users;

We can also place a condition within which the data we want can be collected.

SELECT first_name, last_name FROM users WHERE birth_day='1980-01-01';

Updating the data in the database

It is also possible to modify the data we inserted in the table in case you need to change the data already existing in the database table.
Syntax:

` UPDATE table_name SET column1 = value1, column2 = value2, ... WHERE condition; `
Lets change the name of John Doe to Joe Doo

` UPDATE users SET first_name = 'Joe', last_name = 'Doo' WHERE email = 'johndoe@example.com'; `

Deleting a record from the table

We can also remove records from a table.
Syntax:

` DELETE FROM table_name WHERE condition; `
Example

` DELETE FROM users WHERE born_day < 2010-03-03; `

SQL Constraints

Constraints ensure that the data in a table is consistent and accurate by limiting the type of data that can be inserted into a table, or by defining relationships between tables. Here are some examples of constraints in SQL:

NOT NULL This constraint ensures that a column cannot have a NULL value. Example:

ALTER TABLE users MODIFY last_name VARCHAR(60) NOT NULL

PRIMARY KEY This constraint uniquely identifies each record in a table. A primary key column cannot have NULL values, and the values in the column must be unique. Example:

ALTER TABLE users ADD PRIMARY KEY (id)

FOREIGN KEY
This constraint establishes a relationship between two tables by defining a column in one table as a foreign key that refers to the primary key of another table.
` ALTER TABLE users ADD CONSTRAINT fk_customer FOREIGN KEY(age) REFERENCES customers(id); `
UNIQUE
This constraint ensures that the values in a column are unique.
ALTER TABLE users MODIFY email VARCHAR(100) UNIQUE
CHECK
This constraint allows you to specify a condition that must be true for a record to be inserted or updated.
ALTER TABLE customers ADD CONSTRAINT ck_age CHECK (age >= 18);
ENUM
This constraint contains a string object with a value chosen from a list of permitted values.

` -- Creating the table CREATE TABLE shops ( id INTEGER PRIMARY KEY, name VARCHAR (55), quality ENUM ('High','Average','Low') ); -- Insert into the table INSERT INTO shops VALUES(1 'Johny' 'Low'); INSERT INTO shops VALUES(1 'Doe' 'Average'); INSERT INTO shops VALUES(1 'Marrie' 'High'); `

SET A set can have zero or more values chosen from a list of permitted values.

` CREATE TABLE students ( id INTEGER PRIMARY KEY, name VARCHAR(60), cert SET('A1','A2','B1','B2') ); `
Each student can have O,1 or more of these certificates. In ENUM however, you can have only one distinct value.
INSERT INTO students VALUES(1, 'Paul', A1,B1,B2);

Aggregate Functions in SQL

They are used to perform various operations on the data stored , hence enabling us to compute data not found in the database.
Some of the most commonly used functions include:

AVG():It returns the average value of a numeric column.

` Syntax: SELECT AVG(column_name) FROM table_name; Example: SELECT AVG(age) FROM users; `

COUNT():It returns the number of rows in a table or the number of non-null values in a column.

` Syntax: SELECT COUNT(column_name) FROM table_name; Example: SELECT COUNT(*) FROM users; `

MAX():It returns the maximum value in a column.

` Syntax: SELECT MAX(column_name) FROM table_name; Example: SELECT MAX(age) FROM users; `

MIN():It returns the minimum value in a column.

` Syntax: SELECT MIN(column_name) FROM table_name; Example: SELECT MIN(age) FROM users; `

SUM():It returns the sum of values in a numeric column.

` Syntax: SELECT SUM(column_name) FROM table_name; Example: SELECT SUM(age) FROM users; `

UPPER(): It converts a string to uppercase.

` Syntax: SELECT UPPER(column_name) FROM table_name; Example: SELECT UPPER(first_name) FROM user; `

LOWER(): It converts a string to lowercase.

` Syntax: SELECT LOWER(column_name) FROM table_name; Example: SELECT LOWER(last_name) FROM employees; `

SUBSTRING():It extracts a sub string from a string.

` Syntax: SELECT SUBSTRING(column_name, start_position, length) FROM table_name; Example: SELECT SUBSTRING(first_name, 1, 3) FROM user; `

CONCAT():It concatenates two or more strings.

` Syntax: SELECT CONCAT(string1, string2, ...) FROM table_name; Example: SELECT CONCAT(first_name, ' ', last_name) FROM users; `

DATE():It returns the current date.

` Syntax: SELECT DATE(); Example: SELECT DATE(); `

MONTH():It returns the month of a date.

` Syntax: SELECT MONTH(date_column) FROM table_name; Example: SELECT MONTH(birth_day) FROM users; `

YEAR():It returns the year of a date.

` Syntax: SELECT YEAR(date_column) FROM table_name; Example: SELECT YEAR(birth_day) FROM users; `

DATEDIFF():It calculates the difference between two dates.

` Syntax: SELECT DATEDIFF(date1, date2) FROM table_name; Example: SELECT DATEDIFF(born_date, birth_day) FROM employees; `

NOW():It returns the current date and time.

` Syntax: SELECT NOW(); Example: SELECT NOW(); `

The where clause

This clause is used to filter sql statements base on a certain condition or set of conditions.It is used in conjunction with the SELECT, UPDATE, DELETE statements to filter data from tables. Forexample:

SELECT * FROM people WHERE age >= 18;

The above example will select all the data for people above 18 years from the database.

The AND clause

We can also combine our where clause with an AND clause in order to select data based on multiple conditions. Lets view data about all developers above 18 years.

` SELECT * FROM people WHERE age >= 18 AND job = 'dev'; `
We can also improve our comparison and add the OR clause as shown.

` SELECT * FROM people WHERE (age >= 18 OR age < 40) AND (job = 'dev' OR job = 'IT'); `

The BETWEEN clause

Sometimes it might be necessary to return values within a specified range, inclusive of the start and end values.
We can modify the example above as shown.

` SELECT * FROM people WHERE age BETWEEN 18 AND 40; `
We can also combine the BETWEEN clause with AND / OR. Now lets pick data for all products with prices between 10 and 20 0r 30 and 40.

` SELECT * FROM products WHERE (price BETWEEN 10 AND 20) OR (price BETWEEN 30 AND 40); `

The IN and NOT IN clauses

The IN clause is used to specify multiple values in a WHERE clause.Lets select name and age of people with 18,19 and 20 years of age.

` SELECT id, name, age FROM people WHERE age IN (18, 19, 20); `
The NOT IN clause fulfills the opposite of an IN clause. Lets return data of people and their ages and name except those who are 18, 19 and 20.

` SELECT id, name, age FROM people WHERE age NOT IN (18, 19, 20); `

The LIKE clause

It can be used in the WHERE clause to search for a pattern in a column.
The '%' pattern matches 0, 1 or many characters in a text.
The '_' pattern matches a single character.
We can have a table called employee with the data shown bellow

id	name	department
1	John Smith	IT
2	Jane Doe	dev
3	Mike Brown	Sales
4	Sarah Lee	dev
5	Bill Wong	IT

We can use the '%' pattern to select employees whose name contain 'i' character.

` SELECT * FROM employees WHERE name LIKE '%i%'; `
This prints

name
Mike Brown
Bill Wong

We can also select employees whose name begins with 'J' and contain 'ohn'.

` SELECT * FROM employees WHERE name LIKE '_ohn'; `
This prints

name
John Smith

NOT LIKE - is used to find patterns that do not match the records specified.

The GROUPBY clause

Used to put together rows with the same value in one column or set of columns and then perform aggregate functions (such as SUM, COUNT, AVG, MIN, MAX) on the grouped data.
Now lets group the people column according to their ages

` SELECT age, COUNT(*) as total_age FROM people GROUP BY age; `

age	total_age
18	26
19	11
20	6
32	8
40	12

All the aggregate functions discussed above can be used with the GROUPBY clause. The column to be grouped by MUST be selected as well otherwise you will get an error.

ORDER BY clause

It sorts the results of a query in ascending or descending order based on the selected columns.
We can select the name and age columns from people and order them by their age in descending order.

` SELECT name, age FROM people ORDER BY age DESC; `

Note that by default the columns are ordered in ascending order. So you can remove the 'DESC' to maintain the same order.

The GROUP BY and ORDER BY even the WHERE clause can be used in one SELECT query. When this happens, make sure the ORDER BY clause comes after the GROUP BY. For example:

` SELECT name, age, count(age) as total_age FROM people WHERE (age = 18 OR age =19) GROUP BY age, name ORDER BY age, name DESC; `
In the query above we have grouped the people's data by age and name and arranged them in descending order.

The HAVING clause

It is used to specify the conditions that must be met with the group in order to be included in the query result. It also helps us to filter based on the result of an aggregate functions which cannot be used in a WHERE clause.

` SELECT age, SUM(*) FROM people GROUP BY age HAVING SUM(*) > 100; `

The query above prints the sum of each age category in the database with a sum greater than 100.

SQL JOINS

Joins are used to combine two or more tables in order to easily see data from the two tables and their relationship.
The join is specified in the FROM clause of a SELECT query.
SELECT * FROM users NATURAL JOIN orders

The above query creates a joined table with matched values on both tables. This establishes a one to many relationship between the user and the customers table. Meaning one user can be a customer of various products that the company offers. Note that the users who are not customers are not going to be selected.

Join Types

A natural join however, does not take into consideration the primary key and the foreign key which are very vital in a table relationship. It only considers common columns in both tables.If there are any common columns that are not primary key and foreign key, they will also be matched and only one copy of the output produced.
Lets look at some useful join types that can be used in place of a natural join.

INNER JOIN It returns common rows in both table, just like a natural join taking into consideration both the Primary Key and the Foreign Key.

` SELECT * FROM users INNER JOIN customers ON user.id = customers.user_id; `
You can also specify an alias for the tables for easy identification as shown.
` SELECT * FROM users AS u INNER JOIN customers AS c ON u.id = c.user_id; `

OUTER JOIN

One effect of INNER JOIN and NATURAL JOIN is that unmatched primary key is dropped ie a user who is not a customer of the company is simply dropped.
We can use an OUTER JOIN to include such customers.
A FULL OUTER JOIN returns all the rows from both tables along with NULL values where there is no match in either table.

` SELECT * FROM users FULL OUTER JOIN customers ON users.id = customers.user_id; `
We can also have a FULL OUTER JOIN without intersection. Meaning we shall return all the records matching the ON clause, excluding rows found in both tables.
` SELECT * FROM users FULL OUTER JOIN customers ON users.id = customers.user_id; WHERE users.id IS NULL OR customers.user_id IS NULL `

To give users a higher priority we use LEFT OUTER JOIN

`
SELECT *
FROM users
LEFT OUTER JOIN customers
ON users.id = customers.user_id;

`
To give customers a higher priority we use a RIGHT OUTER JOIN ie all values in the right and matching values in the left and NULL where a value in the left is missing.

Self Joins This is a selfish type of join where a table is joined to itself as if it were two different tables with different aliases for each instances of the table. Consider the student table below:

id	name	student_id
1	John	3
2	Jane	3
3	Tom	4
4	Sarah	null
5	Michael	4
6	Jessica	2

We can do a self join of the table above as shown below:
` SELECT s.name AS student, l.name as leader FROM student AS s LEFT JOIN student AS l ON s.student_id = l.id; `

The output of the above code will be:

student	leader
John	Tom
Jane	Tom
Tom	Sarah
Sarah	null
Michael	Sarah
Jessica	Jane

Cross Joins They are used to create all the possible combinations of two tables.It is also known as a Cartesian product, since it produces a result set that contains all possible combinations of rows between the two tables. It is similar to the Inner Join, where the join condition is not available with this clause.

The syntax is as follows:

` SELECT * FROM table1 CROSS JOIN table2; `
Consider the following department and students table:
departments

id	name
1	IT
2	Sales
3	Medicine

students

id	name	department
1	Alice	IT
2	Bob	Sales
3	Charlie	Medicine

The query will be as shown:

` SELECT * FROM students CROSS JOIN departments; `
The resultant table will be as shown:

id	name	department	id	name
1	Alice	IT	1	IT
1	Alice	IT	2	Sales
1	Alice	IT	3	Medicine
2	Bob	Sales	1	IT
2	Bob	Sales	2	Sales
2	Bob	Sales	3	Medicine
3	Charlie	Medicine	1	IT
3	Charlie	Medicine	2	Sales
3	Charlie	Medicine	3	Medicine

Note that in the result set, there are 9 rows, which is the product of the number of rows in students (3) and the number of rows in departments (3).

Set Operations

They are used to manipulate and combine data from multiple tables and queries.

UNION

The UNION operator combines the result sets of two or more SELECT statements into a single result set. The columns in the SELECT statements must have the same data type.

` SELECT first_name, last_name FROM users UNION SELECT prod_1, prod_2 FROM customer; `

INTERSECT

The INTERSECT operator returns only the rows that appear in both result sets of two or more SELECT statements.
` SELECT first_name, last_name FROM users INTERSECT SELECT prod_1, prod_2 FROM customer; `
-EXCEPT

The EXCEPT operator returns only the distinct rows that appear in the first result set of two SELECT statements but not in the second result set.

` SELECT first_name, last_name FROM users EXCEPT SELECT prod_1, prod_2 FROM customer; `

Conclusion

SQL is a very powerful query language , with very many advantages when it comes to data analysis and manipulation. There is a lot that is not covered in this article, I just picked a few things I found worth recalling. For more information, visit the W3school SQL documentation and snowflake as well. Once you understand the SQL syntax you can referrer to this cheat-sheet for a quick reminder of the queries. Happy coding and as always stay golden dev!!

Consuming Restful Apis in reactjs

Marriane Akeyo — Tue, 10 May 2022 21:10:40 +0000

If you are a react developer and you would like to learn more about consuming restapis ,or you are having issues with consuming rest apis then this article is for you.
Rest apiscan be defined as endpoints which we use to fetch data to and from users in our react application.They are very usefull in creating a connection between our client and our backend.So when we talk about consuming restapis, we try to figure out how to call our endpoints in our frontend application so that when a user of aour application wants to make requests to our database he can do so easiy.

Consuming REST apis in react can be done in various ways .However, in this article we are going to discuss two most porpular ways known as Axios (a promise-based HTTP client) and Fetch API (a browser in-built web API).

Note:_ A good knowledge of ReactJS, React Hooks, JavaScript and CSS will come in handy as you work your way throughout this post's content._

Using the Fetch Api

The fetch() method is an in-built Javascript method for fetching resources from the server or an API endpoint.This method always takes in a compulsory URL argument containing the path to the resource you want to fetch and returns a promise that points to the request's response weather it was successful or not.

Parameters we can use for the Fetch API

Resource:This is the url with the path to the resource to be fetched.
headers
This is for specifying any headers you would like to add to your request, usually contained in an object or an object literal.
body
This is for specifying a body that you want to add to your request: this can be a Blob, BufferSource, FormData, URLSearchParams, USVString, or ReadableStream object.
mode
This is for specifying the mode you want to use for the request, e.g., cors, no-cors, or same-origin.
credentials
This for specifying the request credentials you want to use for the request, this option must be provided if you consider sending cookies automatically for the current domain.

Fetch request syntax

fetch('https://github.com/Marriane791?tab=repositories')
  .then(response => response.json())
  .then(data => console.log(data));

In the code above , I used my personal github repository as an example url.The response is just a regular HTTP response and not the actual JSON. In order to get the JSON body content from the response, we’d have to change the response to actual JSON using the json() method on the response.

Using axios
Axios() is a promised based method which is commonly preffered when fetching api endpoints in javscript.Since it is a promised based method , we take advantage of the async await just as we shall see below to make our requests.Axios also contains an in-built feature that protects the client aganist cross-site fogery.It also enables the following:

Streamlined error handling.
Request and response interception.
Upload progress.
Response timeout .
Cancellation of requests
Just to mention a few.The syntax includes:

axios.post(url,{data}).then(response => {
console.log(response.data)
});

You can now freely view your application in the browser,right click then inspect your code to view the actual data being fetched from the api.
Example:

This is an example of a code snippet from an application I created called diagnostic disease prediction which can be found here which enables doctors to chat with each other.
The method getChats, contains all the logic to fetch our data from an endpoint.The method is then called in the useEffect method and the implementation continues.

The Ultimate Guide to Getting Started in Data Science

Marriane Akeyo — Sat, 02 Apr 2022 19:16:27 +0000

We can define data science as an inter disciplinary field that uses scientific methods to prepare data for analysis, including cleansing, aggregating, and manipulating the data to perform advanced data analysis.In this article I am going to provide a guide on the various topics you can cover ,in order to get a head start at this awesome field.

1.Introduction to Python, Mastering Python basics.

Python is a high-level, general-purpose programming language. Its design philosophy emphasizes code readability with the use of significant indentation.Due to its simple structure and the qualifications above it is highly preferred for data science and data analytics.You can visit my posts on introduction to python and python data structures to get a quick introduction to this wonderfull language.
A quick and easy way to run your python codes would be using Jupyter notebook on anaconda.Jupyter notebook is a text editor which comes in very handy when advancing from writing simple python codes to using python libraries to perform data science tasks.
Anaconda on the other hand, is an open distribution of the R and python programming languages that simplifies package management and deployment.You are first required to install anaconda then install jupyter notebook present in anaconda.
Here is a blog post that sheds more light on jupyter notebook for python and anaconda.

Tip:Once you have anaconda installed and you are familiar with vs code , you can install jupyter notebook as an extension and create a new file using the (.ipynb) extension to use jupyter notebook in vs code.
Once you run a file,you will be prompted to select the environment you want to use to run your notebook.Ensure you select the base environment to smoothen your process.

You can also use the google colab which is much easier since all the packages are pre installed ,all you need to do is import and use them,so long as you have a stable network connection.Please opt for this option after you get the basics right so as to ease your coding process

2.Introduction to Databases

Once you have tried out some python projects and feel comfortable with moving forward , then learn about databases.
A database simply consists of a collection of data which are stored and accessed electronically.Data is fetched from the database with the help of queries which ask the database to perform various functions.There exists two types of databases , relational and non-relational databases.
A relational database usually has tables and data is stored in the tables in form of rows and columns.An example is the SQL database.
A non-relational database is also refferred to as NoSQL database.It stores data in a format known as JSON which uses human readable formart to store and retrieve data. Data is stored as a collection of objects containing key value pairs which are separated from each other with the help of a semi-colon.

user
 {
'id':'qwe245ert',
'name':'John',
'occupation':'Doctor'
}

In the example above , we have a user object with keys id,name and occupation each containing its own value.Some non-relational databases include Mongodb and Amazon DynamoDB.
For further insight on the differences of relational database and non-relational database checkout this video.

3.Understanding Python libraries used in Datascience

This is always the exciting part for me and I do hope it excites you too.Python contains very useful libraries that are used to perform various data science tasks .Thus understanding these packages is essential to your progress in this field.To use these libraries we must install them using pip3 install library_name and then import it into our file.

The libraries include:

Pandas
Numpy
Matplotlib
Seaborn
Pyforest

a)Pandas

This is the python package or library that is used in data analysis.It provides a dataframe that allows you to play around with your data and structure it however you want.Pandas is mostly prefered for its flexibility and ability to work with big data.
Most people prefer to refer to a dataframe as (df) when analysing their data.Some functionalities that come with pandas include :
Reading the dataset in any format:

df.read_csv('url_of_the_excel_sheet')

Viewing part of the dataset

df.head()

Locate anything:

df.loc['column_name']

#locating using an integer
df.iloc[column(s)]

Sorting

df.sort_values('column_to_sort_by' , ascending=False)

Just to mention a few .Check out this blog for more clarifications

b)Numpy
Numpy is the same a lists in python.It is used to store multidimentional arrays eg 1D,2D or 3D arrays.Numpy is ussually preferred over lists since it uses fixed values wherease values in list vary.Numpy less storage space compared to list due to the fact that all the blocks storing data are next to each other (contiguous blocks) whereas in list the blocks are far apart and pointers are used to store data.Numpy enables us to do all the mathematics we need .
Using this library, we can perform more that we could not with lists eg

a=[1,2,3,4]
b=[4,5,6,7]
a*b

The above code produces and error.However, in numpy we accomplish this using:

a=np.array([1,2,3,4])
b=np.array([4,5,6,7])
a*b

we get [4,10,12,28]
Now click here and get more information on how to get started with numpy.

c.)Matplotlib
This is an open source drawing library that converts our dataset into drawings that can be used to understand and further explain our dataset.You can generate various charts eg plots,histograms,bar charts,pie charts and scatter plots.If you are curious about what I am talking about , let this tutorial feed your interests.
Note: this library feeds on data that is as a result of the two libraries above it .In short, analyse your data by performing mathematical operations on it then plot a suitable chart of it.
d.)Seaborn
This is where we visualise information from matrices and dataframes in order to make attractive statistical plots.This is just a compliment of matplotlib and not a substitute.In short it adds more flavor to our charts and makes them look more appealing.In order to use seaborn, you need to understand how it approaches data visualizations.
More info..

Now ,after understanding the functionalities of the above libraries, we can use one awesome library called pyforest to import all our required libraries with ease.How cool is that guys.....

#First install the library in your terminal 
pip3 install pyforest 

#Now write this 
from pyforest import *

#You can veiw all imports by typing
lazy_imports()

4. Connecting to a database of your choice
We now know the python syntax ,libraries and databases present.How about we combine the knowledge and come up with something cool....
Yes I am talking about connecting our python files to the database.This is very important since we have seen we will work with a collection of data that we will regularly store and retrieve.Check this post about connecting a postgres database to python,Python and Dynamodb dataand connecting to MYSQL.When working with databases ,we will also get to understand some usefull techniques that data must undergo like data flattening and how to flatten data into a table
and some common errors found in data like missing values ,bad values , duplicates and methods of avoiding such errors .

I hope this article helps you to get started with data science.Do not let the topics overwhelm you.Try to learn on each day and for sure you will be at a better place within a month or even weeks.....
Here are some code examples of the topics covered aboveincase you need more examples.
Happy coding!!!

Denormaliztion ,its pros and cons

Marriane Akeyo — Thu, 17 Mar 2022 09:29:37 +0000

Denormalization can be defined as an optimization technique used to make queries more efficient by reducing the number of costly joins necessary to retrieve data.In short we decide to be okey with redundancy unlike in normalization,where no data redundancy is allowed.Instead each table contains its own data and only joins are used to link on data with the other.

Pros of denormalization

Queries are executed much more faster.
This is because we fetch data from one table instead of joining numerous tables to find the data we need.
Writing queries becomes much more faster.
Since there is no need to search for joining keys to fetch our data.However , one should always remember to update the tables accordingly once each update is made.

Cons of denormalization

Perfoming updates and insert queries becomes more expensive and even harder to write.
This is due to the increase in data processing as a result of data redundancy and possible data duplication.
Increase in table size
This demands an large storage space for the table.
Data may be inconsistent
A failure to update the required tables might lead to inconsistent data which might be expensive to debug.

Thanks for reading this short post ....i hope this gives you some insight.

Mastering python the right way

Marriane Akeyo — Fri, 04 Mar 2022 22:47:51 +0000

Python is a high level programming language with alot of benefits to beginners ,intermidiate and also experts in programming.It is also one of the functional programming languages .For more information about it's syntax and getting started with python , check out my article here.
Today however, we are going to focus on the tricks in understanding and mastering this language the right way.
We can further break down our mastery into the following:

Basic syntax
The basic syntax of any language never change.It is therefore important start by understanding the foundations that make up the language.They include the simple operations,datatypes ,comparisons,functions and so on as described here

Control Structures
This describes the order in which we excecute our code.This is ussually made possible using loops and function calls.They are classified into three main group

Sequencial-which is the default mode.As the name suggests,excecution happens in a sequence such that the later line of code depends on the former one.

a=4
b=5
c=5%4
print(c)

Selection best for decision making eg if else and nested if statements.

day = rainy
if day == rainy:
   print("Carry an umbrella")

Repetition used to excecute code more than once.For example while loops and for loops

numbers=[1,2,3,4,5]
for i in numbers:
  print(numbers[i])

Data Structures
List for example,store multiple items in a single variable.
Tuples are simillar to lists, except that they cannot be changed and use parantesis ().
Dictionaries,store data in form of key,value pairs which map abitrary keys to their values.A set on the other hand is an unordered collection of items and every set element stored os unique and must be immutable.
You can visit this link for perfect code examples on the same.

Mastering when to use what data structure is also very important aspect.

It is therefore important to know that:

Dictionaries are used when you need a logical association between a key and value pair,when you need a fast look up for your data based on a custom key or when your data is being constantly modified since they are mutable.
Lists when dealing with data that does not deal with random access.
Sets when the elements are unique in nature.
Tuples when your data cannot change.

Indentation and whitespaces
Python uses a maximum of 4 whitespaces for indentation and a minimum of one, however ,the number of spaces must be uniform in a block of code.The first line of code cannot be indented.Indentation begins from the second line if need be.This reduces the lines of code and also increases readability.

Decorators
They aid in modifying functions or extend their functionality using other functions.We use the @ symbol followed by the decorator_name.

def meal_decorator(function):
       def cook():
          return "preparing something nice"
   @meal_decorator
   def eat():
      return "Lets dive in"
   print (eat())

Python Classes
A class is a blueprint of an object and in turn we can classify an object as an instance of a class.
It contains attributes and methods as shown below.

class Person:
     name = "John" #attribute

     def skate(self,wheels):
        return f'{name}, skates using {wheels}' 

     #constructor
     def __init__(self,age):
       self.age = age
#instantiate
Student = Person(25)

print(Student.skate())

It is important to master that a class can also inherit from another class without modifying it.The class thus becomes a child class of the parent class and has all the characteristics of the parent class.
In this case we shall just add the parent name to the new class as follows

class Banker[Person]

Encapsulation
This involves hiding the complexity of the code from the user .
Polymorphisim
Involves using a common interface for many forms.

Frameworks
Lastly we shall look at frameworks .Python frameworks help hide the complexity of the code into something simple that can be understood and used by many people.Some common python frameworks include FlaskAPI,FastApi and Django, just to mention a few. Each framework has its own strength and weaknesses, depending on the task at hand.Having a better understanding of this also enables one to master python the right way.

I hope this gives you an incite of the modern python and ts mastery.Thanks for reading.

Data Structures and Algorithims with python.

Marriane Akeyo — Mon, 21 Feb 2022 18:25:31 +0000

Data Structures basically describes how we can ogarnise and stored in memory when a program processes it.For example, we can store a list of items having the same data-type using the array data structure.

An Algorithm on the other hand, is step by step set of instruction to process the data for a specific purpose. So, an algorithm uterlises various data structures in a logical way to solve a specific computing problem.This is a very important concept in computer science .It gives us more insight on the data we are working with.It enables data scientists to make better machine learning predictions.This is however a challenging topic to most people in the tech industry , according to most people.We are going to look at various python data structures in python and their code examples.

Lists

The list is a most versatile datatype available in Python which can be written as a list of comma-separated values (items) between square brackets.Lists are mutable and ordered. It can contain a mix of different data types.

list1 = ['chicken', 'pizza', 2022, 2000]
list2 = [1, 2, 3, 4, 5 ]
list3 = ["a", "b", "c", "d"]

We can access values in a list using their index.
NOTE: we start counting from 0

print (list1[0]) #prints the element in the 0 index

We also use the .append() method to add new items into the list eg

list2.append(6) #add 6 to the existing list2

Incase you want to add to a specific place in the list ,we do it as follows

list3[2] = "e" # returns ["a", "b", "e", "d"]

Some basic list operations include:

len([1, 2, 3]) resulting to 3 which tests the Length
[1, 2, 3] + [4, 5, 6]resulting to[1, 2, 3, 4, 5, 6]|Concatenation
['Hi!'] * 3 resulting to ['Hi!', 'Hi!', 'Hi!'] which tests Repetition
3 in [1, 2, 3] resulting to True which tests Membership
for x in [1, 2, 3]:print x resulting to 1 2 3 which tests Iteration

Dictionaries

Dictionary is a mutable and unordered data structure. It permits storing a pair of items (i.e. keys and values).Each key is separated from its value by a colon (:), the items are separated by commas, and the whole thing is enclosed in curly braces. An empty dictionary without any items is written with just two curly braces, like this − {}.
Keys are unique within a dictionary while values may not be. The values of a dictionary can be of any type, but the keys must be of an immutable data type such as strings, numbers, or tuples.
Accessing Values in Dictionary
To access dictionary elements, you can use the familiar square brackets along with the key to obtain its value.
Example:

dict = {'Name': 'Marrie', 'Age': 27, 'Language': 'Python'}
print "dict['Name']: ", dict['Name']
print "dict['Age']: ", dict['Age']

When the above code is executed, it produces the following result :

dict['Name']:  Marrie
dict['Age']:  27

If we attempt to access a data item with a key, which is not part of the dictionary, we get an error as follows:
Example

#Using the data above.
print "dict['Alice']: ", dict['Alice']

When the above code is executed, it produces the following result :

dict['Alice']:
Traceback (most recent call last):
   File "test.py", line 4, in <module>
      print "dict['Alice']: ", dict['Alice'];
KeyError: 'Alice'

Updating Dictionary
You can update a dictionary by adding a new entry or a key-value pair, modifying an existing entry, or deleting an existing entry as shown below in the simple example:

dict = {'Name': 'Marrie', 'Age': 27, 'Language': 'Python'}
dict['Age'] = 28; # update existing entry
dict['School'] = "LuxAcademy"; # Add new entry

print "dict['Age']: ", dict['Age']
print "dict['School']: ", dict['School']

When the above code is executed, it produces the following result :

dict['Age']:  28
dict['School']:LuxTech

Delete Dictionary Elements
You can either remove individual dictionary elements or clear the entire contents of a dictionary. You can also delete entire dictionary in a single operation.
To explicitly remove an entire dictionary, just use the del statement.

dict = {'Name': 'Marrie', 'Age': 27, 'Language': 'Python'}
del dict['Name']; # remove entry with key 'Name'
dict.clear();     # remove all entries in dict
del dict ;        # delete entire dictionary

print "dict['Age']: ", dict['Age']
print "dict['School']: ", dict['School']

Note −that an exception is raised because after del dict dictionary does not exist any more:

dict['Age']:
Traceback (most recent call last):
   File "test.py", line 8, in <module>
      print "dict['Age']: ", dict['Age'];
TypeError: 'type' object is unsubscriptable

Properties of Dictionary Keys
Dictionary values have no restrictions. They can be any arbitrary Python object, either standard objects or user-defined objects. However, same is not true for the keys.
There are two important points to remember about dictionary keys:
*More than one entry per key not allowed. Which means no duplicate key is allowed. When duplicate keys encountered during assignment, the last assignment wins.

dict = {'Name': 'Marrie', 'Age': 27, 'Name': 'Python'}
print "dict['Name']: ", dict['Name']

When the above code is executed, it produces the following result:

dict['Name']:  Python

*Keys must be immutable. Which means you can use strings, numbers or tuples as dictionary keys but something like ['key'] is not allowed.
An example is as follows :

dict = {['Name']: 'Marrie', 'Age': 27}
print "dict['Name']: ", dict['Name']

When the above code is executed, it produces the following result :

Traceback (most recent call last):
   File "test.py", line 3, in <module>
      dict = {['Name']: 'Marrie', 'Age': 27};
TypeError: list objects are unhashable

Tuples

A tuple is another container. It is a data type for immutable ordered sequences of elements. Immutable because you can’t add and remove elements from tuples, or sort them in place.
Creating a tuple is as simple as putting different comma-separated values. Optionally you can put these comma-separated values between parentheses also.
For example:

tuple_one = ('python', 'javascript', 'c++', 2000);
tuple_two = (1, 2, 3, 4, 5 );
tuple_3 = "a", "b", "c", "d";

The empty tuple is written as two parentheses containing nothing −

languages = ();

To write a tuple containing a single value you have to include a comma, even though there is only one value −

tup1 = (50,);

Like string indices, tuple indices start at 0, and they can be sliced, concatenated, and so on.
Accessing Values in Tuples
To access values in tuple, use the square brackets for slicing along with the index or indices to obtain value available at that index.
For example

tuple_one = ('python', 'javascript', 'c++', 2000);
tuple_two = (1, 2, 3, 4, 5 );
print "tuple_one[0]: ", tuple_two[0];
print "tuple_two[1:5]: ",tuple_two[1:5];

When the above code is executed, it produces the following result :

tuple_one[0]:  python
tuple_two[1:5]:  [2, 3, 4, 5]

Updating Tuples
Tuples are immutable which means you cannot update or change the values of tuple elements. You are able to take portions of existing tuples to create new tuples as the following example demonstrates −

tup1 = (12, 34.56);
tup2 = ('abc', 'xyz');

# Following action is not valid for tuples
# tup1[0] = 100;

# So let's create a new tuple as follows
tup3 = tup1 + tup2;
print tup3;

When the above code is executed, it produces the following result:

(12, 34.56, 'abc', 'xyz')

Delete Tuple Elements
Removing individual tuple elements is not possible. There is, of course, nothing wrong with putting together another tuple with the undesired elements discarded.
To explicitly remove an entire tuple, just use the del statement.
For example:

tuple_one = ('python', 'javascript', 'c++', 2000);
print tuple_one;
del tuple_one;
print "After deleting tup : ";
print tuple_one;

Note − an exception raised, this is because after del tup tuple does not exist anymore.
This produces the following result:

('python', 'javascript', 'c++', 2000)

After deleting tup :

Traceback (most recent call last):
   File "test.py", line 9, in <module>
      print tuple_one;
NameError: name 'tuple_one' is not defined.

Assuming our file name is "test.py"

Basic Tuples Operations
Tuples respond to the + and * operators much like strings; they mean concatenation and repetition here too, except that the result is a new tuple, not a string.
In fact, tuples respond to all of the general sequence operations we used on strings in the prior chapter.

len((1, 2, 3)) which results to 3 used to test Length
(1, 2, 3) + (4, 5, 6)which results to(1, 2, 3, 4, 5,6) and is used to test Concatenation
('Hi!',) * 4 which results to('Hi!', 'Hi!', 'Hi!', 'Hi!') and is used to test Repetition
3 in (1, 2, 3) which results to True and is used to test Membership
for x in (1, 2, 3): print x which results to 1 2 3 and is used to test Iteration

Sets

Set is a mutable and unordered collection of unique elements. It can permit us to remove duplicate quickly from a list.The sets in python are typically used for mathematical operations like union, intersection, difference and complement etc.
A Python set is similar to this mathematical definition with below additional conditions:
*The elements in the set cannot be duplicates.
*The elements in the set are immutable(cannot be modified) but the set as a whole is mutable.
*There is no index attached to any element in a python set. So they do not support any indexing or slicing operation.

Creating a set
A set is created by using the set() function or placing all the elements within a pair of curly braces.

Days=set(["Mon","Tue","Wed","Thu","Fri","Sat","Sun"])
Months={"Jan","Feb","Mar"}
Dates={21,22,17}
print(Days)
print(Months)
print(Dates)

Note how the order of the elements has changed in the result.

set(['Wed', 'Sun', 'Fri', 'Tue', 'Mon', 'Thu', 'Sat'])
set(['Jan', 'Mar', 'Feb'])
set([17, 21, 22])

Accessing Values in a Set
We cannot access individual values in a set. We can only access all the elements together as shown above. But we can also get a list of individual elements by looping through the set.

#Considering the data above.
Days=set(["Mon","Tue","Wed","Thu","Fri","Sat","Sun"])

for d in Days:
   print(d)

When the above code is executed, it produces the following :

Wed
Sun
Fri
Tue
Mon
Thu
Sat

Adding Items to a Set
We can add elements to a set by using add() method. Remember,there is no specific index attached to the newly added element.

#Adding to the data above. 
Days.add("Sun")
print(Days)

results

set(['Wed', 'Sun', 'Fri', 'Tue', 'Mon', 'Thu', 'Sat'])

Removing Item from a Set
We can remove elements from a set by using discard() method.
Example

#Using the data above.
Days.discard("Sun")
print(Days)

Output

set(['Wed', 'Fri', 'Tue', 'Mon', 'Thu', 'Sat'])

Union of Sets
The union operation on two sets produces a new set containing all the distinct elements from both the sets. In the below example the element “Wed” is present in both the sets.

DaysA = set(["Mon","Tue","Wed"])
DaysB = set(["Wed","Thu","Fri","Sat","Sun"])
AllDays = DaysA|DaysB
print(AllDays)

Output will be as shown,note the result has only one “wed”.

set(['Wed', 'Fri', 'Tue', 'Mon', 'Thu', 'Sat'])

Intersection of Sets
The intersection operation on two sets produces a new set containing only the common elements from both the sets. In the below example the element “Wed” is present in both the sets.

DaysA = set(["Mon","Tue","Wed"])
DaysB = set(["Wed","Thu","Fri","Sat","Sun"])
AllDays = DaysA & DaysB
print(AllDays)

Output

set(['Wed'])

Difference of Sets
The difference operation on two sets produces a new set containing only the elements from the first set and none from the second set. In the below example the element “Wed” is present in both the sets so it will not be found in the result set.

DaysA = set(["Mon","Tue","Wed"])
DaysB = set(["Wed","Thu","Fri","Sat","Sun"])
AllDays = DaysA - DaysB
print(AllDays)

Output
When the above code is executed, it produces the following result. Please note the result has only one “wed”.

set(['Mon', 'Tue'])

Compare Sets
We can check if a given set is a subset or superset of another set. The result is True or False depending on the elements present in the sets.
Example

DaysA = set(["Mon","Tue","Wed"])
DaysB = set(["Mon","Tue","Wed","Thu","Fri","Sat","Sun"])
SubsetRes = DaysA <= DaysB
SupersetRes = DaysB >= DaysA
print(SubsetRes)
print(SupersetRes)

Output

True
True

Queue

The queue is a linear data structure where elements are in a sequential manner. It follows the F.I.F.O mechanism that means first in first out.
Below the aspects that characterize a queue.
Two ends:
*front → points to starting element
*rear → points to the last element
There are two operations:
*enqueue → inserting an element into the queue. It will be done at the rear.
*dequeue → deleting an element from the queue. It will be done at the front.
There are two conditions:
*overflow → insertion into a queue that is full
*underflow → deletion from the empty queue
Lets see a code example of this:

class myqueue:

    def __init__(self):
        self.data = []

    def length(self):
        return len(self.data)

    def enque(self, element): # put the element in the queue
        if len(self.data) < 5:
            return self.data.append(element)
        else:
            return "overflow"

    def deque(self): # remove the first element that we have put in queue
        if len(self.data) == 0:
             return "underflow"
        else:
            self.data.pop(0)

b = myqueue()
b.enque(2) # put the element into the queue
b.enque(3)
b.enque(4)
b.enque(5)
print(b.data)
b.deque()# # remove the first element that we have put in the queue
print(b.data)

This will procuce the following results.

[2, 3, 4, 5]
[3, 4, 5]

Stack

In the english dictionary the word stack means arranging objects on over another. Stack is a linear data structure which follows a particular order in which the operations are performed. The order may be LIFO(Last In First Out) or FILO(First In Last Out).
In the following program we implement it as add and and remove functions. We declare an empty list and use the append() and pop() methods to add and remove the data elements.
Pushing into a Stack
Example

class Stack:
   def __init__(self):
      self.stack = []

   def add(self, dataval):
# Use list append method to add element
      if dataval not in self.stack:
         self.stack.append(dataval)
         return True
      else:
         return False
# Use peek to look at the top of the stack
   def peek(self):     
       return self.stack[-1]

AStack = Stack()
AStack.add("Mon")
AStack.add("Tue")
AStack.peek()
print(AStack.peek())
AStack.add("Wed")
AStack.add("Thu")
print(AStack.peek())

Output

Tue
Thu

POP from a Stack
As we know we can remove only the top most data element from the stack, we implement a python program which does that. The remove function in the following program returns the top most element. We check the top element by calculating the size of the stack first and then use the in-built pop() method to find out the top most element.

class Stack:
   def __init__(self):
      self.stack = []

   def add(self, dataval):
# Use list append method to add element
      if dataval not in self.stack:
         self.stack.append(dataval)
         return True
      else:
         return False

# Use list pop method to remove element
   def remove(self):
      if len(self.stack) <= 0:
         return ("No element in the Stack")
      else:
         return self.stack.pop()

AStack = Stack()
AStack.add("Mon")
AStack.add("Tue")
AStack.add("Wed")
AStack.add("Thu")
print(AStack.remove())
print(AStack.remove())

Output:

Thu
Wed

Linked list

A linked list is a linear data structure, in which the elements are not stored at contiguous memory locations. The elements in a linked list are linked using pointers as shown in the below image:

Now lets create a linked list

class Node:
   def __init__(self, dataval=None):
      self.dataval = dataval
      self.nextval = None

class SLinkedList:
   def __init__(self):
      self.headval = None

list1 = SLinkedList()
list1.headval = Node("Mon")
e2 = Node("Tue")
e3 = Node("Wed")
# Link first Node to second node
list1.headval.nextval = e2

# Link second Node to third node
e2.nextval = e3

Traversing a Linked List
Singly linked lists can be traversed in only forward direction starting form the first data element. We simply print the value of the next data element by assigning the pointer of the next node to the current data element.
Example

class Node:
   def __init__(self, dataval=None):
      self.dataval = dataval
      self.nextval = None

class SLinkedList:
   def __init__(self):
      self.headval = None

   def listprint(self):
      printval = self.headval
      while printval is not None:
         print (printval.dataval)
         printval = printval.nextval

list = SLinkedList()
list.headval = Node("Mon")
e2 = Node("Tue")
e3 = Node("Wed")

# Link first Node to second node
list.headval.nextval = e2

# Link second Node to third node
e2.nextval = e3

list.listprint()

Output:

Mon
Tue
Wed

Insertion in a Linked List
Inserting element in the linked list involves reassigning the pointers from the existing nodes to the newly inserted node. Depending on whether the new data element is getting inserted at the beginning or at the middle or at the end of the linked list, we have the below scenarios.
Inserting at the Beginning
This involves pointing the next pointer of the new data node to the current head of the linked list. So the current head of the linked list becomes the second data element and the new node becomes the head of the linked list.
Example

class Node:
   def __init__(self, dataval=None):
      self.dataval = dataval
      self.nextval = None

class SLinkedList:
   def __init__(self):
      self.headval = None
# Print the linked list
   def listprint(self):
      printval = self.headval
      while printval is not None:
         print (printval.dataval)
         printval = printval.nextval
   def AtBegining(self,newdata):
      NewNode = Node(newdata)

# Update the new nodes next val to existing node
   NewNode.nextval = self.headval
   self.headval = NewNode

list = SLinkedList()
list.headval = Node("Mon")
e2 = Node("Tue")
e3 = Node("Wed")

list.headval.nextval = e2
e2.nextval = e3

list.AtBegining("Sun")
list.listprint()

Output:

Sun
Mon
Tue
Wed

Inserting at the End
This involves pointing the next pointer of the the current last node of the linked list to the new data node. So the current last node of the linked list becomes the second last data node and the new node becomes the last node of the linked list.
Example

class Node:
   def __init__(self, dataval=None):
      self.dataval = dataval
      self.nextval = None
class SLinkedList:
   def __init__(self):
      self.headval = None
# Function to add newnode
   def AtEnd(self, newdata):
      NewNode = Node(newdata)
      if self.headval is None:
         self.headval = NewNode
         return
      laste = self.headval
      while(laste.nextval):
         laste = laste.nextval
      laste.nextval=NewNode
# Print the linked list
   def listprint(self):
      printval = self.headval
      while printval is not None:
         print (printval.dataval)
         printval = printval.nextval

list = SLinkedList()
list.headval = Node("Mon")
e2 = Node("Tue")
e3 = Node("Wed")

list.headval.nextval = e2
e2.nextval = e3

list.AtEnd("Thu")

list.listprint()

Output

Mon
Tue
Wed
Thu

Inserting in between two Data Nodes
This involves changing the pointer of a specific node to point to the new node. That is possible by passing in both the new node and the existing node after which the new node will be inserted. So we define an additional class which will change the next pointer of the new node to the next pointer of middle node. Then assign the new node to next pointer of the middle node.

class Node:
   def __init__(self, dataval=None):
      self.dataval = dataval
      self.nextval = None
class SLinkedList:
   def __init__(self):
      self.headval = None

# Function to add node
   def Inbetween(self,middle_node,newdata):
      if middle_node is None:
         print("The mentioned node is absent")
         return

      NewNode = Node(newdata)
      NewNode.nextval = middle_node.nextval
      middle_node.nextval = NewNode

# Print the linked list
   def listprint(self):
      printval = self.headval
      while printval is not None:
         print (printval.dataval)
         printval = printval.nextval

list = SLinkedList()
list.headval = Node("Mon")
e2 = Node("Tue")
e3 = Node("Thu")

list.headval.nextval = e2
e2.nextval = e3

list.Inbetween(list.headval.nextval,"Fri")

list.listprint()

Output:

Mon
Tue
Fri
Thu

Removing an Item
We can remove an existing node using the key for that node. In the below program we locate the previous node of the node which is to be deleted.Then, point the next pointer of this node to the next node of the node to be deleted.
Example

class Node:
   def __init__(self, data=None):
      self.data = data
      self.next = None
class SLinkedList:
   def __init__(self):
      self.head = None

   def Atbegining(self, data_in):
      NewNode = Node(data_in)
      NewNode.next = self.head
      self.head = NewNode

# Function to remove node
   def RemoveNode(self, Removekey):
      HeadVal = self.head

      if (HeadVal is not None):
         if (HeadVal.data == Removekey):
            self.head = HeadVal.next
            HeadVal = None
            return
      while (HeadVal is not None):
         if HeadVal.data == Removekey:
            break
         prev = HeadVal
         HeadVal = HeadVal.next

      if (HeadVal == None):
         return

      prev.next = HeadVal.next
         HeadVal = None

   def LListprint(self):
      printval = self.head
      while (printval):
         print(printval.data),
         printval = printval.next

llist = SLinkedList()
llist.Atbegining("Mon")
llist.Atbegining("Tue")
llist.Atbegining("Wed")
llist.Atbegining("Thu")
llist.RemoveNode("Tue")
llist.LListprint()

Output:

Thu
Wed
Mon

Algorithims

Algorithms are instructions that are formulated in a finite and sequential order to solve problems.
The word algorithm derives itself from the 9th-century Persian mathematician Muḥammad ibn Mūsā al-Khwārizmī, whose name was Latinized as Algorithmi. Al-Khwārizmī was also an astronomer, geographer, and a scholar in the House of Wisdom in Baghdad.

As you already know algorithms are instructions that are formulated in a finite and sequential order to solve problems.

When we write an algorithm, we have to know what is the exact problem, determine where we need to start and stop and formulate the intermediate steps.

There are three main approaches to solve algorithms:

*Divide et Impera (also known as divide and conquer) → it divides the problem into sub-parts and solves each one separately
*Dynamic programming → it divides the problem into sub-parts remembers the results of the sub-parts and applies it to similar ones
*Greedy algorithms → involve taking the easiest step while solving a problem without worrying about the complexity of the future steps

Tree Traversal Algorithm
Trees in python are non-linear data structures. They are characterized by roots and nodes. I take the class I constructed before for the binary tree.

Tree Traversal refers to visiting each node present in the tree exactly once, in order to update or check them.

# create the class Node and the attrbutes 
class Node:
    def __init__(self, letter):
        self.childleft = None
        self.childright = None
        self.nodedata = letter

# create the nodes for the tree
root = Node('A')
root.childleft = Node('B')
root.childright = Node('C')
root.childleft.childleft = Node('D')
root.childleft.childright = Node('E')

There are three types of tree traversals:
*In-order traversal → refers to visiting the left node, followed by the root and then the right nodes.
Here D is the leftmost node where the nearest root is B. The right of root B is E. Now the left sub-tree is completed, so I move towards the root node A and then to node C.

def InOrd(root):
    if root:
        InOrd(root.childleft)
        print(root.nodedata)
        InOrd(root.childright)
InOrd(root)

Out:

D
B
E
A
C

*Pre-order traversal → refers to visiting the root node followed by the left nodes and then the right nodes.
In this case, I move to the root node A and then to the left child node B and to the sub child node D. After that I can go to the nodes E and then C.

def PreOrd(root):
    if root:
        print(root.nodedata)
        PreOrd(root.childleft)
        PreOrd(root.childright)
PreOrd(root)

out:

A
B
D
E
C

*Post-order traversal → refers to visiting the left nodes followed by the right nodes and then the root node.
I go to the most left node which is D and then to the right node E. Then, I can go from the left node B to the right node C. Finally, I move towards the root node A.

def PostOrd(root):
    if root:
        PostOrd(root.childleft)
        PostOrd(root.childright)
        print(root.nodedata)
PostOrd(root)

out:

D
E
B
C
A

Sorting Algorithm
The sorting algorithm is used to sort data in some given order. It can be classified in Merge Sort and Bubble Sort.
*Merge Sort → it follows the divide et Impera rule. The given list is first divided into smaller lists and compares adjacent lists and then, reorders them in the desired sequence. So, in summary from unordered elements as input, we need to have ordered elements as output. Below, the code with each step described.

def merge_sort(ourlist, left, right): #left and right corresponds to starting and ending element of ourlist
    if right -left > 1: # check if the length of ourlist is greater than 1
        middle = (left + right) // 2 # we divide the length in two parts
        merge_sort(ourlist, left, middle) # recursevely I call the merge_sort function from left to middle
        merge_sort(ourlist, middle, right) # then from middle to right
        merge_list(ourlist, left, middle, right) # finally I create ourlist in complete form(left, middle and right) 

def merge_list(ourlist, left, middle, right):# I create the function merged_list
    leftlist = ourlist[left:middle] # I define the leftlist
    rightlist = ourlist[middle:right] # I define the right list
    k = left # it is the the temporary variable
    i = 0 # this variable that corespond to the index of the first group help me to iterate from left to right
    j = 0 # this variable that corespond to the index of the second group help me to iterate from left to right
    while (left + i < middle and middle+ j < right): # the condition that I want to achive before to stop my iteration
        if (leftlist[i] <= rightlist[j]): #if the element in the leftlist is less or equal to the element in the rightlist
            ourlist[k] = leftlist[i] # In this case I fill the value of the leftlist in ourlist with index k
            i = i + 1 #now I have to increment the value by 1
        else: # if the above codition is not match
            ourlist[k] = rightlist[j] # I fill the rightlist element in ourlist with index k
            j = j + 1 # I increment index j by 1
        k = k+1 # now I increment the value of k by 1
    if left + i < middle: # check if left + i is less than middle
        ourlist[k] = leftlist[i] # I place all elements of my leftlist in ourlist
        i = i + 1
        k = k + 1
    else: # otherwise if my leftlist is empty
        while k < right: # untill k is less then right
            ourlist[k] = rightlist[j] # I place all elements of rightlist in ourlist 
            j = j + 1
            k = k + 1

ourlist = input('input - unordered elements: ').split() # insert the input and split
ourlist = [int(x) for x in ourlist]
merge_sort(ourlist, 0, len(ourlist))
print('output - ordered elements: ')
print(ourlist)

Out:

input - unordered elements: 15 1 19 93
output - ordered elements: 
[1, 15, 19, 93]

*Bubble Sort → it first compares and then sorts adjacent elements if they are not in the specified order.

def bubble_sort(ourlist): # I create my function bubble_sort with the argument called ourlist
    b=len(ourlist)-1 # for every list, I will have a minus 1 iteration
    for x in range(b): # for each element in the range of b, I check if they are ordered or not
        for y in range(b-x): 
            if ourlist[y]>ourlist[y+1]: # if one element is greater than the nearest elemnt in the list
                ourlist[y],ourlist[y+1]=ourlist[y+1],ourlist[y] # I have to swap the elemnts, in other words
                                                          # I exchange the position of the two elements
        return ourlist

ourlist=[15,1,9,3]
bubble_sort(ourlist)

Out:

[1, 3, 9, 15]

*Insertion Sort → it picks one item of a given list at the time and places it at the exact spot where it is to be placed.

def ins_sort(ourlist):
    for x in range(1, len(ourlist)): # loop for each element starting from 1 to the length of our list
        k = ourlist[x] # element with the index x
        j = x-1 # j is the index previus the index x
        while j >=0 and k < ourlist[j]: # untill each elermnt of the list are less than their previous element my loop don't stop
                ourlist[j+1] = ourlist[j] # the elemnt indexed before the element considered is set to the next one index
                j -= 1 # I decrement index j by 1
        ourlist[j+1] = k # now k is the element in the index j+1
    return ourlist

ourlist = [15, 1, 9, 3]
ins_sort(ourlist)

out:

[1, 3, 9, 15]

There are other Sorting Algorithms like Selection Sort and Shell Sort.

Searching Algorithms

*Searching algorithms are used to seek for some elements present in a given dataset. There are many types of search algorithms such as Linear Search, Binary Search, Exponential Search, Interpolation Search, and so on. In this section, we will see the Linear Search and Binary Search.

*Linear Search → in a single-dimensional array we have to search a particular key element. The input is the group of elements and the key element that we want to find. So, we have to compare the key element with each element of the group. In the following code, Lets try to seek element 27 in our list.

def lin_search(ourlist, key):

    for index in range(0, len(ourlist)):
        if (ourlist[index] == key):
            return  index
    else:
        return "not fund"

ourlist = [15, 1, 9, 3]

lin_search(ourlist, 27)

out:

'not fund'

*Binary Search → in this algorithm, we assume that the list is in ascending order. So, if the value of the search key is less than the element in the middle of the list, we narrow the interval to the lower half. Otherwise, we narrow to the upper half. We continue our check until the value is found or the list is empty.

def bin_search(ourlist, key):
    left = 0 # I assign left position to zero
    right = len(ourlist)-1 # I assign right position by defining the length of ourlist minus one 
    matched = False
    while(left<=right and not matched): # the loop will continue untill the left element is less or equal to the right element and the matched is True
        mid = (left+right)//2 # I find the position of the middle element
        if ourlist[mid] == key: # if the middle element correponds to the key element
             matched = True
        else: #otherwise 
            if key < ourlist[mid]: # if key element is less than the middle element
                right = mid - 1 #I assign the position of the right element as mid - 1
            else: #otherwise
                left = mid + 1 #left position will become the middle position plus 1
    return matched

print(bin_search([1, 3, 9, 15], 17))
print(bin_search([1, 3, 9, 15], 3))

out:

False
True

Data Structures and algorithims in Javascript

Marriane Akeyo — Mon, 21 Feb 2022 18:16:05 +0000

Data Structures basically describes how we can ogarnise and stored in memory when a program processes it.For example, we can store a list of items having the same data-type using the array data structure.

An Algorithm on the other hand, is step by step set of instruction to process the data for a specific purpose. So, an algorithm uterlises various data structures in a logical way to solve a specific computing problem.This is a very important concept in computer science .It gives us more insight on the data we are working with.It enables data scientists to make better machine learning predictions.This is however a challenging topic to most people in the tech industry , according to most people.We are going to look at various python data structures in python and their code examples.

Lists

The list is a most versatile datatype available in Javascript which can be written as a list of comma-separated values (items) between square brackets.Lists are mutable and ordered. It can contain a mix of different data types.

list1 = ['chicken', 'pizza', 2022, 2000]
list2 = [1, 2, 3, 4, 5 ]
list3 = ["a", "b", "c", "d"]

We can access values in a list using their index.
NOTE: we start counting from 0

console.log (list1[0]) //prints the element in the 0 index

We also use the .push() method to add new items into the list eg

list2.push(6) //add 6 to the existing list2

Incase you want to add to a specific place in the list ,we do it as follows

list3[2] = "e" // returns ["a", "b", "e", "d"]

Dictionaries

Dictionary is a mutable and unordered data structure. It permits storing a pair of items (i.e. keys and values).Each key is separated from its value by a colon (:), the items are separated by commas, and the whole thing is enclosed in curly braces. An empty dictionary without any items is written with just two curly braces, like this − {}.
Keys are unique within a dictionary while values may not be. The values of a dictionary can be of any type, but the keys must be of an immutable data type such as strings, numbers, or tuples.

Accessing Values in Dictionary
To access dictionary elements, you can use the familiar square brackets along with the key to obtain its value.
Example:

dict = {'Name': 'Marrie', 'Age': 27, 'Language': 'Javascript'}
console.log( "dict['Name']: ", dict['Name'])
console.log( "dict['Age']: ", dict['Age'])

When the above code is executed, it produces the following result :

dict['Name']:  Marrie
dict['Age']:  27

Updating Dictionary
You can update a dictionary by adding a new entry or a key-value pair, modifying an existing entry, or deleting an existing entry as shown below in the simple example:

dict = {'Name': 'Marrie', 'Age': 27, 'Language': 'Python'}
dict['Age'] = 28; // update existing entry
dict['School'] = "LuxAcademy"; # Add new entry

console.log ("dict['Age']: ", dict['Age'])
console.log ("dict['School']: ", dict['School'])

When the above code is executed, it produces the following result :

dict['Age']:  28
dict['School']:LuxAcademy

Delete Dictionary Elements
You can either remove individual dictionary elements or clear the entire contents of a dictionary. You can also delete entire dictionary in a single operation.
To explicitly remove an entire dictionary, just use the del statement.

dict = {'Name': 'Marrie', 'Age': 27, 'Language': 'Python'}
del dict['Name']; // remove entry with key 'Name'
dict.clear();     // remove all entries in dict
del dict ;        // delete entire dictionary

console.log( "dict['Age']: ", dict['Age'])
console.log ("dict['School']: ", dict['School'])

Note −that an exception is raised because after del dict dictionary does not exist any more.

Properties of Dictionary Keys
Dictionary values have no restrictions. They can be any arbitrary javavscript object, either standard objects or user-defined objects. However, same is not true for the keys.
There are two important points to remember about dictionary keys:
*More than one entry per key not allowed. Which means no duplicate key is allowed. When duplicate keys encountered during assignment, the last assignment wins.

dict = {'Name': 'Marrie', 'Age': 27, 'Name': 'Javascript'}
console.log( "dict['Name']: ", dict['Name'])

When the above code is executed, it produces the following result:

dict['Name']:  Javascript

*Keys must be immutable. Which means you can use strings, numbers or tuples as dictionary keys but something like ['key'] is not allowed.

Tuples

A tuple is another container. It is a data type for immutable ordered sequences of elements. Immutable because you can’t add and remove elements from tuples, or sort them in place.
Creating a tuple is as simple as putting different comma-separated values. Optionally you can put these comma-separated values between parentheses also.
For example:

tuple_one = ('javascript', 'java', 'c++', 2000);
tuple_two = (1, 2, 3, 4, 5 );
tuple_3 = "a", "b", "c", "d";

The empty tuple is written as two parentheses containing nothing −

languages = ();

To write a tuple containing a single value you have to include a comma, even though there is only one value −

tup1 = (50,);

Like string indices, tuple indices start at 0, and they can be sliced, concatenated, and so on.
Accessing Values in Tuples
To access values in tuple, use the square brackets for slicing along with the index or indices to obtain value available at that index.
For example

tuple_one = ('python', 'javascript', 'c++', 2000);
tuple_two = (1, 2, 3, 4, 5 );
console.log ("tuple_one[0]: ", tuple_two[0]);
console.log ("tuple_two[1:5]: ",tuple_two[1:5]);

When the above code is executed, it produces the following result :

tuple_one[0]:  python
tuple_two[1:5]:  [2, 3, 4, 5]

Updating Tuples
Tuples are immutable which means you cannot update or change the values of tuple elements. You are able to take portions of existing tuples to create new tuples as the following example demonstrates −

tup1 = (12, 34.56);
tup2 = ('abc', 'xyz');

// Following action is not valid for tuples
// tup1[0] = 100;

// So let's create a new tuple as follows
tup3 = tup1 + tup2;
console.log(tup3);

When the above code is executed, it produces the following result:

(12, 34.56, 'abc', 'xyz')

Delete Tuple Elements
Removing individual tuple elements is not possible. There is, of course, nothing wrong with putting together another tuple with the undesired elements discarded.
To explicitly remove an entire tuple, just use the del statement.
For example:

tuple_one = ('python', 'javascript', 'c++', 2000);
console.log( tuple_one);
del tuple_one;
print "After deleting tup : ";
print tuple_one;

Note − an exception raised, this is because after del tup tuple does not exist anymore.
This produces the following result:

('python', 'javascript', 'c++', 2000)

Sets

Set is a mutable and unordered collection of unique elements. It can permit us to remove duplicate quickly from a list.The sets in javacript are typically used for mathematical operations like union, intersection, difference and complement etc.
A javascript set is similar to this mathematical definition with below additional conditions:
*The elements in the set cannot be duplicates.
*The elements in the set are immutable(cannot be modified) but the set as a whole is mutable.
*There is no index attached to any element in a python set. So they do not support any indexing or slicing operation.

Creating a set
A set is created by using the set() function or placing all the elements within a pair of curly braces.

Days=set(["Mon","Tue","Wed","Thu","Fri","Sat","Sun"])
Months={"Jan","Feb","Mar"}
Dates={21,22,17}
console.log(Days)
console.log(Months)
console.log(Dates)

Note how the order of the elements has changed in the result.

set(['Wed', 'Sun', 'Fri', 'Tue', 'Mon', 'Thu', 'Sat'])
set(['Jan', 'Mar', 'Feb'])
set([17, 21, 22])

Accessing Values in a Set
We cannot access individual values in a set. We can only access all the elements together as shown above. But we can also get a list of individual elements by looping through the set.

//Considering the data above.
Days=set(["Mon","Tue","Wed","Thu","Fri","Sat","Sun"])

for d in Days:
   console.log(d)

When the above code is executed, it produces the following :

Wed
Sun
Fri
Tue
Mon
Thu
Sat

Adding Items to a Set

We can add elements to a set by using add() method. Remember,there is no specific index attached to the newly added element.

//Adding to the data above. 
Days.add("Sun")
 console.log(Days)

results

set(['Wed', 'Sun', 'Fri', 'Tue', 'Mon', 'Thu', 'Sat'])

Removing Item from a Set
We can remove elements from a set by using discard() method.
Example

//Using the data above.
Days.discard("Sun")
 console.log(Days)

Output

set(['Wed', 'Fri', 'Tue', 'Mon', 'Thu', 'Sat'])

Union of Sets
The union operation on two sets produces a new set containing all the distinct elements from both the sets. In the below example the element “Wed” is present in both the sets.

DaysA = set(["Mon","Tue","Wed"])
DaysB = set(["Wed","Thu","Fri","Sat","Sun"])
AllDays = DaysA|DaysB
 console.log(AllDays)

Output will be as shown,note the result has only one “wed”.

set(['Wed', 'Fri', 'Tue', 'Mon', 'Thu', 'Sat'])

Intersection of Sets
The intersection operation on two sets produces a new set containing only the common elements from both the sets. In the below example the element “Wed” is present in both the sets.

DaysA = set(["Mon","Tue","Wed"])
DaysB = set(["Wed","Thu","Fri","Sat","Sun"])
AllDays = DaysA & DaysB
console.log(AllDays)

Output

set(['Wed'])

Difference of Sets
The difference operation on two sets produces a new set containing only the elements from the first set and none from the second set. In the below example the element “Wed” is present in both the sets so it will not be found in the result set.

DaysA = set(["Mon","Tue","Wed"])
DaysB = set(["Wed","Thu","Fri","Sat","Sun"])
AllDays = DaysA - DaysB
 console.log(AllDays)

Output
When the above code is executed, it produces the following result. Please note the result has only one “wed”.

set(['Mon', 'Tue'])

Compare Sets
We can check if a given set is a subset or superset of another set. The result is True or False depending on the elements present in the sets.
Example

DaysA = set(["Mon","Tue","Wed"])
DaysB = set(["Mon","Tue","Wed","Thu","Fri","Sat","Sun"])
SubsetRes = DaysA <= DaysB
SupersetRes = DaysB >= DaysA
 console.log(SubsetRes)
 console.log(SupersetRes)

Output

True
True

Queue

The queue is a linear data structure where elements are in a sequential manner. It follows the F.I.F.O mechanism that means first in first out.
Below the aspects that characterize a queue.
Two ends:
*front → points to starting element
*rear → points to the last element
There are two operations:
*enqueue → inserting an element into the queue. It will be done at the rear.
*dequeue → deleting an element from the queue. It will be done at the front.
There are two conditions:
*overflow → insertion into a queue that is full
*underflow → deletion from the empty queue
Lets see a code example of this:

// program to implement queue data structure

class Queue {
    constructor() {
        this.items = [];
    }

    // add element to the queue
    enqueue(element) {
        return this.items.push(element);
    }

    // remove element from the queue
    dequeue() {
        if(this.items.length > 0) {
            return this.items.shift();
        }
    }

    // view the last element
    peek() {
        return this.items[this.items.length - 1];
    }

    // check if the queue is empty
    isEmpty(){
       return this.items.length == 0;
    }

    // the size of the queue
    size(){
        return this.items.length;
    }

    // empty the queue
    clear(){
        this.items = [];
    }
}

let queue = new Queue();
queue.enqueue(1);
queue.enqueue(2);
queue.enqueue(4);
queue.enqueue(8);
console.log(queue.items);

queue.dequeue();
console.log(queue.items);

console.log(queue.peek());

console.log(queue.isEmpty());

console.log(queue.size());

queue.clear();
console.log(queue.items);

This will procuce the following results.

[1, 2, 4, 8]
[2, 4, 8]
8
false
3
[]

Stack

In the english dictionary the word stack means arranging objects on over another. Stack is a linear data structure which follows a particular order in which the operations are performed. The order may be LIFO(Last In First Out) or FILO(First In Last Out).
In the following program we implement it as add and and remove functions. We declare an empty list and use the append() and pop() methods to add and remove the data elements.
Pushing into a Stack
Example

let city = ["New York", "Madrid", "Kathmandu"];

// add "London" to the array
city.push("London");


console.log(city);

// Output: [ 'New York', 'Madrid', 'Kathmandu', 'London' ]

POP from a Stack
As we know we can remove only the top most data element from the stack, we implement a python program which does that. The remove function in the following program returns the top most element. We check the top element by calculating the size of the stack first and then use the in-built pop() method to find out the top most element.

let cities = ["Madrid", "New York", "Kathmandu", "Paris"];

// remove the last element
let removedCity = cities.pop();

console.log(cities)         // ["Madrid", "New York", "Kathmandu"]
console.log(removedCity);   // Paris

Linked list

A linked list is a linear data structure, in which the elements are not stored at contiguous memory locations. The elements in a linked list are linked using pointers as shown in the below image:

Traversing a Linked List
Singly linked lists can be traversed in only forward direction starting form the first data element. We simply print the value of the next data element by assigning the pointer of the next node to the current data element.

struct node *temp = head;
printf("\n\nList elements are - \n");
while(temp != NULL) {
  printf("%d --->",temp->data);
  temp = temp->next;
}

Output

List elements are - 
1 --->2 --->3 --->

Insertion in a Linked List
Inserting element in the linked list involves reassigning the pointers from the existing nodes to the newly inserted node. Depending on whether the new data element is getting inserted at the beginning or at the middle or at the end of the linked list, we have the below scenarios.
Inserting at the Beginning
This involves pointing the next pointer of the new data node to the current head of the linked list. So the current head of the linked list becomes the second data element and the new node becomes the head of the linked list.
Example

struct node *newNode;
newNode = malloc(sizeof(struct node));
newNode->data = 4;
newNode->next = head;
head = newNode;

struct node *newNode;
newNode = malloc(sizeof(struct node));
newNode->data = 4;
newNode->next = NULL;

struct node *temp = head;
while(temp->next != NULL){
  temp = temp->next;
}

temp->next = newNode;

Inserting in between two Data Nodes
This involves changing the pointer of a specific node to point to the new node. That is possible by passing in both the new node and the existing node after which the new node will be inserted. So we define an additional class which will change the next pointer of the new node to the next pointer of middle node. Then assign the new node to next pointer of the middle node.

struct node *newNode;
newNode = malloc(sizeof(struct node));
newNode->data = 4;

struct node *temp = head;

for(int i=2; i < position; i++) {
  if(temp->next != NULL) {
    temp = temp->next;
  }
}
newNode->next = temp->next;
temp->next = newNode;

Removing an Item

We can remove an existing node using the key for that node. In the below program we locate the previous node of the node which is to be deleted.Then, point the next pointer of this node to the next node of the node to be deleted.
Example

struct node* temp = head;
while(temp->next->next!=NULL){
  temp = temp->next;
}
temp->next = NULL;

Algorithims

Algorithms are instructions that are formulated in a finite and sequential order to solve problems.
The word algorithm derives itself from the 9th-century Persian mathematician Muḥammad ibn Mūsā al-Khwārizmī, whose name was Latinized as Algorithmi. Al-Khwārizmī was also an astronomer, geographer, and a scholar in the House of Wisdom in Baghdad.

As you already know algorithms are instructions that are formulated in a finite and sequential order to solve problems.
When we write an algorithm, we have to know what is the exact problem, determine where we need to start and stop and formulate the intermediate steps.

There are three main approaches to solve algorithms:
*Divide et Impera (also known as divide and conquer) → it divides the problem into sub-parts and solves each one separately
*Dynamic programming → it divides the problem into sub-parts remembers the results of the sub-parts and applies it to similar ones
*Greedy algorithms → involve taking the easiest step while solving a problem without worrying about the complexity of the future steps

Tree Traversal Algorithm
Trees in python are non-linear data structures. They are characterized by roots and nodes. I take the class I constructed before for the binary tree.
Tree Traversal refers to visiting each node present in the tree exactly once, in order to update or check them.

struct node {
    int data;
    struct node* left;
    struct node* right;
}

There are three types of tree traversals:
*In-order traversal → refers to visiting the left node, followed by the root and then the right nodes.

inorder(root->left)
display(root->data)
inorder(root->right)

*Pre-order traversal → refers to visiting the root node followed by the left nodes and then the right nodes.

display(root->data)
preorder(root->left)
preorder(root->right)

*Post-order traversal → refers to visiting the left nodes followed by the right nodes and then the root node.

postorder(root->left)
postorder(root->right)
display(root->data)

Sorting Algorithm
The sorting algorithm is used to sort data in some given order. It can be classified in Merge Sort and Bubble Sort.

*Merge Sort → it follows the divide et Impera rule. The given list is first divided into smaller lists and compares adjacent lists and then, reorders them in the desired sequence. So, in summary from unordered elements as input, we need to have ordered elements as output.
*Bubble Sort → it first compares and then sorts adjacent elements if they are not in the specified order.

*Insertion Sort → it picks one item of a given list at the time and places it at the exact spot where it is to be placed.
There are other Sorting Algorithms like Selection Sort and Shell Sort.

Searching Algorithms

Introduction to Modern Javascript 101

Marriane Akeyo — Mon, 14 Feb 2022 20:54:34 +0000

Javascript is a lightweight programming language which is dynamically typed.Some of the key benefits of learning javascript include:

JavaScript usage has now extended to mobile app development, desktop app development, and game development. This opens many opportunities for you as Javascript Programmer.
Javascript is everywhere, it comes installed on every modern web browser and so to learn Javascript you really do not need any special environment setup. For example Chrome, Mozilla Firefox , Safari and every browser you know as of today, supports Javascript.
Great thing about Javascript is that you will find tons of frameworks and Libraries already developed which can be used directly in your software development to reduce your time to market.
Javascript is the most popular programming language in the world and that makes it a programmer’s great choice. Once you learnt Javascript, it helps you developing great front-end as well as back-end softwares using different Javascript based frameworks like jQuery, Node.JS etc.
Javascript is everywhere, it comes installed on every modern web browser and so to learn Javascript you really do not need any special environment setup. For example Chrome, Mozilla Firefox , Safari and every browser you know as of today, supports Javascript.
Let us now look at a simple hello world code

console.log("Hello world")

How simple is that right...
Javascript mostly uses the console as noticed from the code above.We are now going to look at the syntax that will enable us write usefull and appealing codes in javascript.
So in javascript one can declare a const which means it is a constant value and thus cannot be changed ,a var which means the value declared is a variable and can be changed or a let which means reassigning of the value is possible.

data types.
In simple terms they explain or demonstrate the type of data being stored.Javascript supports the following datatypes:

NULL which includes an object without any value.
undefined which includes a variable or constant with no value.eg:

var m;

so when we say :

typeof(m);

then, we shall get undefined. Since javascript is a dynamically typed language you do not need to define other datatypes like strings and numbers ,once you write them , the compiler will automatically understand.
However you can convert from one datatype to another, eg: string() to convert to string and parseInt() to convert to int.

Boolean
Unlike in other languages like python, the boolean values are declared using small letters in javascript.ie true or false.

Conditional Statements
Javascript supports conditional statements like if else eg

if (x<3)
{
console.log (true)
}else{
console.log(false)
}

NOTE
Double equals (==) in javascript are use to show equality of values.(===) are used to check the type of values wherease (=) is used during assignment of values.

Logical Operators
Javascript makes use of the &&(and ) and ||(or) operators as better illustrated bellow.

if (x<3 && y<10)
{
console.log (true)
}
elseif(z==8 || w<10)
{
console.log(true)
}
else{
console.log(false)
}

Functions
We can declare functions in javascript using two diffrent ways:

function (name)
{
console.log(`Hello ${name}`)
}

NOTE the above function accepts parameters.
The other way is using the arrow key function.

username = () =>
{
console.log(`Hello ${name}`}
}

Operators
Javascript supports all the common operators ie +,-,*,/,increment(++) and decrement(--).eg

var num = 0
while (num < 10)
{
console.log(num);
num++;
}

Extra
You might be wondering ...when do I use var,let or even const when writing your javascript code. I would advise you use const when you don't want your values to be changed. Use var when working with the global scope and finally, use let when you are referring only to the block scope .

I hope you got some insight on the modern javascript.
Thank you and happy coding!!!

Introduction to modern python

Marriane Akeyo — Fri, 11 Feb 2022 21:34:30 +0000

Python is an interpreted, high-level language created by Guido van Rossum and released in 1991. It is dynamically typed and garbage collected.Python programs have the extension.py and can be run from the command line by typing python file_name.py.The common features provided by python include:
-Simplicity: This includes less of the syntax of the language and more of the code.
-Open Source: A powerful language and it is free for everyone to use and alter as needed.
-Portability: Python code can be shared and it would work the same way it was intended to.
-Embeddable & Extensible: Python can have snippets of other languages inside it to perform certain functions.
-Being Interpreted: The worries of large memory tasks and other heavy CPU tasks are taken care of by Python itself leaving you to worry only about coding.
-Huge amount of libraries:from data science all the way to web development.
-Object Orientation: Objects help breaking-down complex real-life problems into such that they can be coded and solved to obtain solutions.
A simple hello world in python can be written as:

print("Hello world")

What therefore do we mean when we say modern python?How is it helpful in a programmers life and why should we prefer it to the old python language?
Modern python simply means agreed modern best practices that are used when coding in python in order to give you better results .They include:

Tooling

1.Use of python3
Python3 is the latest existing version of python that is being supported by all the major libraries and frameworks.Hence advisable for use.
2.Use of virtual environment
It enables you to define separate sets of packages for each Python project, so that they do not conflict with each other.
There are several tools that help you to manage your Python projects, and use virtual environments. The most popular are pipenv and poetry. Poetry is considered to be better designed, but pipenv is more widely supported. If you prefer, you can also manually set up and manage virtual environments.
3.Formatting your code
Use a formatting tool with a plugin to your editor, so that your code is automatically formatted to a consistent style.
Black is being adopted by the Python Software Foundation and other projects. It formats Python code to a style that follows the PEP 8 standard, but allows longer line lengths. Use Black for new projects.

Language Syntax

1.Format strings with f string

The new f-string syntax is both more readable and has better performance than older methods.A good example of the fstring includes:

name = "nduta"
print(f'Hi, my name is {name}')

You can use either the double or single quotes.
2.Use enum or Named Tuples for Immutable Sets of Key-Value Pairs.
The properties of an enumeration and named tuples are useful for defining an immutable, related set of constant values that may or may not have a semantic meaning.
3.Create Data Classes for Custom Data Objects
The data classes feature enables you to reduce the amount of code that you need to define classes for objects that exist to store values.
4 Use collections.abc for Custom Collection Types
They provide the components for building your own custom collection types.These classes are well dvocated for because they are fast and well-tested.
5.Use breakpoint() for Debugging
This function drops you into the debugger,both the inbuilt and external debuggers can use these breakpoints.An example would be:

foo()
breakpoint()
bar()

The above code simply states that python will enter into the debugger after executing foo() and before executing bar().

Application design

1.Use Logging for Diagnostic Messages, Rather Than print()

The built-in print() statement is convenient for adding debugging information, but you should include logging in your scripts and applications. Use the logging module in the standard library, or a third-party logging module.

2.Only Use asyncio Where It Makes Sense

The asynchronous features of Python enable a single process to avoid blocking on I/O operations. You can achieve concurrency by running multiple Python processes, with or without asynchronous I/O.

To run multiple Web application processes, use Gunicorn or another WSGI server. Use the multiprocessing package in the Python standard library to build custom applications that run as multiple processes.

Code that needs asynchronous I/O must not call any function in the standard library that synchronous I/O, such as open(), or the logging module.

If you would like to work with asyncio, always use the most recent version of Python. Each new version of Python has improved the performance and features of async.

Libraries

1.Handle Command-line Input with argparse

The argparse module is now the recommended way to process command-line input. Use argparse, rather than the older optparse and getopt.
The optparse module is officially deprecated, so update code that uses optparse to use argparse instead.

2.Use pathlib for File and Directory Paths

Use pathlib objects instead of strings whenever you need to work with file and directory pathnames.

Consider using the the pathlib equivalents for os functions.

The existing methods in the standard library have been updated to support Path objects.

To list all of the the files in a directory, use either the .iterdir() function of a Path object, or the os.scandir() function.
3.Use os.scandir() Instead of os.listdir()

The os.scandir() function is significantly faster and more efficient than os.listdir(). Use os.scandir() wherever you previously used the os.listdir() function.

This function provides an iterator, and works with a context manager:

import os

with os.scandir('some_directory/') as entries:
    for entry in entries:
        print(entry.name)

The context manager frees resources as soon as the function completes. Use this option if you are concerned about performance or concurrency.

The os.walk() function now calls os.scandir(), so it automatically has the same improved performance as this function.
4.Run External Commands with subprocess
The subprocess module provides a safe way to run external commands. Use subprocess rather than shell backquoting or the functions in os, such as spawn, popen2 and popen3. The subprocess.run() function in current versions of Python is sufficient for most cases.
5.Use Requests for HTTP Clients
Use the requests package for HTTP, rather than the urllib.request in the standard library.
6.Test with pytest
The pytest package has super-ceded nose as the most popular testing system for Python hence most advisable .You can use unittest for situations where you cannot add the pytest library to your project.
This is just a sample of the modern improvements done in python . As they always say , practice makes perfect.So working with the language and its packages will give you a clear picture of the dos and don't s.If you haven't started using the library ,my advise would be to give it a shot and see how far it goes.The language also has a wide community which is willing to help once you get stuck.
Those who are advanced in the same my take would be to be on the look out since change is inevitable especially in the tech industry.
I hope this gives you some incite.