DEV Community: Fullstack Dev

Separation of concern & Murphy's law

Fullstack Dev — Sun, 06 Oct 2024 15:51:34 +0000

Anything that can go wrong will go wrong is what Murphy's law tells us. This comes as a principle in software engineering known as separation of concern.

Separation of concern is a design principle in programming used to break an application into sections or modules.

SoC ensures that each section is responsible for a specific aspect of functionality or behavior. this ensures that the applications is easy to maintain and easy to scale.

This principle allows us to create reusable components that can be used across the entire application.

Murphy's Law in Separation of Concerns

When multiple aspects or functionalities are mixed or are in one file in your code something will eventually break.

By separating concerns for example by separating the express application and the web server the chances that everything will break Is reduced.

if there's an issue in your express app it won't affect the logic of your application. The more you compartmentalized the behavior or responsibility of your application the chances of a one failure affecting your entire application becomes slimmer.

example where no SoC was applied:

const express = require('express');
const app = express();

// Application logic (handling routes)
app.get('/hello', (req, res) => {
  res.send('Hello, World!');
});

// Server logic (listening on a port)
app.listen(3000, () => {
  console.log('Server is running on port 3000');
});

Here's how murphy's law works

if the server failure happens (port is already in use), you won't be able to test your routes and the entire app tops working.

Example where SoC was applied

//app.js
const express = require('express');
const app = express();

// Application logic (handling routes)
app.get('/hello', (req, res) => {
  res.send('Hello, World!');
});

module.exports = app;

//server.js
const app = require('./app'); 
// Server logic (listening on a port)
app.listen(3000, () => {
  console.log('Server is running on port 3000');
});

if the server fails to start the application will still be able to work since your application logic is safe.
You can still test your application logic without directly running the server by using testing frameworks like jest and supertest

const request = require('supertest');  
const app = require('./app');          

// Test case for GET /hello
test('GET /hello should return "Hello, World!"', async () => {
  const response = await request(app).get('/hello');  
  expect(response.text).toBe('Hello, World!');        });

Master SQL Querying: A Comprehensive Guide.

Fullstack Dev — Thu, 25 Jul 2024 19:39:41 +0000

At first glance, SQL might seem simple and not hard at all, that is true until you find yourself in front of complex database structures or having to write complex queries. It does get confusing or difficult sometimes (Yes even experienced people can find themselves scratching their heads).

In this guide we're gonna lay out all the concepts so you can pick up SQL in no time.

SELECT basics

A SELECT statement retrieves data from a table found in your database. Each table has rows and columns. You can select specific columns from a table.

Here is the Syntax:

SELECT "column_name", "column_name"
FROM "table_name";

Keep in mind that you can select one or multiple columns.

Here is an example of our table named general_info

Country	Capital	Year_of_Independence
United States	Washington, D.C.	1776
France	Paris	1789
India	New Delhi	1947
Brazil	Brasilia	1822
China	Beijing	1949
Australia	Canberra	1901

Our SELECT statement will need the column names and the table name from which we want to retrieve data can select one or multiple columns.

SELECT "Country", "Year_of_Independence" FROM "general_info";

This query will return all the names of the countries and their year of independence.

Select with WHERE clause

A WHERE clause is a condition that must be met in order for a row to be present in the result set of the query. In other words it helps filter the result and only returns the rows that match the condition you specified.

Country	Capital	Year_of_Independence
United States	Washington, D.C.	1776
France	Paris	1789
India	New Delhi	1947
Brazil	Brasilia	1822
China	Beijing	1949
Australia	Canberra	1901

We're going to use the same Table general_info to look at an example.

Imagine we want to return the capital of a country that got its independence in 1901.

SELECT "Capital"
FROM "general_info" 
WHERE "year_of_independence" = 1901;

Functions

Functions are built-in blocks of code, each of these functions execute a specific task.

Function Name	Description
SUM	Calculates the sum of a set of values
AVG	Calculates the average of a set of values
MIN	Returns the smallest value in a set
MAX	Returns the largest value in a set
COUNT	Counts the number of rows or non-null values
ROUND	Rounds a number to a specified decimal place
TRUNCATE	Truncates a number to a specified decimal place
CONCAT	Combines two or more strings into a single string
LENGTH	Returns the length of a string
UPPER	Converts a string to uppercase
LOWER	Converts a string to lowercase
SUBSTRING	Extracts a substring from a string
TRIM	Removes leading and trailing spaces from a string
LTRIM	Removes leading spaces from a string
RTRIM	Removes trailing spaces from a string
LIKE	Used for pattern matching in strings
YEAR	Extracts the year from a date
MONTH	Extracts the month from a date
DAY	Extracts the day from a date
HOUR	Extracts the hour from a time
MINUTE	Extracts the minute from a time
SECOND	Extracts the second from a time
CASE WHEN	Performs conditional logic
COALESCE	Returns the first non-null value from a list of expressions
NULLIF	Returns NULL if two expressions are equal, otherwise returns the first expression

Here are some examples:

SUM

Let's calculate the total of prices In the fruits table.

SELECT SUM("Price") AS 'total'
FROM "Fruits";

The AS allows us to rename the result column as total.

AVG

SELECT AVG("Price") AS 'Average_price'
FROM "Fruits";

Here we're getting the average price of all the fruits in the Fruits Table.

UPPER

SELECT UPPER("Fruit") AS "Fruit_name_upper"
FROM "Fruits";

This query converts all the fruit names in the Fruits Table to uppercase.

TURNCATE

TRUNCATE in the context of numbers means to remove digits after a specified decimal place without rounding.

SELECT TURNCATE("name_of_column",2) AS 'New_Value'
FROM "table_name";

We can also use TURNCATE to empty a table without messing with or deleting the table structure.

TURNCATE TABLE "Fruits";

All the data will be removed from the table. It is an irreversible operation.

SELECT with GROUP BY

The GROUP BY clause groups rows that have the same value, it categorizes them based on a specific column of your choosing.

Imagine a list of fruits and their prices.

Table name: Fruits

Fruit	Price
Apple	0.5
Banana	0.3
Orange	0.4
Grape	0.4
Strawberry	0.5
Mango	0.3

Using this example, we can group the fruits by price.

SELECT "Price"
FROM "FRUITS"
GROUP BY "Price";

The result would look like that:

Price
0.3
0.4
0.5

What essentially happened is that all the rows that had the same prices were put together into a single group.

SELECT with JOIN

Sometimes you need to view the data of more than just one table to get the required data.

Table 1: Customers

CustomerID	CustomerName	City
1	John Doe	New York
2	Jane Smith	Los Angeles
3	Kate Johnson	Chicago
4	Ilene Brown	Dallas
5	David Michaels	Houston
6	Olivia Davids	Miami

Table 2: Orders

OrderID	CustomerID	OrderDate
101	1	2023-11-08
102	2	2023-12-23
103	3	2023-11-25
104	1	2023-12-18
105	4	2023-11-22
106	2	2023-12-15

In this example, the Customers table is related to a row of the Orders table. If we want to find out Who placed the order and When the order was placed, we need to join the Customers table to the Orders table on the common field that is the CustomerID in both tables.

SELECT Customers.CustomerName, Orders.OrderDate
FROM Customers JOIN Orders
ON Customers.CustomerID = Orders.CustomerID;

This would be the result of this query:

CustomerName	OrderDate
John Doe	2023-11-08
John Doe	2023-12-18
Jane Smith	2023-12-23
Jane Smith	2023-12-15
Kate Johnson	2023-11-25
Ilene Brown	2023-11-22

LEFT JOIN

A LEFT JOIN returns all the rows from the left table even if there are no matches in the right one.
This means that the returned result will have all the data from the left table; if there's a match in the right table then the relevant columns are filled and if there is no match found in the right table the columns from the right table will have null values.

SELECT Customers.CustomerName, Orders.OrderID
FROM Customers
LEFT JOIN Orders
ON Customers.CustomerID = Orders.CustomerID;

This will help us know which customer did not place an order.

CustomerName	OrderID
John Doe	101
John Doe	104
Jane Smith	102
Jane Smith	106
Kate Johnson	103
Ilene Brown	105
David Michaels
Olivia Davids

RIGHT JOIN

A RIGHT JOIN is essentially the opposite of a LEFT JOIN. It returns all rows from the right table even if there are no matches in the left table.
This means that the returned result will have all the data from the right table; if there's a match in the left table then the relevant columns are filled and if there is no match found in the left table the columns from the left table will have null values.

SELECT Customers.CustomerName, Orders.OrderID
FROM Customers
RIGHT JOIN Orders
ON Customers.CustomerID = Orders.CustomerID;

This query will return all orders, including those without corresponding customers. If an order doesn't have a corresponding customer, the CustomerName column will be NULL.

Subqueries

Subqueries queries are queries nested inside another query.

Subquerying with JOIN

Let's look at one using the JOIN statement and our Customers and Orders tables

SELECT CustomerName, TotalOrders
FROM Customers
JOIN(SELECT Orders.CustomerID, COUNT(Orders.CustomerID) AS TotalOrders
FROM Orders
GROUP BY CustomerID) AS OrderCounts
ON Customers.CustomerID = Orders.CustomerID;

Let's break our query down by starting from the inner query to the outer one:

SELECT Orders.CustomerID,COUNT(Orders.CustomerID) AS TotalOrders FROM Orders: We are calculating the total number of orders made by each customer in the Orders by using a built-in function named COUNT.
As TotalOrders, is used to rename the column we got as a result.
GROUP BY CustomerID, this query groups the result by CustomerID this will allow us to know how many Orders each Person Placed.
AS OrderCounts Gives an Alias to the subquery allowing us to treat it like a normal table temporarily.
SELECT CustomerName, TotalOrders FROM Customers We are selecting the CustomerName From the Customers table and the TotalOrder From the temporary table we just created that has the column TotalOrders.
Lastly ON Customers.CustomerID = Orders.CustomerID Joins both tables by using their common field which is CustomerID.

Our result would be:

CustomerName	TotalOrders
John Doe	2
Jane Smith	2
Kate Johnson	1
Ilene Brown	1
David Michaels	0
Olivia Davids	0

Subquerying with IN

Let's find customers who placed orders after the date 2023-11-08.

SELECT CustomerName 
FROM Customers
WHERE CustomerID IN (
SELECT CustomerID
FROM Orders
WHERE OrderDate > '2023-11-08'
);

SELECT CustomerID FROM Orders WHERE OrderDate > '2023-11-08': This part retrieves the CustomerID from the Orders table where the OrderDate is after November 08, 2023.
SELECT CustomerName FROM Customers WHERE CustomerID IN (...): This part selects the CustomerName from the Customers table where CustomerID is found in the result of the subquery.

A Deeper Dive

After Presenting the basic concepts of how each provided statement above works let's see more statements at the works.

Here are The same tables with a little modification.

Customers Table

CustomerID	CustomerName	City
1	John Doe	New York
2	Jane Smith	Los Angeles
3	Kate Johnson	Chicago
4	Ilene Brown	Dallas
5	David Michaels	Houston
6	Olivia Davids	Houston

Orders Table

OrderID	CustomerID	OrderDate	OrderAmount
101	1	2023-11-08	100
102	2	2023-12-23	150
103	3	2023-11-25	80
104	1	2023-12-18	200
105	4	2023-11-22	120
106	2	2023-12-15	90

Basic SELECT Queries

Get all Customers' data:

SELECT * FROM Customers

The asterisk * means all data.

Get specific Columns:

SELECT CustomerName, City 
FROM Customers;

Filtering Data

Get orders placed after a specific date:

SELECT * FROM Orders 
WHERE OrderDate > '2023-11-08';

Find Customers in a Specific City:

SELECT * FROM Customers
WHERE City = 'Los Angeles';

Find Orders With an order amount greater than 100:

SELECT * FROM Orders WHERE OrderAmount >100;

Sorting Data (ORDER BY)

When sorting Data with ORDER BY We have the DESC and ASC Keywords.

ASC Sorts the data from A to Z
DESC Sorts the data from Z to A

ASC is the default sort order.

Sorting Customers by name in descending order:

SELECT * FROM Customers 
ORDER BY CustomerName DESC;

Aggregating Data (GROUP BY, HAVING,other functions)

Group By

We've already come across GROUP BY. It will categorize data and group them based on a specific column such as an ID for example.

Find the total order amount for each customer:

SELECT CustomerID, SUM(OrderAmount) AS TotalOrderAmount
FROM Orders
GROUP BY CustomerID;

HAVING

HAVING is similar to the WHERE clause but it uses the results of GROUP BY rather than individual rows. This means that HAVING Filters the groups made by GROUP BY and checks if the condition provided is applied.

Find Customers with a total order amount greater than 200:

SELECT CustomerID, SUM(OrderAmount) AS TotalOrderAmount
FROM Orders
GROUP BY CustomerID
HAVING SUM(OrderAmount) > 200;

INNER JOIN

We've previously seen LEFT JOINs and RIGHT JOINs, and how they return null values if the right or left row is missing a value.

What if we want to return the rows that only have matching values in both tables and avoid the null values?

This is Where INNER JOIN comes into play.

SELECT Customers.CustomerName, Orders.OrderDate
FROM Customers
INNER JOIN Orders
ON Customers.CustomerID = Orders.CustomerID;

COALESCE FUNCTION

COALESCE function to replace null values with a specific one.

SELECT Customers.CustomerName, COALESCE(Orders.OrderDate, 'No Orders') AS OrderDate
FROM Customers
LEFT JOIN Orders
ON Customers.CustomerID = Orders.CustomerID;

Correlated Subqueries

A correlated subquery is when a subquery references a column from the outer query for example:

SELECT CustomerName
FROM Customers
WHERE EXISTS (
  SELECT *
  FROM Orders
  WHERE Orders.CustomerID = Customers.CustomerID AND OrderDate > '2023-11-30'
);

WITH

The with clause also known as a common table expression, makes it possible to temporarily define result sets that can be referenced within a SELECT, INSERT, UPDATE, or DELETE statement.

This means that it allows you to break down complex queries into smaller ones and make them more manageable.

Let's say we want to find customers who have placed orders after a specific date using common table expression.

WITH RecentOrders AS (
  SELECT CustomerID
  FROM Orders
  WHERE OrderDate > '2023-11-30'
)
SELECT CustomerName
FROM Customers
WHERE CustomerID IN (SELECT CustomerID FROM RecentOrders);

The WITH clause improves readability and performance and it only exists for the duration of the query.

Complex Grouping

Let's say we want to calculate the total sales for each month and year in the Orders Table.

SELECT
    YEAR(OrderDate) AS OrderYear,
    MONTH(OrderDate) AS OrderMonth,
    SUM(OrderAmount) AS TotalSales
FROM
    Orders
GROUP BY
    YEAR(OrderDate), MONTH(OrderDate);

We will select three columns year of the order, the month of the order, and the total sales amount for that month and year. Then we will need to group by the year and month.

In conclusion, understanding the fundamentals and basic querying at the start of your journey to learn SQL is the building block of Mastering it. Understanding the concepts laid out in this article will give you a significant step forward and will help you become proficient in database management. The KEY is to PRACTICE and CONSISTENT APPLICATION.

How to optimize your MERN workflow with a solid architecture

Fullstack Dev — Fri, 12 Jul 2024 20:06:41 +0000

We've probably encountered a MERN stack project on the internet that was perhaps the messiest thing we've seen. Where everything was crammed into one single file. Where the front-end and back-end logic were squeezed together, files and variables have random names, making the codebase hard to explore and no error handling whatsoever.

This is why a strong foundation has to be created before developing a MERN stack application or any other type of application for that matter.

A disorganized project structure can easily turn an app into a mess. With a proper template, you can develop an application that is functional, scalable, manageable, and easy to work with.

Following standard practices can considerably improve the developer experience and code quality. Let's dive right in.

Principles of project structure

Before we go into these principles, here are some best practices you should follow to improve code readability and make your project easier to maintain.

Best practices

Data management is the proper handling of data flow and status.
Use Linting and formatting tools to enforce code standards.
Use version control like Git and Bitbucket to manage your code.
implement unit tests to be able to find issues early on prevent unpredictable behavior from your code and ensure code stability
Error management requires putting in place robust error-handling procedures.

Separation of concerns

Separation of concerns or SoC, is an essential concept in programming that recommends breaking down a huge system into smaller, more manageable components, each with a role. Such as separating the Front-end from the Back-end.

Organization

Creating a directory structure with consistent naming convention. Component-based architecture requires breaking down the user interface into reusable components. Routing includes setting up effective navigation within the app.

Project architecture

This is how our project architecture should look.

project-name/
├── client/
│   ├── public/
│   │   ├── index.html
│   │   └── assets/
│   ├── src/
|   │   ├── services/
│   │   ├── components/
│   │   ├── pages/
│   │   └── App.js
│   └── index.js
├── server/
│   ├── controllers/
│   ├── models/
│   ├── routes/
│   ├── middleware/
│   └── index.js
├── package.json
├── .env
└── README.md

`
Let's talk about each folder and file in our structure and their role.

Project Root

package.json: Contains information about the project, its dependencies, and scripts for running the application.
.env: contains environment variables like API keys and other information.
README.md: is the project documentation.

Client Directory (frontend)

public

index.html: The main HTML file for the application.
assets: Folder that can contain images, fonts, and other static resources.

src

services: For making HTTP requests to the backend using Axios or Fetch.
components: Reusable components.
pages: Main views or screens of the application.
App.js: The main entry point for the React application.
index.js: The entry point for the client-side build process.

Server Directory (backend)

controllers

Contains logic for handling incoming HTTP requests and responding to them.

models

Defines the data structures (schemas) for the database.

routes

Defines the API endpoints and maps them to controller functions.

middleware

Contains functions that process requests before they reach the controllers ( like error handling).
index.js: The entry point for the Node.js server, typically responsible for setting up the Express app, database connections, and route handlers.

This architecture is MVC (Model-View-Controller) on the backend and component-based architecture on the frontend.

The MVC architecture has a clear division, the three components are interconnected and each is responsible for a different aspect of the application making it easier to maintain.

How machines Learn: A look into machine learning.

Fullstack Dev — Wed, 08 May 2024 20:44:38 +0000

We've all witnessed how powerful machine learning is by using applications like ChatGPT, Gemini, Midjourney, and LeonardoAI. Now you might or may not be familiar with how they work, but essentially algorithms are the secret sauce behind machine learning applications.

A computer can only perform certain tasks in a certain order when it is given a set of precise instructions, this is what is known as algorithms. Think of algorithms as a chocolate cake recipe, which is a step-by-step plan you follow that should give you the same outcome every time. You can add certain ingredients to a recipe to make it better which is similar to feeding the algorithm more data to make it give you a better outcome.

Now when it comes to machine learning there are three terms that you will constantly come across When you're starting out in this field:

Supervised learning
Unsupervised learning
Reinforcement learning These are the most common methods that machines learn.

1. Supervised Learning

A known link between input and output is used in supervised learning. Labeled data is valuable in this situation because it helps the algorithm learn from the "correct answers" in the training set and use those lessons for predicting future input data.

The two key goals of supervised learning are Regression and Classification.
There are two main goals for regression analysis.
Regression is used to understand the type of relationship between the variables, and to estimate the value of the dependent variable based on the independent variables, and evaluating the relevance and strength of this relationship.

In classification, the goal is to sort findings based on their characteristics into predefined groups or categories.

Here's a simple example of how a supervised algorithm works.
Let's imagine we have a set of images where each depicts either a banana or an apple.

Create a set of data marked as "correct" data, consisting of images of apples and bananas together with the matching class names.
Give the model the labeled training dataset. Between the input which is the picture and the output which is the class, the machine learning algorithm starts to "see" patterns. The algorithm may pick up connections, such as the color or the shape of the fruits.
Measure how well the model predicts the class by running it over some unknown data.

2. Unsupervised Learning

The main distinction between supervised and unsupervised learning is the "correct" data we provided. Unsupervised learning algorithms just need input data. This learning method works well for cases when we don't know exactly what the solutions should look like. Its goal is for the algorithm to learn from the data.

Let's keep it simple and stay with the fruit example.
Imagine you're sorting a basket of fruits. There are no labels on the items identifying an orange or an apple. Unsupervised learning is like sorting these fruits based on their similarities, like color, shape, or size.
Filtering or sorting these fruits based on similarities in size, color, or form is similar to unsupervised learning. In machine learning, your data points are the fruits, and the characteristics you use to group them are the sorting criteria.

3. Reinforcement learning

In reinforcement learning, there is an agent which is the learner that learns through trial and error in an environment. This agent uses the algorithm to make decisions.

Here's an example:
Picture a toddler starting to walk. The kid gets a positive reward such as a hug a kiss or even verbal appreciation for taking steps and negative reinforcement such as a fall when taking a wrong step for errors, but no instructions are given. The kid gradually picks up the best techniques to reach the goal of walking.

The goals of reinforcement learning are different from those of unsupervised learning. In comparison with unsupervised learning, which aims to identify clusters in the data, reinforcement learning looks for a suitable action model that increases the agent's total reward.

What i actually do to stay productive

Fullstack Dev — Tue, 07 May 2024 17:51:13 +0000

It's no secret that sometimes work can be overwhelming. It's no secret either that sometimes you'd have so much work you don't even know where to start. It is like that for everyone, some people can manage just fine, and other times you just have to put in a little or A LOT MORE extra effort to get things done.

It took me some time to figure out what works for me, even the things and techniques I saw online looked stupid to me, and even though they wouldn't work until I tried them. I am now going to share what worked for me and to be completely honest what worked for me might now work for you, but that is OKAY don't be discouraged. It's a trial and error kind of thing, you keep trying things until you find and stick to what works for you.

1. Learn the ins and outs of your environment

Learn how your IDE/code editor works. Learn the hot keys and the navigation shortcuts. You might be asking "What's the big deal?". Trust me it is a big deal, using fewer mouse movements and more shortcuts will move a lot faster and lessen the chances of making errors, and will also reduce the strain on your wrist.

2. Make a list

"Make a list" as in not in actually making a list. Take a task and break it into smaller chunks. Take a chunk and estimate how much time it would take you.

Prioritize Your tasks and break them down
Do the chunk that takes less time
If a chunk is taking more time than expected then It can be broken into smaller chunks
Write off the finished chunks on the list.

3. Time management

When I first heard of this technique, I laughed at it and was very doubtful it would work. The Pomodoro Technique turned out to be very useful to me. We all know how it works, but in case you don't know it here's how it works.

Set a timer for 25 minutes
Work for 25 minutes without any interruptions
After 25 minutes take a small break preferably between 3 to 5 minutes
After 4 pomodoros, meaning after 2 hours of work take a longer break which would be between 20 to 35 minutes.

If you are a doom-scroller aka someone who likes to spend time on social media and get stuck scrolling you can use your longer breaks to scroll through social media😂

** Do not use the small breaks to scroll through social media it will interrupt your productivity. Instead, get up and move around. Walk around and drink water! It is important to stay hydrated to keep your mind sharp. More importantly, give your eyes a break from the screen during your short breaks.**

4. Notifications

Put your phone on Do not disturb while working or silence all the apps in your settings so that you do not receive notifications. Unless It is urgent calls or urgent texts do not interrupt your workflow, it will really be hard to regain focus after that. You can also leave your phone in another room (That doesn't mean you can open social media in the browser😂)

5. Stay Physically active

Try to find some time to work out during the day or at night. Working out doesn't just keep your body in shape but also keeps your brain sharp. Staying physically active will improve your memory and keep you creative. It is also a way to relieve your stress and keep your mental health in check.

6. Listen to your body

When you feel like you are tired and need a break, take a break. Do not push beyond what you can handle. It is a bad idea that will lead to burnout. Burnout is very real and it can be difficult to recover from if you do not listen to your body. So When you feel like you need a break, take a break.

At the end of the day, You have to learn how to prioritize your tasks and how to make your life easier. Work smarter not harder right? Experiment with all the techniques you can find you might find some that work for you if my advice isn't fit for you.
You can also try all the tools out there on the internet but Focus and dedication come from within you and not from outside and certainly not from an app.

I hope you found this useful! Thank you for reading! Would love to hear your opinions.

Everyone's Learning Python: 7 Reasons to Catch Up

Fullstack Dev — Mon, 06 May 2024 17:36:03 +0000

As one of the most popular programming languages, Python is one of the best to learn especially for beginners. Python delivers a special mix of power and accessibility, making it suitable for broadening your skill set. Here are 7 reasons why you should dive in and explore Python.

1. Simple enough for beginners

Python has an easy-to-read syntax and reads practically like plain English, in contrast to some languages that seem complex to read even if you're new or somewhat experienced in programming. The simplicity of the syntax allows you to concentrate more on the concept and idea you're trying to achieve rather than get stuck on the syntax.

2. Flexibility

One of the main benefits of Python is its wide range of applications. Python can be used for multiple purposes such as data research, machine learning, web development, and automation. Because of its flexibility, it's a great tool for looking to improve skills.

3. Data Science

Python is one of the leading data science languages in data science. Python allows you to easily analyze, and manipulate data using the strong and large range of libraries it has, such as NumPy, Pandas, and Matplotlib.

4. Artificial intelligence and Machine Learning

Python is an essential language for machine learning and artificial intelligence, making it both AI and ML-ready. Because frameworks like TensorFlow and PyTorch use Python's capabilities so makes it the most suitable language for developing intelligent applications.

5. Large collections of Libraries and Frameworks

The Python ecosystem includes many frameworks and libraries that make development easier. Almost every coding work can be made easier with a Python library, from web development with Django to web scraping with Beautiful Soup.

6. Awesome community

When it comes to programming communities, the Python community is among the friendliest and most supportive. When working with Python, you won't be far from support and direction thanks to the abundance of online forums, tutorials, and communities.

7. Career

Python has great job opportunities because of the growing demand for programmers in many of fields. Here's how having Python skills could get you good chances and high pay:

Python is a popular choice for skills in modern areas like AI and data science as we previously mentioned, and it's used by startups as well as major IT giants.

Since Python is very popular across the world, developing skills with Python could bring forward remote work or if you're looking to leave your country it can bring the possibility of relocation.

In conclusion, Python is a great investment for anybody looking to expand their skillset or get into IT given it has great flexibility, the demand keeps increasing, and it has job opportunities that pay well. Begin your Python dive today to open the door to the future you imagine.

Linux Ninja: Master Text Manipulation from Beginner to Badass in the CLI

Fullstack Dev — Fri, 03 May 2024 21:28:29 +0000

Standard out

From your command line you are able to create a new file and insert text into it.

$ echo Hello from the command line > test.txt

If you try this command in your terminal, you'll notice a new file named test.txt was created and if you open this file, you'll see the text Hello from the command line inside of it.

echo prints the text Hello from the command line to the screen.
By default the echo takes the input (standard input or stdin) from the keyboard and returns the output (standard output or stdout) to the screen.

> is a redirection operator that allows us to send the standard output to go into a file.

if you want to add more text to the file test.txt you can use the >> to append text to it instead of overwriting the file.

Standard input

Just like the standard output redirection operator > we can also use the is operator as well < and what is it called? yes you guessed it! Standard input redirection operator!

Let's look at an example

$ cat < test.txt > readme.txt

if you don't know what the cat command does Check out my article 17 commands to master the Linux Command Line

Here we redirected test.txt file to be our standard input and the output of this file Hello from the command line will be redirecte to the readme.txt

so if we tried to read the readme.txt file it will have Hello from the command line as output

$ cat readme.txt

Standard Error

So far we've seen 2 different streams standard input (stdin) that receives data as input < and standard output which is what you see on the terminal and can be redirected to a file using >.

Standard Error or stderr is specifically for error messages produced by program on the terminal even if stdout is redirected. Here's an example:

$ ls /nonexistent_directory/ > test.txt

The error will be No such file or directory

You can redirect an error message to the file by using file descriptors.
What are file descriptors you may ask? Well a file descriptor is a positive number that is used to access a file. The file descriptors for stdin stdout and stderr is 0 1 and 2 respectively.

To redirect the error message to a file:

$ ls /nonexistent_directory/ 2> test.txt

The error message should be inside test.txt

We can also send the error message to the file and the result of ls with it as well. (even if the directory doesn't exist there might be a minimal output which is not common). Common output will be an empty string or a new line.

$ ls /nonexistent_directory/ > test.txt 2 > &1

if we want to get rid of sderr messages you can use this command

$ ls /nonexistent_directory/ 2 > /dev/null

Pipe and Tee

The pipe operator is represented by |, it acts as a connector by taking the output of a command and sending it as the input to another program.

let's list the details about the dev directory.

$ ls -la /dev/

Now you can use the pipe and the 'less' command to navigate through the list using the up and down arrow keys.

$ ls -la /dev/ | less

You can send the output of of the command to a file. You can write of this command to two streams which will allow you to see the result of the command on the terminal and then in the file.

$ ls -la /dev/ | tee my_output.txt

Environment

An environment variable (PATH) exists on your system. When you write a command name this variable informs the computer where to look for the programs.
If you download a program and place it in a directory where the variable PATH does not check and you try to use the tool it by typing its name in the terminal, what typically happens is thet the computer searches for it in the listed directory in PATH and since where you placed it is not checked by path you will get command not found

$ echo $HOME
$ echo $USER
$ echo $PATH

The home directory path, your username and a list of path separated by colons that your system searches when it runs a command will be displayed on the terminal respectively.

cut

The command cut is used for extracting pieces of text from files. To grab the pieces of data the cut command does it based on fields which is a descrete pieve of data within a line such as a comma, tab, semicolon etc...
We use the flag -f which stands for fields to tell cut what to extract. By default cut uses the tab as delimiters to seperate fields.

For example, we created a file named text.txt with some data in it

$ echo 'The bear jumped over the fence; The   snake followed it to the forest' > text.txt

$ cut -f 2 text.txt

The command using cut will extract the word snake. Let's break it down further.
We told cut to use the default delimiter which is tab by using -f and not specifying a delimiter and 2 stands for the second column which is what comes after the ; semicolon. since the word snake is preceeded with tab then it will be extracted.

Now let's specify a delimiter other than tab.

$ cut -f 1 -d ";" text.txt

Here we're telling cut to select the first column 1 and assigned a new delimiter instead of tab which is the ; by using the -d flag. The output will be The bear jumped over the fence.

Paste

The paste command appends lines in a file for example we have a file named testing.txt

Hello

 I am

 Learning

 How to use

 Linux commands

Let's say we want all the content of testing.txt merged into one line

$ paste -d ' ' -s testing.txt

Here we specified the the delimiter to be a space. There is a space from the second line down to the end in testing.txt. paste will make the testing.txt a oneliner by bringing each sentence preceeded by a space to be appended to the first sentence.

Head

The head command will show you the first 10 lines in a file by default. You can also modify how many lines you want to see. For example you can choose to see 20 lines instead of 10 by using the -n flag (number of lines).

$ head path_of_file
$ head -n 20 path_of_file

Tail

The tail command is the same as the head command but it shows you the last 10 lines of a file instead of the first 10.

Expand and unexpand

Let's say you want to convert tabs in a file to spaces

$ expand name_of_file.txt

If you want to convert the spaces back to tabs you can use this command

$ unexpand -a name_of_file.txt

Joining and splitting files

If you have two text files you want to combine you can use this command

$ join text_file_1.txxt text_file_2.txt

They will be joined by fields meaning the first line of both files will be on the first line, the second line of both files will be joined on the second line etc..

This command will split a file into different files the default naming of the files will start with x followed by alphabetical suffixes aa, ab etc...

-a #: Sets the number of characters used in the suffix (default is 2).

$ join -t ',' accounts.txt logged.txt

-t ',': defines the delimiter which is separating the fields

The purpose of join is to merge lines from multiple files based on a matching filed. For example both of your files could have numbered lines and will be merged based on that.

You can split by size (bytes -b) or by lines -l

$ split -b 1000 name_of_file.txt
$ split -l 20 name_of_file.txt

Sort

The sort command sorts lines in a file by alphabelitcal order

$ sort file_name.txt

You can also sort in reverse alphabetical order

$ sort -r filename.txt

We can also numerically sort by using the -n flag. If you've got a file with words in it, it will be sorted based on the ASCII value of the first character in each word.

Here's an example fruits.txt

Mango
Kiwi
Date
Pear
Banana

The ASCII value for the first character of each fruit is 77, 75, 68, 80, 66 respectively.

Now when we use it to numerically sort

$ sort -n fruits.txt

The output

Banana
Date
Kiwi
Mango
Pear

This command is also used to sort numbers inside a file. Example:

25
9
peach
5

The output

5
9
25
peach

Translate

The tr command is used to translate a set of characters into another.
For example let's convert from lower to uppercase.

$ tr a-z A-Z

hello

HELLO

Unique

The uniq command allows you to remove duplicates within a file

$ uniq file.txt

Using the -c flag you can get how many occurances each word has in the file

$ uniq -c file.txt

You can also find the unique value inside a file.

$ uniq -u file.txt

You can also know what words are the duplicates inside a file.

$ uniq -d text.txt

Word count

You can know the total word count in a file by using the wc coommand

$ wc file.txt

This displays the lines, number of words and number of bytes respectively. You can select which one you want to see by using these flags

-l: number of lines
-w: number of words
-c: number of bytes

nl is another command you can use to check the number of lines in a file

$ nl file.txt

grep

It allows you to search files for characters that match a certain pattern.

You want to know if a word exists in a file?

$ grep hello testing.txt

This will find the word hello in the file testing.txt

you can use regular expression in your pattern so you can add the -i flag directly before the pattern you're looking for if it is case sensitive.

$ grep -i [a-zA-Z0-9]{6,10} file.txt

In this example we're trying to find a string that has a length between 6 and 10 characters and is alphanumeric and is case sensitive.

17 commands to master the Linux Command Line

Fullstack Dev — Thu, 02 May 2024 14:04:31 +0000

The Shell

The shell is a program (Terminal) that takes commands from the user.
The default shell program on almost all the Linux distributions is the Bourne Again shell also known as Bash.

You can find other shells available such as

Korn Shell ksh
Z shell zsh
TENEX C shell tcsh

For most distributions, the shell prompt should have this format:

username@hostname:current_directory $

The $ is there by default and doesn't need to be added.

$ echo Hello from bash
$ date 
$ whoami

The echo command prints the string Hello from bash to the display.

The date will print the date and the whoami will print your username

Print working directory

Linux like every other Unix-based system has its files organized in a hierarchical structure and acts as a blueprint.

/

|-- bin

|   |-- file1

|   |-- file2

|-- etc

|   |-- file3

|   `-- directory1

|       |-- file4

|       `-- file5

|-- home

|-- var

The root directory / contains a number of folders and file and allows you to store more folders and files.

Each of the location of these folders and files are called paths.

For example a folder named home that contains a folder named projects its path would look like this /home/projects

You can use the pwd command to which directory you are in.

change directory

cd means change directory. It is used to navigate from one directory to another in the filesystem.

There are two types of paths:

Absolute path, which means, this path starts from the root /.

/home/pictures/vacations

Relative path, which means, this path starts from the directory you are in. For example let's say you are in /home/pictures/vacations and you want to get inside vacations to a folder named Italy

username@hostname:/home/pictures/vacations $ cd Italy

In other words, we're navigating from our current directory to the "Italy" folder.

. (current directory). This is the directory you are currently in.
.. (parent directory). Takes you to the directory above your current.
~ (home directory). This directory defaults to your “home directory”. Such as /home/pictures.
- (previous directory). This will take you to the previous directory you were just at.

list directories

You want to know what folders and files are available in a certain directory, you will need this command ls.

You can also specify the path you want to list the directories of

$ ls /home/pictures/vacations

Note that files starting with . will not be visible to us using the ls command but we can pass the -a flag to see them

$ ls -a

Another helpful ls flag is -l for long format, which displays the list of files in a long format. beginning on the left: file size, last modification time, owner name, owner group, number of links, file permissions, and file/directory name.

you can combine both flags to have the same combined results ls -la

Here are some useful flags:

$ ls -r
$ ls -t

ls -r Lists the current directory's content in reverse order.
ls -t Lists and sorts the current directory's content by modification time starting by the newest.

touch command

The touch command is used to create empty files.

$ touch my_new_file

The touch command will also modify the modification and access time to the current system time.

-a: Updates only the access time.
-c: Updates only the modification time
-t: Allows setting a specific date and time for the timestamps.

You can use file name_of_file to get a description of the file's content.

The cat command allows you to display the content of a file $ cat file_name
You can also combine multiple files and show you the output $ cat file_name1 file_name2

Less

The less command opens the file in the terminal
you can press q to quit.

g moves to the beginning of the file
G moves to the end of the file
-N displays line numbers
/search is used to search for a specific text inside the file

Viewing all the previous commands

$ history

This command is useful to see all the commands you previously used. You can also use this shortcut !! instead.

$ !!

You can hit the up arrow to go back to the previous command instead of typing it again

if you are not sure about a certain command or remember a part of a command you can type CTRL + R and start typing a part of the command you want and a match will become visible to you.

You can also clear the terminal by writing clear

$ clear

or you can type CTRL + l

Copy, move, make and remove files and directories.

By giving the name of the file and the path to the location you wish to copy it to, you can copy a single file.

$ cd name_of_file /home/myFiles/newDocs

You can copy several files and folders and use wildcards. A wildcard is a character that can increase the flexibility of your search by substituting it for a pattern-based choice. Wildcards are additionally versatile as they can be used in any command.

* it's used to represent all single characters or any string.
? used to represent one character
[] used to represent any character within the brackets

$ cp *.txt /home/Documents/videos

this will copy all the files with the .txt extension in the current directory to the videos directory.

Using the -r flag when running a command can be helpful since it copies files and directories within a directory recursively.

$ cp -r vacation/ /home/Documents/videos

If you copy a file to a directory that has the same filename, the file will be overwritten with whatever you're copying over. You can use the -i flag to avoid that.
The -i flag will prompt you to confirm if you wanna overwrite a file before you overwrite it.

cp -i myfile /home/Documents/

moving files and directories

The mv command allows you to move a file to a different directory by taking the file's name and the location to where you want it moved.
This also works for directories.

$ mv file1 /home/Documents
$ mv myfolder /home/documents

You can also move multiple files by providing the names of the files and the path to where you want them moved.

$ file1 file2 /home/Documents

You can also use the -i flag to prompt a confirmation before overwriting anything.

we can also create a backup of a directory using the -b flag

$ mv -b myfolder myfolder_backup

You can also use the mv command to rename files like so

$ mv myfile new_file

making a new directory and removing a directory

You can use the mkdir command to create a directory.

$ mkdir folder folder1

you can also use the -p flag to create subdirectories at the same time

$ mkdir -p folder/folder1/folder2

You can remove files using the rm command

$ rm myfile

Using rm doesn't mean the removed files will be in the trash. Once this command is used the files will be completely gone.

You can use the -f flag to force remove the files even if they're write-protected ( you can't rename, move, or delete its parent directory.)

You can also use the -i flag to prompt for confirmation.

You can't use the rm on its own to remove a directory, you need to use the -r flag. Using this flag will allow the removal of all the files and any subdirectories it has.

$ rm -r myfolder

You can remove a directory by using rmdir command

$ rmdir folder1

Find a specific file

You can use the find command if you're trying to find a specific file. You need to specify the directory you'll be searching and what you're searching for. It also looks inside subdirectories.

$ find /home -name coffee.jpg

You can also specify the type of file you are trying to find.

$ find /home/Documents -type d -name videos

Manual

If you need some information about a specific command you can use the man or whatis command.

$ man ls
$ whatis cp

Alias

You can give an alias to repetitive commands instead of typing them especially if they're long ones.
To create an alias:

alias la= 'las -la`

Now you can type la instead of ls -la command and you will have the same output.

If you want to remove the alias you can use

$ unalias la

Check flags

If you wanna know what flags are available for a certain command you can use help

$ ls --help

Feel free to leave a question in the comment section!