A Beginner's Guide To Data Engineering Concepts, Tools, And Responsibilities.

Rebeccacheptoek — Sun, 04 Aug 2024 19:18:20 +0000

Data engineering is the process of collecting, organizing, and processing of data to make it accessible, usable, and meaningful
This is to ensure that it can be effectively utilized to derive insights and support business goals.

Python 101: Ultimate Python Tutorial For Begginer

Rebeccacheptoek — Sat, 23 Apr 2022 21:06:37 +0000

Introduction

Python is a high-level, popular programming language.
It was created by Guido van Rossum and first released on February 20, 1991.
It is used for web and software development, mathematics, data analysis, AI and machine learning
It works on different platforms and it has simple syntax.
Due to its simplicity, Python is very productive it can help focus on solving developer's problem
The most recent version of Python is python3. Python is written in an Integrated Development Environment (IDE), such as Pycharm, VS Code, Spyder, Jupyter, Sublime Text, Thonny. Python was designed for readability and it is therefore simple to use
In this course we are going to learn everything you need to get started with programming in python

Installation

Python is available on a variety of platforms, that is Windows, Linux ans MacOS
Open a terminal window and type "python" to check if it is installed already. This is usually the case in most PC's and if it's not you will have to download python on a web browser
Go to https://www.python.org/downloads/
Follow the link to download either for Linux/Unix, Windows or MacOSthe latest version of python
Make sure to add python to path if you are using windows. Next we need to install a code editor where we write our codes. They are many including Pycharm, Jupyter, VS Code.
Python files are created using .py file extension ##Python Syntax
Python uses indentation to indicate a block of code
Indentation : the spaces at the beginning of a code line ###Example

if 10 > 5:
    print("10 is greater")

If you skip the indentation you'll get an error

Python Variables

Variables are temporary storage of data in a computer's memory
number = 10

number is a variable and it is a memory location where the value 10 is stored Variables don't need to be declared with a type Variables are case-sensitive:

X = 'City'
#x and X are different variable names
#X will not overwrite x

Literal - this is a value/data given to a variable

>Example
>> name = 'David'
# David is the literal, the value given to the variable name

Identifier: It is a name used to identify a variable, function, objects or constants
name = 'Becky' *name* is an identifier
Keyword : This is a reserved word known to the compiler e.g. for, while, in, def

Rules in naming Variables

A variable name must start with a letter or an underscore
A variable name cannot start with a number
Variable names are case-sensitive i.e. case, Case and CASE are different variables
Keywords should not be used as variables
Do not use special characters such as ., #, @, -

Python Comments

These are texts that are ignored during compilation
Comments are used to explain the code and it makes it more readable
Comments in python starts with a #.
To comment multiple lines with #we use Ctrl + /
Multiple line comments are done by adding triple quotes

"""
this is
a multiline
comment
"""

"""""" """ this is a multiline comment """

Python Data Types

Variables store data of different data types. Python data types are: Numeric types : int, float, complex

x = 10     #int
x = 10.2   #float
x = 3 + 2j #complex

Sequence types : list, tuple,range

x = ["bus", "car", "cart"] #list
x = ("bus", "car", "cart") #tuple
x = range(3)

Mapping type : dict

x = {"vehicle" : "car",  "name" : "Toyota"}

Set types : set, frozenset

x = {"bus", "car", "cart"}

x = frozenset({"bus", "car", "cart"})

Boolean :bool

x = True
y = False

Text type : str

x = 'Hello World'

Type Conversion

To get the data type of variable we use the type () function ###Example

name = 'Becky'
print(type(name))
print(name)

Output

 <class 'str'>
 Becky

Getting Input

We use input () function to receive input from the user ###Example

name = input('What is your name? ') 
print('My name is ' + name)

Output

What is your name? *Becky*
My name is *Becky*

Formatted Strings

Are useful in situations where you want to dynamically generate some texts with your variables Prefix your string with an 'f' for a formatted string

Example

first = 'Becky'
last = 'Toek'
message = first + '[' + last + '] loves coding' #string concatenation
msg = f'{first} [{last}] loves coding' #formatted string
print(msg)

Output

Becky [Toek] loves coding

Python Operators

Operators are used to perform operations on variables and values Python has different operators: ## Arithmetic Operators
Are use with numeric values to perform mathematical operations
They are:

Name	Operator	Example
Addition	+	x + y
Subtraction	-	x - y
Multiplication	*	x * y
Division	/	x + y
Modulus	%	x % y
Floor division	//	x // y
Exponential	**	x ** y

Assignment Operators

Assigns values to variables

Operator	Example
=	x = y
	+=
-=	x -= y
*=	x *= y
/=	x /= y
%=	x %= y
//=	x //= y
**=	x **= y

Logical Operators

They are used to combine conditional operators

Operator	Name	Description	Examples
and	Logical AND	returns True if both statements are true	x > 0 and y > 0
or	Logical OR	returns True if one of the statements is true	x < 10 or y < 2
not	Logical NOT	returns True if the statement is false	not x

Bitwise Operators

Used to compare binary values

Operator	Name	Description

x & y	Bitwise AND	Bits defined in both x and y
x l y	Bitwise OR	Bits defined in x or y or both
x ^ y	Bitwise XOR	Bits defined in x or y but not both
x << y	Bit shift left	Shift bits of x left by y units
x >> y	Bit shift right	Shift bits of x right by y units
~ x	Bitwise NOT	Inverts all the bits of x

Comparison Operators

This operator is used to compare two values
Colons can be used to align columns.

Operators	Name	Example
==	Equal to	x == y
!=	Not equal to	x != y
>	Greater than	x > y
<	Less than	x < y
=>	Greater than or equal to	x >= y
<=	Less than or equal to	x <= y

Identity Operators

Checks for object identity. This is different from equality such that it compares the objects if they are the same objects, with the same memory location

Operator	Description	Example
is	It returns True if both variables are the same object	x is y
is not	Is true if both variables are not the same object	x is not y

Membership Operators

Used to test if a sequence is presented in an object

Operator	Description	Example
in	Returns True if a sequence of the specified value is present in the object	x in y
not in	Returns True if a sequence of the specified value is not present in the object	x not in y

Python Lists

Lists are used to store multiple items in a variable
They are created using square brackets

Example

vehicles = ["bus", "car", "truck"]
print(vehicles)
Lists items have a defined order and it will not change
Lists allows duplicate values
i.e.

mylist = ["apple","mango", "orange", "mango", "kiwi" ]
print(mylist)
The items are indexed and the first item has index [0]
The data type for lists is 'list'.
Items in a list could contain any data type (int, bool,str,)

Python Tuples
Tuples store multiple items in a single variable

They are written with round brackets()

Example

mytuple = ("apple","mango","orange","kiwi")
print(mytuple)

Output

('apple', 'mango', 'orange', 'kiwi')

Tuple items like lists are ordered, unchangeable and allow duplicate
The items are indexed and the first item has index [0]
The data type of tuple is 'tuple'
Items in a tuple could contain any data type

Python Sets

Sets are used to store multiple items in a single variable
They are written with curly brackets {}
Unlike tuples and lists sets are unordered, unchangeable and do not allow duplicates
Sets are unindexed
Sets can be of any data type and they are of the data type sets >Example > > >> myset = {"apple","mango","orange","kiwi"} >> print(myset) > >

Output

{'apple', 'mango', 'orange', 'kiwi'}

Python Dictionaries

Dictionaries are used store values in key: value pairs
It is ordered, changeable, and do not allow duplicates
They are written with curly brackets and have keys and values
Values in a dictionary can be of any data type
Dictionaries are defined as objects with data type dict

Example
mydict = {
"food" : "beans",
"type" : "protein",
"amountKg": 2
}
print(mydict)
Output

{'food': 'beans', 'type': 'protein', 'amountKg': 2}

DEV Community: Rebeccacheptoek

A Beginner's Guide To Data Engineering Concepts, Tools, And Responsibilities.

Python 101: Ultimate Python Tutorial For Begginer

Introduction

Installation

Python Variables

Rules in naming Variables

Python Comments

Python Data Types

Type Conversion

Output

Getting Input

Formatted Strings

Example

Output

Python Operators

Assignment Operators

Logical Operators

Bitwise Operators

Comparison Operators

Identity Operators

Membership Operators

Python Lists

Python Tuples

Python Sets

Python Dictionaries