DEV Community: Mcvean Soans

AutoDL celebrates Hacktoberfest 2021

Mcvean Soans — Thu, 30 Sep 2021 05:02:14 +0000

At last it's "that time" of the year again! The month of October is right around the corner, and surely you might've already guessed it... 👀

The much awaited Hacktoberfest is back for the year 2021!

What's more, AutoDL is geared up for participants to contribute as well!

// Detect dark theme var iframe = document.getElementById('tweet-1441415078494945281-229'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1441415078494945281&theme=dark" }

Now if you're unaware, Hacktoberfest is DigitalOcean's month-long celebration of Open-Source software and communities.

Code Newbies and Open-Source Connoisseurs alike are all welcome to take up Issues on AutoDL's GitHub repositories and submit a valid Pull Request (abiding by the quality standards) pertaining to close open issues on GitHub.

P.s. Look for issues with the hacktoberfest label on it. 😉

Make 4 valid PRs throughout the month to stand a chance to be rewarded with a commemorative t-shirt pack as official swag from DigitalOcean! Additionally, AutoDL would be distributing swags for top contributors in its repositories too! 🤩

Do make sure to read the Contributing guidelines and the Code of Conduct to gain an insight on how to contribute!

If you feel stuck, need any help, or you need to tackle a different issue not addressed on GitHub entirely, feel free to discuss on AutoDL's Slack Workspace. ✨

P.s. There's a separate channel for hacktoberfest as well 😊

Now, as tantalizing (or lucrative) the perks may sound, remember that as mentioned on the official website of Hacktoberfest -

Quantity is fun, quality is key

Quality pull requests showcase you as a responsible member of the Open-Source community! Do utilize this exciting opportunity to contribute, be it your first pull request or your bazillianth one! ✌️

Mastering Datatypes in Python

Mcvean Soans — Thu, 26 Nov 2020 10:42:05 +0000

This topic might be a bore for non-enthusiasts and beginners alike 😕, but really is quite easy and simple to pick up 🤭. Okay, before you criticize me for my demeanor, let me make it clear that the aim of this post is to strengthen the fundamentals of beginners, and for the experienced folks to brush up on their amazing skills!

A datatype, simply put, represents the type of data stored into a variable, in memory. These may be classified into 2 main categories: Built-in and User-defined. For now, let us focus on built-in datatypes only.

Built-in Datatypes

Python provides a few built-in data types, which come preloaded when it is installed:

None
Numeric
bool
Sequences
Sets
Mappings (Dictionaries)

Let us look at each of these in detail! 😄

None Type

In Python, the 'None' datatype represents an object which does not contain any value (similar to 'null' in Java). Python being a dynamically-typed language, 'None' is used less frequently.

Numeric Type

These represent numbers, and are further subdivided into:

int - These represent integers (a number without any decimal point). For example, a = -30 or b = 289.
float - These represent floating-point numbers (a number containing a decimal point). For example, n = 25.69 or x = 30.26e3, where 'e' or 'E' represents exponentiation to the power of 10.
complex - These represent complex numbers written in the form a+bj or a+bJ. In this form, 'a' is the real part of the number and 'b' is the coefficient, while 'j' or 'J' is the square root of -1. For example,
c1 = 1 - 0.5j or c2 = 2 + 5J.

🤔 You may find this interesting too :
Python provides facilities to represent Binary, Octal and Hexadecimal numbers too. A binary number should be written with a prefix '0b' (zero and b) or '0B' (zero and B), octal numbers with a prefix of '0o' (zero and o) or '0O' (zero and O), and hexadecimal numbers with a prefix of '0x' (zero and x) or '0X' (zero and X).

bool Type

This is a special datatype which represents either "True" or "False". It is used mainly in conditional statements to evaluate conditions and provide a logical answer (True or False). Conditional statements will be covered in an upcoming post, but for now we only need to know that they help us decide the flow of control of our program depending on any given condition.

Sequences in Python

These may be looked at as a group of elements or items. In Python, we have six types of sequences:

str - These represent strings (a group of characters) in Python. Strings are enclosed within double or single quotes, and even with triple double quotes or triple single quotes (called docstrings). For example,
word = "Hello" or name = "Mack".
bytes - These represent a group of byte numbers just like an array. A byte number is any positive number from 0 to 255 (inclusive). For example, x = bytes([0, 10, 20, 30, 40, 50]) creates a bytes array having 6 elements.
bytearray - This is exactly similar to the bytes datatype, the only difference being that this datatype is mutable (the elements of an array, once created, cannot be altered). We will look at mutability later on so do continue reading! 😁
list - Lists in Python are similar to arrays in C or Java, the main difference being a list is capable of storing multiple types of objects, whereas an array can only store one type of elements. Hence, we can see that lists are a very important datatype. For example,
names = ['John', 'Roy', 'Michelle', 'Sharon'] or
misc = [1, 'Michael', 23.60, -30].
tuple - A tuple is exactly similar to a list, the only main difference being that a tuple is immutable whereas a list is mutable. Hence, we can say that a tuple is a read-only list. For example,
names = ('John', 'Roy', 'Michelle', 'Sharon') or
misc = (1, 'Michael', 23.60, -30).
range - These represent a sequence of numbers. Similar to a tuple, the elements are not modifiable i.e., this datatype is immutable. These are generally used in loops and iterative structures. For example,
r = range(10), the variable 'r' stores numbers from 0 upto 9.

🤔 A key takeaway :
From the above examples, we can see that a list encloses elements within opening and closing square / box brackets ('[' and ']'), while a tuple encloses elements within opening and closing parenthesis ('(' and ')').

Sets in Python

A set is an unordered collection of elements. What does this mean? Well, whenever we create a sequence in Python, the order of the elements stays put (fixed). This is not true for sets. Whenever we create a set, the elements may not appear in the same order as they are entered. Moreover, a set does not accept duplicate elements. Sets are again sub-divided into two:

set - This is similar to an array and uses opening and closing curly brackets ('{' and '}') to store the elements. We can create a set in two ways, for example, s1 = {1, 2, 3, 4, 5} or s2 = set("Hello"). The second method utilizes the 'set( )' function to convert the letters of the string into a set.
frozenset - It is similar to the set datatype, the only difference being, frozenset datatype as the name suggests is immutable, while the set datatype is mutable. We also have two ways to create a frozenset, we can first create a set and then pass it thorugh the frozenset( ) function fs1 = frozenset(s1) or directly pass another object (such as a string) to create a frozenset, fs2 = frozenset("Hello").

Mappings (Dictionaries) in Python

A map represents a group of elements in the form of key-value pairs. Understanding dictionaries in Python is pretty simple.

Imagine you are given a particular word (say "Photosynthesis") with a dictionary in hand, and you are said to locate the meaning of the word using the dictionary. Logically, we would start searching for the meaning, by first looking up the letters (starting from the first all the way upto the last one) until we find the word (starting from "P" in our case, followed by "h", "o", and all the way upto "s"). This way, we are able to quickly locate the meaning without having to look at the meaning of each and every word in the dictionary. In this case the word is considered as the "key" and the meaning is considered as the "value"

Similarly, the 'dict' datatype in Python is utilized to store key-value pairs. The keys and values are separated using a colon (:) and every key-value pair is separated using a comma (,). Finally, all the elements are to be enclosed within curly brackets ({ }).

Also, dictionaries are mutable and hence the elements stored in the dictionaries can be altered. For example,
roll = {1: "Aaron", 2: "Adele", 3: "Mike", 4: "Ryan"} or d = {} is an empty dictionary.

Mutability

It is a property of an object by which it's elements can be altered after creation. Let us take a look at the following code:

lst = [1,2,3,4,5] # A list of 5 elements
print(lst) # [1,2,3,4,5] is printed
lst[0] = 10 # Changing value at index 0
print(lst) # [10,2,3,4,5] is printed

From the above code, we can make out that the value of the first element in the list (at index 0) was changed from '1' to '10'. We were able to do this because as we have seen earlier, lists are mutable. Finally, let us look at another example:

tup = (1,2,3,4,5) # A tuple of 5 elements
tup[0] = 10 # Throws an error on the screen

As we can see, the second statement throws an error. Why is that? It is because tuples (similar to frozenset and bytes) are immutable. Any attempt to alter the elements, once the object has been created, is simply not allowed.

How Python Sees Variables

Mcvean Soans — Thu, 26 Nov 2020 04:40:04 +0000

In programming languages like C and Java, the concept of a variable is related to a memory location. Hence, often a variable is defined as a named memory location. Let us visualize this with an example, say we have the equation

int a = 1;

This may be viewed as a memory box:

Say we now store another value using the same variable

a = 2;

This may be viewed as an updated memory box:

Let us now store the value of this variable into another

int b = a;

This creates another memory box as follows:

This is basically how other programming languages visualize variables. We can simply say that the "memory location" is emphasized which contains a "value". However, Python views variables as "tags" which are referenced (or tied) to some "value" (or object).

Let us visualize the above examples once again, say we have the equation

a = 1

This may be viewed as the value or object having priority, to which a tag is assigned:

Say we now store another value using the same variable

a = 2

In this case, the tag is simply changed to the new value, and the previous value (now unreferenced) is removed by the garbage collector:

Let us now store the value of this variable into another

b = a

In this case, another tag is created that references to the same object:

Hence, only one memory is referenced by two names.

So to conclude, other languages have variables, while Python has 'tags' to represent the values (or objects). This method allows Python to utilize memory efficiently, and this visualization helps in understanding variables in Python.

Comments and Docstrings in Python

Mcvean Soans — Wed, 25 Nov 2020 16:05:06 +0000

Comments in Python

As we have seen in a previous post Executing a Python Program, a comment is a line (or multiple lines) of code which do not execute when the program is run. The sole purpose of comments is to provide more information about the written code.

Many a times while writing programs, we tend to ignore documenting the steps. This causes an issue not only to others that might read our code in the future, but our future selves too. For example, let us take a look at the following snippets which contain the exact same code -

Snippet 1:

lst = [1,2,3,4,5]
for i in lst:
    print(i)

Snippet 2:

lst = [1,2,3,4,5] # Creating a list
# Looping over each element in the list
for i in lst:
    print(i) # Printing each element

As we can see from the above snippets, the difference in the quality of code varies significantly. Any person can understand the functionality of the code from the second snippet easily as compared to the first snippet. Hence, we can conclude that proper documentation of the code using comments is essential while developing any application.

Python has 2 types of comments:

Single line comments -

These comments begin with the hash (#) symbol and the entire line after the hash symbol is treated as a comment.

Multi line comments -

When we want to comment multiple lines of code, we can do it as follows

Method 1:

# This is the first comment
# This is the second comment
# This is the third comment

Method 2:

"""
This is the first comment
This is the second comment
This is the third comment
"""

Method 3:

'''
This is the first comment
This is the second comment
This is the third comment
'''

Docstrings

As we have previously seen, multi-line comments are regular strings which have the ability to span multiple lines. If these strings are not assigned to a variable, then they are removed from the memory by the garbage collector. Hence, using """ (triple double quotes) and ''' (triple single quotes) are not recommended for comments by developers since they occupy memory and delays the complete execution of the program.

The strings enclose within """ or ''' are called documentation strings or docstrings. These are useful to create an API (Application Programming Interface) documentation file, that provides a description of various features of a software, language or a product.

API documentation is critical if developers intend to distribute their work. Bad or improper documentation may lead to other developers not understanding the complexity in code, and they may completely reject the code simply because it did not make sense!

Python: Memory Management

Mcvean Soans — Wed, 25 Nov 2020 06:38:23 +0000

Memory Management in Python

In languages such as C or C++, programmers find it hectic to allocate and deallocate (free) memory while executing the programs. Python takes care of memory allocation and deallocation during runtime automatically.

Everything is considered as an object in Python, be it strings, lists, and even Functions / modules are objects. For every object, memory has to be allocated. This task is done by the Memory Manager located inside the PVM.

All of the objects created are stored in a separate memory called heap. It is the memory which is allocated during runtime. The size of the heap memory is variable for different systems and depends on the Random Access Memory (RAM) of the system. It can even increase or decrease as per the requirement of the programs.

For our systems, the Operating System (OS) allocates memory for any program that runs on it. Python's Raw Memory Allocator runs on top of the OS which oversees the memory allocated for all of the objects. We also have Object-specific memory allocators which operate on the same heap. These memory allocators utilize various memory management policies (depending on the type of the objects) to allocate memory for the different objects.

Sequentially, we can view memory management in Python as follows:

Garbage Collection in Python

Garbage collector (gc) is a Python module which deletes objects from the memory, which are not used in the program. The simplest way in which it handles deallocation (freeing memory), is by keeping a reference count of the objects in the program. Whenever, an object is utilized in the program, it will be referenced atleast once i.e., it will have a reference count of atleast 1. When an object is found with reference count of 0, it figures out that the object is not being used in the program and deletes it from the memory.

However, this approach does have a catch. Suppose we have a reference cycle of 3 objects, which themselves reference to one another as follows:

Even if the objects A, B and C are not used in the program, they have a reference count of 1. To overcome this situation, gc uses an algorithm to remove objects from this cycle.

The algorithm classifies the objects into three generations. The newly created objects are generation 0 objects. Now, first time when garbage collector sweeps through the memory and does not remove the objects from the memory, the objects are promoted to generation 1. The next sweep ensures that the unused objects are deleted from memory. If the object survives again, it is promoted to generation 2. Hence, Garbage collector tends to delete younger objects (generations 0 and 1) rather than older objects (generation 2).

Garbage collector runs automatically, and Python schedules garbage collector depending on a metric (number) called threshold. This number represents the frequency of how many times garbage collector removed the objects from the memory. We can know the threshold number by using the get_threshold() method of the gc module. When the equation (numberOfAllocations - numberOfDeallocations) > threshold is satisfied, garbage collector runs automatically. However, if more and more objects are created and if the system runs out of memory, garbage collector does not run automatically to deallocate the memory, instead, an exception (runtime error) is thrown.

When the programmer is sure that the program does not contain reference cycles, then automatic garbage collector is the most suitable choice. Whereas, the presence of reference cycles in the program requires the gc module to be called manually. The collect() method is useful in this case.

Manual garbage collection is done using 2 ways:

Time-based - Garbage collector is called in certain intervals of time.
Event-based - Garbage collector is called on the basis of an event (eg: When the network connectivity of the application is lost, etc.)

A thing to keep in mind is that running the garbage collector too frequently will slow down the program execution.

The bare-bones of a Python program

Mcvean Soans — Wed, 25 Nov 2020 06:31:04 +0000

Let's create our first Python program. Create a file and name it hello.py and in any editor let's write a line of code:

print('Hello World') # Our first line of code

The above program when run, prints out "Hello World!" onto the screen, and the stuff after the # is considered as a comment. Comments and the various types will be covered in a later post, but for now let's assume that it is a line which will have no effect upon execution of the program.

Every program written in Python should have an extension .py in order to be called a Python program.

The next step after writing the code, is compiling the program into the byte code. This is done by the Python compiler. The byte code instructions are contained in a file hello.pyc, where the .pyc extension suggests a python compiled file.

Any computer can execute only binary code (consisting of 1s and 0s), and hence it is essential to convert the byte code into machine code (binary code) so that the computer is able to execute it. This conversion is done by the PVM (Python Virtual Machine). PVM uses an interpreter which understands the byte code and converts it into machine code understandable to the operating system.

An interpreter translates the program source code line-by-line and hence is slower to execute the program. The PyPy flavor of Python utilizes a JIT (Just In Time) compiler in the PVM instead of an interpreter so as to run the programs faster.

Normally, while executing a Python program, the .pyc files produced are not visible and the necessary steps / working is done internally by the Python compiler and the PVM in order to display the output on the screen.

In order to execute the file, we use the python command to call the Python compiler as follows:

>> python hello.py

While taking a closer look at the directory, we can see that no file with a .pyc extension exists. All of the steps have been completed internally and only the final output is displayed.

To separately create a .pyc file from the source code, we can use the following command:

>> python -m py_compile hello.py

The -m option represents module and the py_compile module generates the .pyc file for the specified .py file. The compiler creates a separate directory __pycache__ where the .pyc file is stored. This file contains the machine code instructions and can be run on any platform using the PVM.

Some Flavors of Python (Python compilers)

CPython - The standard Python compiler implemented in the C language.
Jython - Implementation of Python designed to run on Java platform (executed by JVM).
IronPython - Implementation of Python for .NET framework, written in C# (C Sharp) language.
PyPy - Implementation of Python using Python, written in RPython (created in Python language).
RubyPython - Designed for applications that utilize the Ruby language.

Frozen Binaries (Executables)

When a developer creates a software in Python, the 2 possible ways to produce the software to the end user include:

Providing the .pyc files
Providing frozen binaries (.exe files) The first method expects the user to install PVM and run the .pyc files. While, the second method provides the .pyc files along with the PVM and the necessary Python libraries. These separate files are converted into a single executable file (generally with a .exe extension), so that the user can directly execute the file by double clicking it.

Frozen binaries have a slightly larger size as compared to the .pyc files, since they contain the PVM as well as the library files. For creating these frozen binaries, we require third-party softwares. For example, py2exe is a software the produces frozen binaries for the Windows operating system, and pyinstaller is a similar tool for UNIX or LINUX.

Python: Preamble

Mcvean Soans — Wed, 25 Nov 2020 06:29:12 +0000

This is the first post in the Becoming a Python Developer series! This series is intended towards everyone. Be it complete beginners or professional developers, the posts are sure to be helpful in grasping the concepts and a perfect launchpad for your Python journey! 😄

What is Python? Why is it gaining popularity?

In recent times, the need for a developer is a programming language that provides both C-style functional programming as well as Java-style object orientation. Whenever, functional aspects like calculations or processing is involved, C-style coding is preferred, while the need for classes and objects is completed by Java-style coding.

Python is an open-sourced programming language that combines the features of C and Java. It has exceptional procedural as well as object-oriented capabilities. Having said this, always remember that everything in Python is an object.

Some Features of Python

Simple and Easy to learn - Learning Python is very simple compared to other programming languages.
Dynamically Typed - In Python, we need not specifically declare the datatype of the variables used (unlike statically typed languages like C and Java).
Platform Independent - When a Python program is compiled using a Python compiler, it generates byte code. Using a Python Virtual Machine (PVM), anyone can run the byte code instructions on any computer system.
Procedure and Object Oriented - Python gives emphasis on the functions as well as objects.
Scripting Language - A scripting language is a programming language that utilizes an Interpreter to translate the source code into machine code on the fly (while running). Other scripting languages include PHP, Bash, etc.
Database Connectivity - Python provides Application Programming Interfaces (APIs) to connect its programs to all major databases like Oracle, MySql, SQLite Browser, etc.
Batteries Included - Python contains several packages available for immediate use by developers. These packages make development easier due to the various features supported by them. Some of these packages include: numpy - for working with single and multi-dimensional arrays, pandas - for data analysis and data manipulation, matplotlib - for data visualization, etc.

Some differences between C and Python

C	Python
It is a procedure-oriented programming language, and does not contain features such as objects, classes, etc.	It contains features from procedure-oriented programming languages as well as object oriented programming languages.
Program execution is faster.	Slower execution of programs as compared to C. The PyPy flavor of Python programs run a bit faster.
It is a statically-typed language (variable declaration with appropriate datatype is compulsory).	It is a dynamically-typed language. Type declaration is not required.
Memory allocation and deallocation requires user-intervention.	Memory allocation and deallocation is done automatically by the PVM.
Indentation does not have any importance in C.	Indentation plays a major role and the code blocks need to be correctly indented.

Some differences between Java and Python

Java	Python
Java is essentially an object-oriented programming language.	Python blends functional programming features with the object-oriented approach of programming.
Java programs are verbose (contain many lines of code).	Python programs are concise and compact, and very few lines of code are required to achieve any specified task.
It is compulsory to declare the datatypes of the variables, arrays, etc. (statically-typed language)	Type declaration is not required in Python (dynamically-typed language).
Memory allocation and deallocation is done by JVM (Java Virtual Machine).	Memory allocation and deallocation is done by PVM (Python Virtual Machine).
Indentation does not have any importance in Java. It is required only for better readability of the code.	Indentation plays a major role in Python, and improper indentation may lead to errors in the program.