DEV Community: Aman Gupta

Python: List vs Dictionaries

Aman Gupta — Mon, 18 Jan 2021 03:43:14 +0000

Python has multiple types of data structure and from those the List and Dictionary are one of them. so we are discussing here about which is efficient in which condition.
first see some introduction about both.

List:

In python list is a most versatile data structure which stores ordered sequence of objects of different types .
In python list are mutable, which means that these elements can be altered unlike str or tuple. These elements of list called items.
For creating list in python, there are many ways but the simplest way to do this is by just using [ ] and put items separated by ,.
Different ways to create a list:

Using a pair of square brackets to denote the empty list.
Using square brackets, separating items with commas.
Using a list comprehension.
Using the type constructor.

a = [] # provide empty list
b = [1,2,3] 
c = [x for x in iterable]
d = list('abc')
e = list((1,2,3))

Dictionary:

In python dictionary is a mapping object maps hashable values to arbitrary objects.It is python's Mapping type and there are only one standard mapping types which is dictionary. It is also mutable . Python dictionary has a key, value pair where keys are almost arbitrary and the values that are not Hashable like values that contain list, dictionary or other mutable types are not used as the keys in dictionary.

Numeric types used for keys obey the normal rules for numeric comparison: if two numbers compare equal (such as 1 and 1.0) then they can be used interchangeably to index the same dictionary entry.

As the lists, Dictionaries can be created by various ways, but the simplest way to create it is by passing : separated pair of keys and values inside a { }.
Different ways to create a dictionary:

Use a comma-separated list of key: value pairs within braces.
Use a dict comprehension.
Use the type constructor.

a = dict(one=1, two=2, three=3)
b = {'one': 1, 'two': 2, 'three': 3}
c = dict(zip(['one', 'two', 'three'], [1, 2, 3]))
d = dict([('two', 2), ('one', 1), ('three', 3)])
e = dict({'three': 3, 'one': 1, 'two': 2})
f = dict({'one': 1, 'three': 3}, two=2)

All the above examples return a dictionary equal to {"one": 1, "two": 2, "three": 3}.

Now , we come to our topic. We generally use list in our programs because the lists are easy to use but what if we have a huge number of items for example 5000000. Then the lists take much time to lookup any items but if we do same task with dictionary it is done pretty fast, because in Python, the average time complexity of a dictionary key lookup is O(1), since they are implemented as hash tables. The time complexity of lookup in a list is O(n) on average. But there are one more problem with use of dictionary that it takes a lots of space as store keys and values pair as compared to the lists. That is the best situation of space-time tradeoff.

If you search for a fixed number of keys, if you grow your haystack size (i.e. the size of the collection you’re searching through) by 10,000x from 1k entries to 10M entries, using a dict is over 5000x faster than using a list! (Source: Fluent Python by Luciano Ramalho)

But after the python 3.6 we have a new implementation of dict which takes less memory then previous so now it is not more a space-time tradeoff.

The dict type has been reimplemented to use a more compact representation based on a proposal by Raymond Hettinger and similar to the PyPy dict implementation. This resulted in dictionaries using 20% to 25% less memory when compared to Python 3.5. (Source : python.org)

Thanks For reading.

Numpy : Advance Indexing

Aman Gupta — Sun, 10 Jan 2021 13:15:37 +0000

In this blog, we are talking about different ways of indexing array in NumPy which are amazing.

Basic Slicing and Indexing

So here from basic slicing and indexing we are talking about pythons basic concept of slicing and indexing which is similar as python built-in data type tuple, list and str.

for accessing a single element from numpy array we can do it by passing passing each dimension’s index saperated by , in [ ], because numpy support multi-dimension indexing.
for example: a[1,2] = a[1][2].
we can pass value of index either positive start with 0 (which is the first element) or negative where -1 mean the last element of the array.

For accessing more then one element , we can use python's standers slicing syntex per dimension basis all sapetared with , in square bracket.

Basic slice syntex in i:j:k where i is starting index, j is stoping index and k is the steps.

>>> import numpy as np
>>> ## Accessing a single element:
>>> # 1. from 1-D array:
>>> arr = np.arange(10)
>>> arr[0]
 0
>>> arr[-2]
 8
>>> #2. from 2-D array:
>>> nd_arr = np.array([[1,2,3],
...               [4,5,6],
...               [7,8,9]])
>>> nd_arr[1,2]
 6
>>> nd_arr[-1,2]
 9
>>> ## accessing more then one element:
>>> # for 1-D array:
>>> arr[1:5:2]
array([1, 3])
>>> # for 2-D array:
>>> nd_arr[0:2:1,:]
array([[1, 2, 3],
       [4, 5, 6]])
>>> #this can also be achieved by '...':
>>> nd_arr[0:2:1,...]
array([[1, 2, 3],
       [4, 5, 6]])

Ellipsis ('...') expands to the number of : objects needed for the selection tuple to index all dimensions. In most cases, this means that length of the expanded selection tuple is arr.ndim. There may only be a single ellipsis present.

Ellipsis is a python built-in Constant type which used mostly in conjunction with extended slicing syntax for user-defined container data types.

Advance Indexing in Numpy array

advance indexing mean when the selection object is non-tuple object , an ndarray of int or bool , or tuple with al least one sequence object with int or bool data types.
There are two types of advance indexing, one is integer and another is boolean.

In numpy, Advanced Indexing always returns a copy of data while basic slicing returns a view.

One most important thing is that The definition of advanced indexing means that x[(1,2,3),] is fundamentally different than x[(1,2,3)]. The latter is equivalent to x[1,2,3] which will trigger basic selection while the former will trigger advanced indexing. Be sure to understand why this occurs.

Also recognize that x[[1,2,3]] will trigger advanced indexing

Integer array Indexing

Integer array indexing allows selection of arbitrary items in the array based on their N-dimensional index. Each integer array represents a number of indexes into that dimension.

Purely integer array indexing

When the index consists of as many integer arrays as the array being indexed has dimensions, the indexing is straight forward, but different from slicing.

>>> #Example:
>>> x = np.array([[1, 2], [3, 4], [5, 6]])
>>> x[[0, 1, 2], [0, 1, 0]]
array([1, 4, 5])
>>> #Example
>>> x = np.array([[ 0,  1,  2],
...           [ 3,  4,  5],
...           [ 6,  7,  8],
...           [ 9, 10, 11]])
>>> rows = np.array([[0, 0],[3, 3]], dtype=np.intp)
>>> columns = np.array([[0, 2],[0, 2]], dtype=np.intp)
>>> x[rows, columns]
array([[ 0,  2],
       [ 9, 11]])

Advance Indexes always broadcasted and iterated as one. The result we get is similar dimention of broadcasted result.

in above example we broadcast in rows and column but in nympy there are one function ix_ which help this broadcasting.

>>> #Example
>>> x = np.array([[ 0,  1,  2],
...           [ 3,  4,  5],
...           [ 6,  7,  8],
...           [ 9, 10, 11]])

>>> rows = np.array([0, 3], dtype=np.intp)
>>> columns = np.array([0, 2], dtype=np.intp)

>>> x[np.ix_(rows, columns)]
array([[ 0,  2],
       [ 9, 11]])
>>> # if we don't use `np.ix_` then result:
>>> x[rows, columns]
array([ 0, 11])

Combining advanced and basic indexing

When there is at least one slice (:), ellipsis (...) or newaxis in the index (or the array has more dimensions than there are advanced indexes), then the behaviour can be more complicated. It is like concatenating the indexing result for each advanced index element

>>> x[1:2,1:3]
array([[4, 5]])
>>> x[1:2,[1,2]]
array([[4, 5]])

Boolean array Indexing

This types of indexing occurs when the selection object is the array of Boolean, such as returned from comparison operator.

if the dimension of selection onject is same as the array then it returns a 1-D array filled with all result corresponding to the True value . The search order will be row-major , C-style .

If obj has True values at entries that are outside of the bounds of x, then an index error will be raised. If obj is smaller than x it is identical to filling it with False.

>>> #Example:
>>> x = np.array([[1., 2.], [np.nan, 3.], [np.nan, np.nan]])
>>> x[~np.isnan(x)]
array([1., 2., 3.])
>>> #Example:
>>> x = np.array([1., -1., -2., 3])
>>> x[x < 0] += 20
>>> x
array([ 1., 19., 18.,  3.])
>>> #Example : use with basic indexing
>>> x = np.array([[0, 1], [1, 1], [2, 2]])
>>> rowsum = x.sum(-1)
>>> x[rowsum <= 2, :]
array([[0, 1],
       [1, 1]])
>>> #Example : use with np.ix_
>>> x = np.array([[ 0,  1,  2],
...           [ 3,  4,  5],
...           [ 6,  7,  8],
...           [ 9, 10, 11]])
>>> rows = (x.sum(-1) % 2) == 0
>>> columns = [0, 2]

>>> x[np.ix_(rows, columns)]
array([[ 3,  5],
       [ 9, 11]])

Something More intresting then above:

Field Access:

If the ndarray object is a structured array the fields of the array can be accessed by indexing the array with strings, dictionary-like.

Indexing x['field-name'] returns a new view to the array.

If the accessed field is a sub-array, the dimensions of the sub-array are appended to the shape of the result.

>>> x = np.zeros((2,2), dtype=[('a', np.int32), ('b', np.float64,(3,3))])

>>> x['a']
array([[0, 0],
       [0, 0]])
>>> x['b']
array([[[[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]],

         [[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]]],


       [[[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]],

        [[0., 0., 0.],
         [0., 0., 0.],
         [0., 0., 0.]]]])

Flat Iterator indexing

x.flat returns an iterator that will iterate over the entire array (in C-contiguous style with the last index varying the fastest). This iterator object can also be indexed using basic slicing or advanced indexing as long as the selection object is not a tuple. This should be clear from the fact that x.flat is a 1-dimensional view. It can be used for integer indexing with 1-dimensional C-style-flat indices. The shape of any returned array is therefore the shape of the integer indexing object.

>>> x = np.arange(1, 7).reshape(2, 3)
>>> x
array([[1, 2, 3],
       [4, 5, 6]])
>>> x.flat[3]
4
>>> x.T
array([[1, 4],
       [2, 5],
       [3, 6]])
>>> x.T.flat[3]
5
>>> type(x.flat)
numpy.flatiter

>>> #assignment example
>>> x.flat = 3
>>> x
array([[3, 3, 3],
       [3, 3, 3]])
>>> #assignment example

>>> x.flat[[1,4]] = 1
>>> x
array([[3, 1, 3],
       [3, 1, 3]])

Pandas and Creating Dataframe

Aman Gupta — Sat, 09 Jan 2021 12:09:50 +0000

Here today we are talking about pandas, what are data frame and how to create them. So first see about pandas.

Pandas

Pandas is an open-source Python library providing high-performance data manipulation and analysis tools using its powerful data structures. The name Pandas is derived from the word Panel Data – an Econometrics from Multidimensional data.

[pandas] is derived from the term "panel data", an econometrics term for data sets that include observations over multiple time periods for the same individuals.

Pandas have so many uses that it might make sense to list the things it can't do instead of what it can do.

This tool is essentially your data’s home. Through pandas, you get acquainted with your data by cleaning, transforming, and analyzing it.

we import as follows:

>>> import pandas as pd

Python has three main Data Structure :
1 . Series :_ Series is a one-dimensional labeled array capable of holding any data type (integers, strings, floating point numbers, Python objects, etc.). The axis labels are collectively referred to as the index. The basic method to create a Series is to call:

>>> s = pd.Series(data, index=index)

Here, data can be many different things:

a Python dict
an ndarray
a scalar value (like 5)

2 . Data Frame : DataFrame is a 2-dimensional labeled data structure with columns of potentially different types. You can think of it like a spreadsheet or SQL table, or a dict of Series objects. It is generally the most commonly used pandas object. Like Series, DataFrame accepts many different kinds of input:

Dict of 1D ndarrays, lists, dicts, or Series
2-D numpy.ndarray
Structured or record ndarray
A Series
Another DataFrame

>>> d = {'one' : pd.Series([1., 2., 3.], index=['a', 'b', 'c']), 'two' : pd.Series([1., 2., 3., 4.], index=['a', 'b', 'c', 'd'])}
>>> df = pd.DataFrame(d)
>>> df
   one  two
a  1.0  1.0
b  2.0  2.0
c  3.0  3.0
d  NaN  4.0

3 . Panel : Panel is a somewhat less-used, but still important container for 3-dimensional data. The term panel data is derived from econometrics and is partially responsible for the name pandas: pan(el)-da(ta)-s. The names for the 3 axes are intended to give some semantic meaning to describing operations involving panel data and, in particular, econometric analysis of panel data. However, for the strict purposes of slicing and dicing a collection of DataFrame objects, you may find the axis names slightly arbitrary:

items: axis 0, each item corresponds to a DataFrame contained inside
major_axis: axis 1, it is the index (rows) of each of the DataFrames
minor_axis: axis 2, it is the columns of each of the DataFrames

>>> wp = pd.Panel(data)

The most common and used data structure in pandas is DataFrame. Now we see different ways to make dataframe using pandas.

The first one is, creating Dataframe by using list of list:

Example:


import pandas as pd    
data = [['Ram', 10], ['Aman', 15], ['Rishi', 14]]   
df = pd.DataFrame(data, columns = ['Name', 'Age'])   
df

Output:

Next methode is to create Dataframe by using python dict or ndarray.

Example:

import pandas as pd  
data = {'Name':['Ram', 'jhon', 'krish', 'jack'], 
        'Age':[20, 21, 19, 18]}  
df = pd.DataFrame(data) 
df

Output:

Next is by importing data from csv files. For this we use pd.read_csv() function.
Example:

import pandas as pd
df = pd.read_csv('data.csv')

The next way is by connecting DataBase. We can create a DataFrame by using DataBase also. We take an example code which connects SQLite database and creates dataframe.

For this, first create an Connection Object, and then use pd.read_sql_query() for creating dataframe.

import pandas as pd
import sqlite3
conn = sqlite3.connect("database.db")#put name of database
df = pd.read_sql_query(query)

There are some methods from which we can create data frame in pandas but there are several more ways to create data frames. Pandas IO tools support multiple types of file format for reading and writing data such as CSV, JSON, HTML, SAS, and Many more. For reading more about Pandas IO Tools go here or open this link:
https://pandas.pydata.org/docs/user_guide/io.html#io

Thanks for reading

DataBase in Python

Aman Gupta — Sat, 02 Jan 2021 05:55:13 +0000

Now next we discuss how can we work with the database in python and how to connect the database, how to create, and much more. So let's get started. First, discuss the database, what is it, why used, and different types of databases.

Database, also called electronic database, any collection of data, or information, that is specially organized for rapid search and retrieval by a computer. Databases are structured to facilitate the storage, retrieval, modification, and deletion of data in conjunction with various data-processing operations. A database management system (DBMS) extracts information from the database in response to queries.
There are various types of DataBases available as per the application. Some of them are listed here:

Distributed databases:

A distributed database is a type of database that has contributions from the common database and information captured by local computers. In this type of database system, the data is not in one place and is distributed at various organizations.

Relational databases:

This type of database defines database relationships in the form of tables. It is also called Relational DBMS, which is the most popular DBMS type in the market. Database example of the RDBMS system include MySQL, Oracle, and Microsoft SQL Server database.

Object-oriented databases:

This type of computers database supports the storage of all data types. The data is stored in the form of objects. The objects to be held in the database have attributes and methods that define what to do with the data. PostgreSQL is an example of an object-oriented relational DBMS.

Centralized database:

It is a centralized location, and users from different backgrounds can access this data. This type of computers databases store application procedures that help users access the data even from a remote location.

Open-source databases:

This kind of database stored information related to operations. It is mainly used in the field of marketing, employee relations, customer service, of databases.

Cloud databases:

A cloud database is a database which is optimized or built for such a virtualized environment. There are so many advantages of a cloud database, some of which can pay for storage capacity and bandwidth. It also offers scalability on-demand, along with high availability.

Data warehouses:

Data Warehouse is to facilitate a single version of truth for a company for decision making and forecasting. A Data warehouse is an information system that contains historical and commutative data from single or multiple sources. Data Warehouse concept simplifies the reporting and analysis process of the organization.

NoSQL databases:

NoSQL database is used for large sets of distributed data. There are a few big data performance problems that are effectively handled by relational databases. This type of computers database is very efficient in analyzing large-size unstructured data.

Graph databases:

A graph-oriented database uses graph theory to store, map, and query relationships. These kinds of computers databases are mostly used for analyzing interconnections. For example, an organization can use a graph database to mine data about customers from social media.

OLTP databases:

OLTP another database type which able to perform fast query processing and maintaining data integrity in multi-access environments.

Personal database:

A personal database is used to store data stored on personal computers that are smaller and easily manageable. The data is mostly used by the same department of the company and is accessed by a small group of people.

Multimodal database:

The multimodal database is a type of data processing platform that supports multiple data models that define how the certain knowledge and information in a database should be organized and arranged.

Document/JSON database:

In a document-oriented database, the data is kept in document collections, usually using the XML, JSON, BSON formats. One record can store as much data as you want, in any data type (or types) you prefer.

Hierarchical:

This type of DBMS employs the "parent-child" relationship of storing data. Its structure is like a tree with nodes representing records and branches representing fields. The windows registry used in Windows XP is a hierarchical database example.

Network DBMS:

This type of DBMS supports many-to-many relations. It usually results in complex database structures. RDM Server is an example of database management system that implements the network model.

For working in python there are many APIs available from which we can connect with different types of DBs.Some of Them are:

IBM DB2
FireBird
Informix
Ingres
MySQL
Oracle
PostgreSQL
Microsoft SQL Server
Microsoft Access
Sybase

There are many more DataBase Connections available in python which are listed on python official page: DatabaseInterface.

Most Python's database interface remains to Python's DB-API standard, and most of the databases have ODBC support. Other than that, the Java database usually supports JDBC, and programmers can work with that from Jython.

Using the Python structure, DB-API provides standard and support for working with databases. The API consists of:

Bring in the API module
Obtain database connection
Issue SQL statements and then store procedures
Close the connection

Let's check How to we work with Python DataBase API.

MySQL with Python

To access the MySQL DB using python first we need to install and then by using the below code we can create a MySQL DataBase using python.

# importing the module
import MySQLdb

# opening a database connection
db = MySQLdb.connect  ("localhost","testprog","stud","PYDB")

# define a cursor object
cursor = conn.cursor

# drop table if exists
Cursor.execute("IF STUDENT TABLE EXISTS DROP IT")

# query
sql = "CREATE TABLE STUDENT (NAME CHAR(30) NOT NULL, CLASS CHAR(5), AGE INT, GENDER CHAR(8), MARKS INT"

# execute query
cursor.execute(sql)

# close object
cursor.close()

# close connection
conn.close()

As we see above we close the connection and cursor object as we close in the file object. In other languages, developers are forced to use try...except...finally every time they work with a file (or any other type of resource that needs to be closed, like sockets or database connections). Luckily, Python loves us and gives us a simple way to make sure all resources we use are properly cleaned up, regardless of if the code returns or an exception is thrown: context managers.
There is also one more way to create a database using with statement which closes database itself. Let's see the benefit of Context Management.

ensuring that the resources are released even if we encounter some unhandled exceptions
readability
convenience — we make it easier for ourselves, as we will no longer forget to close the connections to external resources.

“Typical uses of context managers include saving and restoring various kinds of global state, locking and unlocking resources, closing opened files, etc.”

Thanks for reading.

Python: File I/O

Aman Gupta — Fri, 01 Jan 2021 15:16:16 +0000

So today here we discuss how we can work with files in python and how taking I/O in python.So lets first we talk about I/O streams in python. In python there are basically three main I/O:

text I/O
binary I/O
raw I/O A concrete object belonging to any of these categories is called a file object.Independent of its category, each concrete stream object will also have various capabilities: it can be read-only, write-only, or read-write. It can also allow arbitrary random access (seeking forwards or backwards to any location), or only sequential access (for example in the case of a socket or pipe).

All streams are careful about the type of data you give to them. For example giving a str object to the write() method of a binary stream will raise a TypeError. So will giving a bytes object to the write() method of a text stream.

Changed in version 3.3: Operations that used to raise IOError now raise OSError, since IOError is now an alias of OSError.

See I/O in some breaf:

1. text I/O

Text I/O expects and produces str objects. This means that whenever the backing store is natively made of bytes, encoding and decoding of data is made transparently as well as optional translation of platform-specific newline characters.

The easiest way to create a text stream is with open(), optionally specifying an encoding:

f = open("myfile.txt", "r", encoding="utf-8")

2. binary I/O

Binary I/O expects bytes-like objects and produces bytes objects. No encoding, decoding, or newline translation is performed. This category of streams can be used for all kinds of non-text data, and also when manual control over the handling of text data is desired.
The easiest way to create a binary stream is with open() with 'b' in the mode string:

f = open("myfile.jpg", "rb")

3. raw I/O

Raw I/O is generally used as a low-level building-block for binary and text streams; it is rarely useful to directly manipulate a raw stream from user code. Nevertheless, we can create a raw stream by opening a file in binary mode with buffering disabled.

f = open("myfile.jpg", "rb", buffering=0)

So we discuss streams in python. So how we can read and write files in python?
In python, we didn't worry about importing any library for handling files. Python has file handling tools built-in part of its core module. Python has a built-in set of functions that handle everything you need to read and write to files. We’ll now take a closer look at them.

Opening File in Python

for opening file in python, there is a built-in method open() which return object called data_file.The basic function usage for open() is the following:

file_object = open(filename, mode)

In above syntax filename is the name of the file that you want to interact with, and mode in the open function tells Python what we want to do with the file. There are multiple modes that you can specify when dealing with text files.

'w' - Write Mode : This mode is used when the file needs to be altered and information changed or added. Keep in mind that this erases the existing file to create a new one. File pointer is placed at the beginning of the file.
'r' - Read Mode : This mode is used when the information in the file is only meant to be read and not changed in any way. File pointer is placed at the beginning of the file.
'a' – Append Mode : This mode adds information to the end of the file automatically. File pointer is placed at the end of the file.
'r+' – Read/Write Mode : This is used when you will be making changes to the file and reading information from it. The file pointer is placed at the beginning of the file.
'a+' – Append and Read Mode : A file is opened to allow data to be added to the end of the file and lets your program read information as well. File pointer is placed at the end of the file.
'x' – Exclusive Creation Mode : This mode is used exclusively to create a file. If a file of the same name already exists, the function call will fail.

When we are using binary files, we will use the same mode specifiers. However, we add a b to the end. So a write mode specifier for a binary file is 'wb'. The others are 'rb', 'ab', 'r+b', and 'a+b' respectively.

File Closing in python

The closing file is most important when we working with files in python. It frees up system resources that program is using for I/O purposes. When writing a program that has space or memory constraints, this lets us manage resources effectively.

Also, closing a file ensures that any pending data is written out to the underlying storage system, for example, our local disk drive. By explicitly closing the file you ensure that any buffered data held in memory is flushed out and written to the file.

The function to close a file in Python is simply fileobject.close(). Using the data_file file object that we created in the previous example, the command to close it would be:

file_object.close()

After you close a file, you can’t access it any longer until you reopen it at a later date.

In Python, the best practice for opening and closing files uses the with keyword. This keyword closes the file automatically after the nested code block completes.

Reading from file

There are a few functions for reading from a file. let's discuss them one by one with usecases.

1. `read(size)`

By using this method we read a specific no of bytes from a file. By default this method will read the entire file and print it out to the console as either a string (in text mode) or as byte objects (in binary mode).
Example :

with open("workData.txt", "r+") as work_data:
    print("This is the file name: ", work_data.name)
    line = work_data.read()
    print(line)

2. `readline(size)`

By using this method we process the file line by line. size parameter here represent no of bytes from each line. by default it prints the first line of the file.
Example :

with open("workData.txt", "r+") as work_data:
     print("This is the file name: ", work_data.name)
     line_data = work_data.readline()
     print(line_data)

3. Processing an Entire Text File Line-By-Line

The easiest way to process an entire text file line-by-line in Python is by using a simple loop:

with open("workData.txt", "r+") as work_data:
    for line in work_data:
        print(line)

Writing in a python File

For writing in a file, python has a built-in function write().
when we open the file as w mode so that clear out all existing data so opening the file in append mode is preferable.

Example :

values = [1234, 5678, 9012]

with open("workData.txt", "a+") as work_data:
    for value in values:
        str_value = str(value)
        work_data.write(str_value)
        work_data.write("\n")

Thank you for reading

Handling Data in Python

Aman Gupta — Fri, 01 Jan 2021 08:09:33 +0000

In this blog, we are discussing how python handles data, what are various data types, and the data structure in it.

Python has many built-in data types and many specialized data types. we are discussing them one by one here. Let's start with built-in data types:

1. `dict`

A mapping object maps hashable values to arbitrary objects. Mappings are mutable objects. There is currently only one standard mapping type, the dictionary.
A dictionary’s keys are almost arbitrary values. Values that are not hashable, that is, values containing lists, dictionaries, or other mutable types (that are compared by value rather than by object identity) may not be used as keys. Numeric types used for keys obey the normal rules for numeric comparison: if two numbers compare equal (such as 1 and 1.0) then they can be used interchangeably to index the same dictionary entry. (Note, however, that since computers store floating-point numbers as approximations it is usually unwise to use them as dictionary keys.)

Dictionaries can be created by placing a comma-separated list of key: value pairs within braces.

for example :

{'jack': 4098, 'sjoerd': 4127}
or
{4098: 'jack', 4127: 'sjoerd'}

Dictionaries can be created by several means:

Use a comma-separated list of key: value pairs within braces.
Use a dict comprehension.
Use the type constructor.

To illustrate, the following examples all return a dictionary equal to {"one": 1, "two": 2, "three": 3}:

a = dict(one=1, two=2, three=3)
b = {'one': 1, 'two': 2, 'three': 3}
c = dict(zip(['one', 'two', 'three'], [1, 2, 3]))
d = dict([('two', 2), ('one', 1), ('three', 3)])
e = dict({'three': 3, 'one': 1, 'two': 2})
f = dict({'one': 1, 'three': 3}, two=2)

2. `list`

Lists are mutable sequences, typically used to store collections of homogeneous items (where the precise degree of similarity will vary by application).

The constructor builds a list whose items are the same and in the same order as iterable’s items. iterable may be either a sequence, a container that supports iteration, or an iterator object. If iterable is already a list, a copy is made and returned, similar to iterable[:]. For example, list('abc') returns ['a', 'b', 'c'] and list( (1, 2, 3) ) returns [1, 2, 3]. If no argument is given, the constructor creates a new empty list, [].

Lists may be constructed in several ways:

Using a pair of square brackets to denote the empty list.
Using square brackets, separating items with commas.
Using a list comprehension.
Using the type constructor.

for example :

a = []
b = [1,2,3]
c = [x for x in iterable]
d = list('abc')
e = list((1,2,3))

3. `set` and `frozenset`

A set object is an unordered collection of distinct hashable objects. Common uses include membership testing, removing duplicates from a sequence, and computing mathematical operations such as intersection, union, difference, and symmetric difference.
There are currently two built-in set types, set and frozenset.

The set type is mutable — the contents can be changed using methods like add() and remove(). Since it is mutable, it has no hash value and cannot be used as either a dictionary key or as an element of another set.
The frozenset type is immutable and hashable — its contents cannot be altered after it is created; it can therefore be used as a dictionary key or as an element of another set.

Sets can be created by several means:

Use a comma-separated list of elements within braces
Use a set comprehension
Use the type constructor

a = {'jack', 'sjoerd'}
b = {c for c in 'abracadabra' if c not in 'abc'}
c = set()
d = set('foobar')
e = set(['a', 'b', 'foo'])

4. `tuple`

Tuples are immutable sequences, typically used to store collections of heterogeneous data (such as the 2-tuples produced by the enumerate() built-in). Tuples are also used for cases where an immutable sequence of homogeneous data is needed (such as allowing storage in a set or dict instance).

Tuples may be constructed in a number of ways:

Using a pair of parentheses to denote the empty tuple
Using a trailing comma for a singleton tuple
Separating items with commas
Using the tuple() built-in:

The constructor builds a tuple whose items are the same and in the same order as iterable’s items. iterable may be either a sequence, a container that supports iteration, or an iterator object.

a = ()
b = ('a', )
c = ('a', 'b', 'c')
d = tuple() #return empty tuple

5. `str`

Textual data in Python is handled with str objects, or strings. Strings are immutable sequences of Unicode code points. String literals are written in a variety of ways:

Single quotes: 'allows embedded "double" quotes'
Double quotes: "allows embedded 'single' quotes".
Triple quoted: '''Three single quotes''', """Three double quotes"""

a = 'Aman'
b = "Aman"
c = '''I love python'''
d = """ I am enjoying it """

Triple quoted strings may span multiple lines - all associated whitespace will be included in the string literal.

6. `bytes` or

Bytes objects are immutable sequences of single bytes. Since many major binary protocols are based on the ASCII text encoding, bytes objects offer several methods that are only valid when working with ASCII compatible data and are closely related to string objects in a variety of other ways.

Firstly, the syntax for bytes literals is largely the same as that for string literals, except that a b prefix is added:

Single quotes.
Double quotes.
Triple quoted.

a = b'still allows embedded "double" quotes'
b = b"still allows embedded 'single' quotes"
c = b'''3 single quotes'''
d = b"""3 double quotes"""

Only ASCII characters are permitted in bytes literals (regardless of the declared source code encoding). Any binary values over 127 must be entered into bytes literals using the appropriate escape sequence.

While bytes literals and representations are based on ASCII text, bytes objects actually behave like immutable sequences of integers, with each value in the sequence restricted such that 0 <= x < 256

7. `bytearray`

bytearray objects are a mutable counterpart to bytes objects.

As bytearray objects are mutable, they support the mutable sequence operations in addition to the common bytes and bytearray operations

Since 2 hexadecimal digits correspond precisely to a single byte, hexadecimal numbers are a commonly used format for describing binary data. Accordingly, the bytearray type has an additional class method to read data in that format

There is no dedicated literal syntax for bytearray objects, instead they are always created by calling the constructor:

Creating an empty instance
Creating a zero-filled instance with a given length
From an iterable of integers
Copying existing binary data via the buffer protocol

a = bytearray()
b = bytearray(10)
c = bytearray(range(20))
d = bytearray(b'Hi!')

Above we see the built-in data-types in python but there are some specialized data types available in python which are also amazing .

8. `datetime`

The datetime module supplies classes for manipulating dates and times.

While date and time arithmetic is supported, the focus of the implementation is on efficient attribute extraction for output formatting and manipulation.

Available Types

datetime.date : An idealized naive date, assuming the current Gregorian calendar always was, and always will be, in effect.
datetime.time : An idealized time, independent of any particular day, assuming that every day has exactly 24*60*60 seconds.
datetime.datetime : A combination of a date and a time.
datetime.timedelta : A duration expressing the difference between two date, time, or datetime instances to microsecond resolution.
datetime.tzinfo : An abstract base class for time zone information objects. These are used by the datetime and time classes to provide a customizable notion of time adjustment
datetime.timezone : A class that implements the tzinfo abstract base class as a fixed offset from the UTC.

9. `zoneinfo`

The zoneinfo module provides a concrete time zone implementation to support the IANA time zone database as originally specified in PEP 615. By default, zoneinfo uses the system’s time zone data if available; if no system time zone data is available, the library will fall back to using the first-party tzdata package available on PyPI.

ZoneInfo is a concrete implementation of the datetime.tzinfo abstract base class, and is intended to be attached to tzinfo, either via the constructor, the datetime.replace method or datetime.astimezone.
For example :

>>> from zoneinfo import ZoneInfo
>>> from datetime import datetime, timedelta

>>> dt = datetime(2020, 10, 31, 12, tzinfo=ZoneInfo("America/Los_Angeles"))
>>> print(dt)
2020-10-31 12:00:00-07:00

>>> dt.tzname()
'PDT'

10. `Calendar`

This module allows you to output calendars like the Unix cal program, and provides additional useful functions related to the calendar. By default, these calendars have Monday as the first day of the week, and Sunday as the last (the European convention). Use setfirstweekday() to set the first day of the week to Sunday (6) or to any other weekday.

There are also some types available in calendar.

calendar.Calendar : Creates a Calendar object.
calendar.TextCalendar : This class can be used to generate plain text calendars.
calendar.HTMLCalendar : This class can be used to generate HTML calendars.

There are also some more data-types in python which are much advance like Collections, heapq, bisect, array etc.

Thanks for reading.

Python: Amazing Features

Aman Gupta — Wed, 30 Dec 2020 15:30:45 +0000

Hello, here we are to discuss some amazing features of python which are making python magical.

1. Getting Code of Python Object

In python, we can get the internal implementation of the python object by using inspect module. It is used to introspect live objects in a program and look at the source code of modules, classes, and functions that are used throughout a program. This is powerful because this module can actually be used to extract the source code of a function itself, parse the arguments which that function accepts, and related library documentation.

For example:

Source Code:

For getting code of complete module:

Output

For getting code of a class or method:

Output

2. `exec()` function in Python

exec() function is used for the dynamic execution of Python program which can either be a string or object code. If it is a string, the string is parsed as a suite of Python statements which is then executed unless a syntax error occurs and if it is an object code, it is simply executed. We must be careful that the return statements may not be used outside of function definitions not even within the context of code passed to the exec() function. It doesn't returns any value, hence returns None.

Syntex of `exec()`:

exec(object[, globals[, locals]])

Parameters:

object :- this can be a string or object code
globals :-This can be a dictionary and the parameter is optional
locals :-This can be a mapping object and is also optional

The exec() function accepts large blocks of code, unlike the eval() function which only accepts a single expression

Example:

x = 'name = "John"\nprint(name)'
exec(x)

Output:

John

3.`call` methode in Python

Python has a set of built-in methods and __call__ is one of them. The __call__ method enables Python programmers to write classes where the instances behave like functions and can be called like a function. This means we can call a class object as a function. That is an amazing benefit of python and feels like magic. There is too much confusion in function, class, and instances so see this using example.

Example :

Output:

Instance Created
200

4. Metaclass

A metaclass in Python is a class of a class that defines how a class behaves. A class is itself an instance of a metaclass. A class in Python defines how the instance of the class will behave. In order to understand metaclasses well, one needs to have prior experience working with Python classes. Before we dive deeper into metaclasses, let's get a few concepts out of the way.

Creating Custom Metaclasses

In Python, we can customize the class creation process by passing the metaclass keyword in the class definition. This can also be done by inheriting a class that has already passed in this keyword.

class MyMeta(type):
    pass

class MyClass(metaclass=MyMeta):
    pass

class MySubclass(MyClass):
    pass

We can see below that the type of MyMeta class is type and that the type of MyClass and MySubClass is MyMeta.

Code:

print(type(MyMeta))
print(type(MyClass))
print(type(MySubclass))

Output:

<class 'type'>
<class '__main__.MyMeta'>
<class '__main__.MyMeta'>

"Metaclasses are deeper magic than 99% of users should ever worry about. If you wonder whether you need them, you don’t (the people who actually need them know with certainty that they need them, and don’t need an explanation about why)."

5. Dataclass

One new and exciting feature coming in Python 3.7 is the data class. A data class is a class typically containing mainly data, although there aren’t really any restrictions. It is created using the new @dataclass decorator, as follows:

from dataclasses import dataclass

@dataclass
class DataClassCard:
    rank: str
    suit: str

A data class comes with basic functionality already implemented. For instance, you can instantiate, print, and compare data class instances straight out of the box:

>>> queen_of_hearts = DataClassCard('Q', 'Hearts')
>>> queen_of_hearts.rank

Output:

'Q'

Thanks for reading

Tips to make Python Code faster

Aman Gupta — Wed, 30 Dec 2020 10:34:39 +0000

At this time we need everything fast and this thing also applies to programming. So we can talk here about how can we make the python program run Faster.

The first way to make a program faster is to understand the Data Structure that which one is fast and which is slow. So let's talk about python list and dict. For searching an element, which one is Faster?

When we searching an element in a list then we have only one way that is by accessing each item from the 0th index to the last index but the dictionary utilizes a data structure called a hashmap (Python dictionaries are optimized versions), and a key will be converted using a hash algorithm from a string (or whatever) into an integer value, and it is a couple of very simple calculations to take that integer and find the right place in the dictionary to look. So the dict Data Structure is faster than list for searching elements and by using a faster data structure we can make our code faster.

Another way to do reduce the run time of a python code is by reducing Memory footprint.
For example:

This is inefficient because a new string gets created upon each pass. Use a list and join it together:

Similarly, avoid the + operator on strings:

The next way to speedup python code is by using built-in functions because built-in functions like sum, max, any, map, etc are implemented in C. They are very efficient and well tested. These functions reduce the runtime of the code and make the program Faster.

Python: interpreted or compiled?

Aman Gupta — Wed, 30 Dec 2020 03:31:39 +0000

One most common question about python is that it is a compiled language or interpreted language or both?

First, see what is terms compiled and interpreted means(in a simple world):
Compiled Language: The high-level Language whose code first converted into machine code by the compiler and then executed by the executor.

Interpreted Language: The high-level language whose code executed by an interpreter on the go.

In various books on python programming, it is mention that Python is an Interpreted Language. And, also as per the above definitions Python is an Interpreted language.

But the world is not as simpler and limited. So that is half correct that Python is an Interpreted language. The Python program first compiled and then interpreted. This compilation part is hidden from programmers, so they think that it is an interpreted language. The compilation part is done when we execute our code and this will generate byte-code and internally this byte-code gets converted by python virtual machine according to the machine and operating system.

The compiled part of the Python program:

now if we run this code using the command prompt or terminal, first save this file using .py extension.
and run the following command :

python -m py_compile main.py

As we press enter the compiled code will get generated and saved in a folder which is created inside the folder contain our python code.

There is one another way to compile python code by using compile() method. This function takes a string, byte-string, or AST as input and returns a python code object which is executed by using eval() method or exec() method.

Syntex of compile() :

compile(source, filename, mode, flags=0, dont_inherit=False, optimize=-1)

Parameters :-

Source : String, byte-string, AST object
Filename : Filename from which code was read
mode : three mode 'exec', 'eval', and 'single'
Flags(optional) and dont_inherit(optional) – Default value=0. It takes care that which future statements affect the compilation of the source.
Optimize(optional) – It tells optimization level of compiler. Default value -1.

Sample Code:

srcCode = 'x = 10\ny = 20\nmul = x * y\nprint("mul =", mul)'
execCode = compile(srcCode, 'mulstring', 'exec') 
exec(execCode)

Output:

mul = 200

Thanks for reading.

My Hacktoberfest 2020

Aman Gupta — Tue, 20 Oct 2020 04:03:32 +0000

The Amazing One Month of Opensource

This is an amazing event for opensource specially for beginners who are new in this domain of technology and opensource . This is my 2nd hacktoberfest and i am happy that i completed it. Today also i got 1 year club badge and i am completed 1 year with DEV community. and This is my First post on DEV . I register for this fest on the day 1 and started my work which i pre-planed and just complete this Fest.
This year i select to plant a tree over the swag.

DEV Community: Aman Gupta

Python: List vs Dictionaries

List:

Dictionary:

Numpy : Advance Indexing

Basic Slicing and Indexing

Advance Indexing in Numpy array

Integer array Indexing

Purely integer array indexing

Combining advanced and basic indexing

Boolean array Indexing

Something More intresting then above:

Field Access:

Flat Iterator indexing

Pandas and Creating Dataframe

Pandas

DataBase in Python

Distributed databases:

Relational databases:

Object-oriented databases:

Centralized database:

Open-source databases:

Cloud databases:

Data warehouses:

NoSQL databases:

Graph databases:

OLTP databases:

Personal database:

Multimodal database:

Document/JSON database:

Hierarchical:

Network DBMS:

MySQL with Python

Python: File I/O

1. text I/O

2. binary I/O

3. raw I/O

Opening File in Python

File Closing in python

Reading from file

1. read(size)

2. readline(size)

3. Processing an Entire Text File Line-By-Line

Writing in a python File

Handling Data in Python

1. dict

2. list

3. set and frozenset

4. tuple

5. str

6. bytes or

7. bytearray

8. datetime

Available Types

9. zoneinfo

10. Calendar

Python: Amazing Features

1. Getting Code of Python Object

2. exec() function in Python

Syntex of exec():

Parameters:

Example:

Output:

3.__call__ methode in Python

Example :

Output:

4. Metaclass

Creating Custom Metaclasses

Code:

Output:

5. Dataclass

Output:

Tips to make Python Code faster

Python: interpreted or compiled?

The compiled part of the Python program:

Sample Code:

Output:

My Hacktoberfest 2020

The Amazing One Month of Opensource

1. `read(size)`

2. `readline(size)`

1. `dict`

2. `list`

3. `set` and `frozenset`

4. `tuple`

5. `str`

6. `bytes` or

7. `bytearray`

8. `datetime`

9. `zoneinfo`

10. `Calendar`

2. `exec()` function in Python

Syntex of `exec()`:

3.`call` methode in Python