DEV Community: Mohamed Fares Ben Ayed

Golang interfaces

Mohamed Fares Ben Ayed — Sun, 25 Sep 2022 10:31:00 +0000

What made me interested in Go programming language is the concept behind its interfaces. The object oriented programming is conceived differently in Go, if we compare Go OOP to other OOP languages, we will notice a different approach of how objects behave in this language.

Go is famous for its simple and readable syntax, it is a language that contains only 25 built-in keywords. For example, a famous keyword like the class keyword (exists in most languages) doesn't exist in the syntax of Go. Instead, the struct is an alternative keyword for the class definition.

Because of this simplicity, concepts like inheritance, polymorphism and abstraction are implemented differently in Go.

Before starting to explain interfaces, an object definition in Go is of type struct. so in order to define a Person struct in Go, we describe it in the following block of code :

type Person struct{
    Name string
    Age int
}

the person has 2 attributes (Name and Age attributes).

What is an interface?

An interface is a set of method signatures that a type can implement. therefore, the interface defines the behaviors of the object (any kind of object).

The definition is as simple as that.

For example, let's say we have an interface called Speak, it is defined as the following :

type Speak interface{
    Arabic()
    French()
    English()
}

To make a type implement the Speak interface, it (object or variable) must define all of the 3 methods implemented abstractly in the Speak interface : Arabic(), French() and English().

The Person type implements Speak, if itself has the 3 methods definitions :

func (p Person) Arabic(){ /* code here */ }
func (p Person) French(){ /* code here */ }
func (p Person) English(){ /* code here */ }

In other languages, you need to inherit from the interface using the keyword implements. But you just need to define the functions.

The capability of an empty interface

Go programming language has static typing, that means that any variable in a program life cycle has to have only one type (built-in or custom). An integer variable remains as an integer variable, the same thing applies for the other types.
Unlike Python, a variable can change its type only by reassigning it with a new different value.

Yet, Go can imitate dynamic typing of Python, and can change the variable from one type to another in a single program, and we can thank empty interfaces for that.

let's see this in an example :

func main(){
    var x int
    x = 5
    x = "change x"
    fmt.Println(x)
}

we reassigned the integer value x by a variable with a different type (a string "change x"), the program then displayed the following error :

./prog.go:10:6: cannot use "change x" (untyped string constant) as int value in assignment

However, if we change the type of x from int to an empty interface interface{}, the program will run successfully and the variable will be reassigned again.

func main() {
    var x interface{}
    x = 5
    fmt.Printf("x=%v of type :%T\n", x, x)
    x = "Fares"
    fmt.Printf("x=%v of type :%T", x, x)
}

The type of x changed dynamically from integer to string, and we can easily reassign it with any existing type in Go.

x=5 of type :int
x=Fares of type :string

Now, if we declare an empty interface variable without assigning, what can be the result?

it will be a nil value.

Polymorphism using interfaces

if you want to do polymorphism in other languages, you need to make a parent class and multiple children classes inheriting from the parent.
then you declare an array of the parent class type and instantiate the subclasses and adds them to the array.

In GO, things go differently, we can deal with an interface as a global custom type. we can build polymorphism on the example above and try this code block :

type Speaker interface{
    Arabic()
    English()
    French()
}
type NativeSpeaker struct{
    Name string
}
type ForeignSpeaker struct{
    Name string
    Nationality string
}
func (NS NativeSpeaker) Arabic(){ /* code here */ }
func (NS NativeSpeaker) English(){ /* code here */ }
func (NS NativeSpeaker) French(){ /* code here */ }
func (FS foreignSpeaker) Arabic(){ /* code here */ }
func (FS ForeignSpeaker) English(){ /* code here */ }
func (FS ForeignSpeaker) French(){ /* code here */ }

func main() {
    listOfSpeakers := make([]Speaker, 0)
    Sp1 := NativeSpeaker{Name: "John"}
    Sp2 := ForeignSpeaker{Name: "Mohamed"}
    listOfSpeakers = append(listOfSpeakers, Sp1, Sp2)
    fmt.Println(listOfSpeakers)
}

As you can see the Speaker interface is the global types that assembles NativeSpeaker and ForeignSpeaker custom types. those shares the same method signatures that were declared in Speak interface.

Working with groupby in pandas: From Zero to Hero.

Mohamed Fares Ben Ayed — Sun, 17 Jul 2022 09:44:00 +0000

What is aggregation?

One of the important Tools in data science is to know how to aggregate data, Aggregation techniques enable the programmer (or the data scientist) to understand more about the data he's working on.

But what is aggregation exactly? Aggregation is to group the data observations into distinct groups (or categories) and then we can apply some statistical functions on those groups like mean, median, standard deviation, sum, product, count, minimum, maximum and many other cool functions.

After reading this post, we'll be able to work with any kind of dataset and apply complex groupby operations. In this post we will work on the pokemon data (The anime of pokemon and its video games). we don't need to worry! we will explain each detail of this data before moving to the code.

To download the dataset, click this link. it is a link from github that contains the raw data of the pokemon dataset. we can notice that the format of data is similar to the format of a csv file

We can read the dataset directly from the provided github link. Here is our starting block of code:

import pandas as pd
path='https://gist.githubusercontent.com/armgilles/194bcff35001e7eb53a2a8b441e8b2c6/raw/92200bc0a673d5ce2110aaad4544ed6c4010f687/pokemon.csv'
pokemon_data = pd.read_csv(path, index_col = '#')

This our post summary:

Dataset review and understanding.
Code steps
Step 1
Step 2
Step 3
Step 4

Dataset review and understanding

Pokemon is an anime which the main character always wants to collect pokemon monsters and we need to help him to know more about those pokemons.

So, our data is definitely about pokemon monsters, each observation of pokemon data represents a pokemon(a certain monster).

Every pokemon has a set of attributes like Attack, Defense, HP and Monster Type.

Our dataset contains a set of 11 columns (excluding the name of the pokemon):

Type 1 : the type of a pokemon, the pokemon can have type like Grass, Fire or any other attribute.
Type 2 : the same as the first attribute, but with different types.
Total : the total sum of the following numerical attributes.
Attack : this column describes the attack points of a monster.
Defense : this column describes the defense points of a monster.
HP : this column describes the health points of a monster.
Sp. ATK : this column describes the special attack points of a monster.
Sp. DEF : this column describes the special defense points of a monster.
Speed : this column describes the speed points of a monster.
Generation : this column describes the generation category, it is ranked from 1 to 6.
Legendary : this column describes whether the monster has a legendary class or not, it is represented by boolean a value (True or False).

Now, after we read the dataset, we need to move on the next steps of learning groupby operations. But before that we need to make a few lines of codes.

del pokemon_data['Name']

The code above deleted the Name column, because we don't need a string information in our data. groupby operations need just categorical or numerical information.

Code steps

From zero to hero.

In this section, we will introduce our learning steps of groupby operations. each step will state a statistical question, and we will show the code that will provide the output to that question.

this is our final working dataset:

Step 1

Apply a groupby operation with a mean function.

Question : what is the mean Total attribute for each pokemon Generation attribute?
Answer : First, you need to group the data into a Generation group, and then apply the mean on the Total.
Code:

pokemon_data[['Total', 'Generation']].groupby('Generation').mean()

Output:

Generation	Total
1	426.813253
2	418.283019
3	436.225000
4	459.016529
5	434.987879
6	436.378049

This can be done with the sum() function, the min() functions and many other statistical functions.

Try this block of code, to try out different aggregate functions:

pokemon_data[['Total', 'Generation']].groupby('Generation').size() #aggregate by counting the values
pokemon_data[['Total', 'Generation']].groupby('Generation').min() #aggregate by minimum
pokemon_data[['Total', 'Generation']].groupby('Generation').max() #aggregate by maximum
pokemon_data[['Total', 'Generation']].groupby('Generation').std() #aggregate by standard deviation
pokemon_data[['Total', 'Generation']].groupby('Generation').sum() #aggregate by sum
pokemon_data[['Total', 'Generation']].groupby('Generation').prod() #aggregate by production

Step 2

Multiple aggregate functions in a single groupby

Question : Now, what is the mean and standard deviation of Speed attribute for each generation ?
Answer : we can aggregate multiple functions in a single output using the agg() function.
Code : There are 2 versions of code that can result the same output, the second one is a simplified version :

Code 1 :

pokemon_data.groupby("Generation").agg(
   average_speed = pd.NamedAgg("Speed","mean"), # here we apply the mean
   std_speed = pd.NamedAgg("Speed", "std"), # here we apply the standard deviation
)

Code 2 :

pokemon_data.groupby("Generation").agg(
average_speed=("Speed","mean"), # applying the mean
std_speed = ("Speed", "std"), # applying the standard deviation
)

Output :

Generation	average_speed	std_speed
1	72.584337	29.675857
2	61.811321	27.263132
3	66.925000	31.331972
4	71.338843	28.475005
5	68.078788	28.726632
6	66.439024	25.691954

Step 3:

Group by multiple columns.

Question : what is the maximum Attack attribute for each Generation attribute? and we want to know which Legendary attribute has the maximum Attack for each Generation.
Answer : we can do that by goruping by Generation and Legendary at the same time.
Code :

pokemon_data.groupby(['Generation', 'Legendary']).agg(
maximum_attack = ('Attack', 'max')
)

Output :

Generation	Legendary	maximum_attack
1	True	155
1	False	190
2	True	185
2	False	130
3	True	165
3	False	180
4	True	170
4	False	160
5	True	147
5	False	170
6	True	150
6	False	160

Step 4:

Sorting group results. (Multiple column case)

Question : What are the types (Type 1 to be specific) of pokemon that have the maximum Attack attributes for each Generation column? the results need to be sorted, so that the maximum value for each Generation and type will be spotted easily.
Answer : Code and output will explain.
Code :

pokemon_data.groupby(['Generation', 'Legendary']).agg(
maximum_attack = ('Attack', 'max')
).sort_values(by = 'maximum_attack', ascending = False)

Output : The output is specified with Generation 1 attribute.

1	Psychic	190
	Bug	155
	Water	155
	Rock	135
	Dragon	134
	Fighting	130
	Fire	130
	Ground	130
	Normal	125
	Grass	105
	Poison	105
	Electric	90
	Ice	85
	Fairy	70
	Ghost	65

Step 5:

use groupbies with filtering:

This step won't be in question/Answer format.
We can filter our data by Type 1 for example:

pokemon_data[pokemon_data['Type 1'] == 'Water'].head()
# or
pokemon_data.loc[ pokemon_data['Type 1'] == 'Water' , ['Attack']]

Now we need to use a grouby operation with a filter:

grouped = pokemon_data.groupby('Type 1')
grouped.filter(lambda x: x['Attack'].mean() > 100)

This code displays only the Dragon type, because the mean attack of all dragon pokemons is above 100.

How I created a mini ORM with python?

Mohamed Fares Ben Ayed — Wed, 05 Jan 2022 20:14:51 +0000

Introduction

When you code in a Django web framework, we all know that you won’t directly work with databases. There is an ORM (Object relational mapping) who will interact with database using migration and SQL operation. So, I want in this tutorial to show you how to implement an ORM manager from scratch.

Object–relational mapping (ORM) in computer science is a technique for converting data between incompatible type systems using object-oriented programming languages in order to create virtual database objects.

Python is a programming language that enables you to design freely anything that you want.

And honestly I was able to implement a Mini ORM manager merely within 3 hours.

Agenda

Project Design
Database manager
Model manager
Migration manager
Conclusion

Project Design

Our Project is divided into 3 main components:

A connection manager to connect directly with a database using SQL commands (we used SQLite3 in our case).
A model manager or model file that contains all the definition of the models we need to migrate to a database.
A simple command manager that enables user to input a command (in the command prompt).

The Project files are also 3 python files (check this Github repository ):

Python file	Functionality
migrate_manager.py	Migration manager
base.py	Model manager
db_manager.py	Database manager

Database manager

We mentioned before that we will use SQLite database. Thanks to python sqlite3 default library, we can connect directly with a database using a python script. If you check the documentation of sqlite3 library, you will notice that it is very easy to connect SQLite using python.

Just a few lines of python code will do the job. So, you need to instantiate an sqlite3 connection object with specifying the database file “example.db”, then make a cursor to execute SQL commands without forgetting to commit the changes and to close the connection.

First import the sqlite3 library and instantiate the connection:

import sqlite3
con = sqlite3.connect('example.db')

Then open a cursor, execute the commands you want and finally close the connection:

cursor= con.cursor()
cursor.execute('''CREATE TABLE stocks (date text, trans text, symbol text, qty real, price real)''')
cursor.execute("INSERT INTO stocks VALUES ('2006-01-05','BUY','RHAT',100,35.14)")
connection.commit()
connection.close()

However, this kind of implementation won’t be reliable in our project, because there is too much redundancy of this block of code. Imagine I will repeat this code with every SQL operation I use including CREATE, DELETE, UPDATE and RETRIEVE.

In fact, this problem has a solution, there is a thing in python called context manager. Context manager is a way to manage resources precisely without any cost in complexity and memory.

In order to make a context manager that can connect with a database, we make a class called ConnectionSqliteManager that opens an SQL connection and instantiates the cursor object.

In this class, we need to write an __enter__ and __exit__ magic class methods as a necessary ingredient for database resource management.

So the structure of the context manager class is as follows :

The __enter__ method connects the example.db file (setup operation) and returns the Connection object to variable Connection.
The __exit__ method takes care of closing the connection on exiting the with block(teardown operation).

In the scope with , you manage database within the scope, without taking the bother of opening and closing the connection.

class ConnectionSqliteManager:
    def __init__(self,filename):
        self.filename = filename

    def __enter__(self):
        print("Connection started ...")
        self.connection = sql3.connect(self.filename)
        self.cursor = self.connection.cursor()
        return self  
    def __exit__(self, type, value, traceback):
        print("Connection ended ...")
        self.connection.close()

with ConnectionSqliteManager("example.db") as Connection:
    # Do what you want with Connection instance

TIP:

You can make a decorator (Inside “ConnectionSqliteManager” class) on top of each SQL operation method, to commit the changes.

Never forget to add some sqlite3 error exception in each command execution.

#inside ConnectionSqliteManager class
def commit(operation):
        def wrapper(self, *args,**kwargs):
            operation(self, *args, **kwargs)
            self.connection.commit()
        return wrapper
@commit
def Sql_operation(self):
    # execute an sql command here
    pass

Model manager

A model is a datastore entity that has a key and a set of properties. . A model is a Python class that inherits from the Model class. The model class defines a new kind of datastore entity and the properties the kind is expected to take.

I won’t dive into too much implementation here. all you have to do is to define a model class like the following:

class Model(baseModel):
    base_model = base
    tablename = "Model"
    fields = ("field_1", "field_2")

In this code, you need to specify the name of the table of the database and its fields.

In order to prepare your model for migration, you have to add it in the model_list list:

model_list = [Model,
              ]

Command manager

Now let’s prepare the user for a good interface in our command prompt. To implement it, we used the argparse default library to let the user input the arguments in the command prompt and execute the migration.

all details are in the migrate_manager.py file, there is no need to explain each line of code. I made it as simple as possible.

So all you have to do is to execute this command:

python migrate_manager.py migrate

The output is as following:

Begin database Migration ...

Model Migration
Connection started ...
Model: created successfully!
2022-01-04 02:29:53.402991: Commit is successful!!
Connection ended ...

Conclusion

It’s good to implement from scratch a technique like ORM, it will help you understand and learn quickly technologies without any difficulties with grasping the concept behind.