DEV Community: mohaned mashaly

Database Design of the app, p.3

mohaned mashaly — Fri, 18 Dec 2020 21:19:52 +0000

This is post is part of the DigitalOcean Hackathon Series.

In this post inshallah, I will present the design of the database I will be using for the entities and the relations in AroundTheHood project.

Any comments or feedbacks are highly appreciated and valued.

Front-end Prototype of the app, p.2

mohaned mashaly — Tue, 15 Dec 2020 22:17:45 +0000

series : ["DigitalOcean Hackathon"]

This is post is part of the DigitalOcean Hackathon Series.

In this post inshallah, I will present the prototype of the App where I will show you how the front-end will look like.

The first image shows a blueprint of how the Home-page will look like and it's main functions.

The second image shows a blueprint of how the Signup/Login form will look like.

The third image shows a blueprint of how the lending form will look like.

The Last image shows the user profile and how it will look like, the profile contains personal info of the user, past transactions, his/her total earnings.

of-course this design is not final as I will be following the agile methodology where I will be developing the app in sprints and add other features on the way.

in the next post inshallah I will share with you the database design and the above features implemented with Html, CSS and bootstrap.

The architecture of the app, p.1

mohaned mashaly — Mon, 14 Dec 2020 20:37:44 +0000

series : ["DigitalOcean Hackathon"]

This is post is part of the DigitalOcean Hackathon Series.

In this post inshallah, I will present The Architecture which highlights the main modules and functions of the platform.

The platform idea is to locate nearby neighbors where neighbors in the same hood can rent and lend gadgets they want.

Data Exploration of women education in us

mohaned mashaly — Sat, 03 Oct 2020 18:22:36 +0000

this article is going to be extremely short since I will show only visuals and graphs from data-set I found on Kaggle where it highlights college majors women in us tend to enrol and finish.

I will present the top 5 college majors women in us got their degree at.

1- Health Professions

2- Education

3- public Administration

4- Foreign Languages

5- Psychology

The full notebook can be found here :- https://www.kaggle.com/mohanedmashaly/data-exploration

Diabetes Classification with knn and Logistic Regression

mohaned mashaly — Wed, 30 Sep 2020 11:10:54 +0000

Introduction
Machine Learning has a lot of novel and great applications in the area of Health-care and can make patient diagnosis much easier and accurate taking in consideration right amount data is used in term of size and has meaningful relation to the problem.

Problem
the problem this article will cover is a classification problem to classify whether a person is a diabetic or not taking into consideration attributes like Insulin, Blood pressure, Skin-Thickness, BMI, Age, Glucose, Pregnancies, Diabetes Pedigree Function and the output should be whether a person has diabetes or not.

Approach
we will be using K-Nearest Neighbour classifier and Logistic Regression and compare the accuracy of both methods and which one fit the requirements of the problem but first let's explain what is K-Nearest Neighbour Classifier and Logistic Regression

K-Nearest Neighbour
K-Nearest is a Distance based Algorithm which means it does take distance in consideration when learning a data set, K-Nearest tries to classify which data point belongs to which class, let's say we have a finite number of data points on a graph from these finite number data points we have five data points near to each other which implies they have a-lot in common so hypothetically it's safe to consider them a class and this what K-nearest tries to achieve to classify points to a class by clustering points similar to each other as a class.

did you ever wonder what is K, it might not seem obvious that K is a variable and it changes depending on the problem but the most common value for k is between 5 and 10, what K represents is the number of data points we take in consideration when forming a class or classifying a data point for example if you chose the K to be 2 or 3 then the point you're trying to classify will look to the nearest 2 or 3 points and what class do they belong to and it will be classified to the class with the shortest distance.

Logistic Regression
Regression sounds like a whole different problem it seems like we are expecting a continuous instead of a discrete output but it's not true Logistic Regression is a classification but why are we using the word regression ?
because Logistic Regression is another scientific name for the function where the algorithm is based on which is called Sigmoid function so it's just a naming convention so in order to have a good grasp of Logistic Regression one should understand Sigmoid function first, Sigmoid function is a function with output range between 0 and 1 so it's widely used in probability predicting models as in Logistic Regression when we try to classify a class for example whether a person is diabetic or not Logistic Regression outputs the probability that a case(input data) belongs to a certain class or not based on the Sigmoid Algorithm so for example if output of a given input is less than 0.5 then he's not diabetic else he's diabetic.

Implementation
First we have to do some Feature Engineering for the dataset, since we know KNN is a distance based Algorithm and it uses distance function like Manhattan or Euclidean to calculate distance between two points we have to keep value of the attributes in control and small range to avoid wasted computational power and complexity so we are going to normalize the dataset since we have features with different range in nature after that we search for missing values 0 or NaN and sometimes the empty values is denoted as '?' also be careful some values can have 0 without any problems like, number of brothers or sisters but in case 0 doesn't make similar to our case where blood pressure in some cases we have do what is called Imputation where we substitute empty data values with relevant values using different strategies like mean or mode of the column of the missing values or by using regression to predict the missing value then we use Sklearn to traing data on knn and logistic classifier and then we present our metrics using evolution matrix and that's it

the full notebook can be found here :-
https://www.kaggle.com/mohanedmashaly/knn-classifier-vs-logistic-regression

nlp in 3 minutes

mohaned mashaly — Fri, 21 Aug 2020 22:14:32 +0000

what is NLP :

NLP is an acronym for Natural processing language which is a sub-field of AI where it tries to interpret and understand human language through different information extraction techniques, nlp aims to make interaction between computers and human more seamless to the extent that a computer can master human language and understand it, NLP has two main fields underneath it's hood which NLU(Natural Language Understanding) & NLG(Natural Language Generation), NLU tries to understand human text and speech by extracting data from different sources(Blog, Articles, etc..) while NLG tries to generate data in a certain language,imagine someone is trying to write a paragraph in a language other than his native one(computer's native language is 0's and 1's) without studying the language first, of-course he can't so does the machine or the computer so to understand human language the pipeline of NLP start with NLU actions and processes then NLG ones because s normal can't write a paragraph or summarise a text without understanding it.

what is NLTK:

NLTK is the Natural Language Tool Kit in python, it's an nlp library used to perform nlp functions in Python like Named-entity recognition or training on a corpus, POS(Part of Speech Tagging), Stemming, Lemmitizing and a-lot of different functions.

some NLP Definitions:

1.Corpus:
Corpus is a very simple term, it's a collection of words and
sentences and it's available in different languages, it covers
dialogues from movies, quotes, names, etc..., the supported
languages in NLTK i worked with were English, German, Arabic
but i believe it support more languages .

2.POS(Part of speech tagging):
Part of a speech tagging is used to identify a word in the
sentence whether it's a noun, adverb, verb, adjective, pro-noun
and many more states i was fascinated by it when i saw in nltk.

3.Lemmitizing:
Lemmitizing is a famous technique used in linguistics, it's not
restricted to nlp only actually a-lot of the techniques
presented so far is related to linguistics more than computer
science or ML since NLP is a joint between Linguistics and
Machine Learning or we can consider one of AI applications or
domains in Linguistics, what lemmitizing basically does it
convert a word to it's normal state by removing additional
characters or converting a word into adjective or a verb.

4.Stemming:
a variation of Lemmitizing, personally i don't find stemming
useful(don't understand it's use), lemmitizing is much more
better and accurate.

5.Named-entity recognition:
Named Entity Recognition is categorising words or sentences
into categories, Name, Company, City,etc... it determines
whether a word in a sentence is a person's name or a company's
or city or whatever category it belongs to , Name-entity
recognition Algorithm is a model trained on different corpus to
learn how to categorise words into entities.

6.Word Tokenization:
any-one familiar with python will find this concept fairly
simple, a tokenizer breaks the sentence into words, nltk has an
awesome feature, there's two types of tokenizers, word and
sentence where word-tokenizer breaks a paragraph into words
while sentence-tokenizer breaks a paragraph into smaller
sentences.

NLP in action:
now i write two small program to practically demonstrate how
useful nltk can be, try to guess the concepts used in the
programs.

   import nltk 
   from nltk.tokenize import word_tokenize
   Text = "I am wondering who did produce Tom and Jerry Series is 
   it Disney or Netflix?"
   tokenized_sentence= new word_tokenize(Text)
   tagged_sentence = nltk.pos_tag(Text)
   for word in tokenized_sentence: 
     print(word) 
   print(tagged_sentence)

   import nltk 
   from nltk.tokenize import word_tokenize
   Text = "Elon Musk wants to Colonize mars but i believe 
   SpaceX and even Nasa still have a long road to take "
   tokenized_sentence= word_tokenize(Text)
   tagged_sentence = nltk.pos_tag(tokenized_sentence)
   Sentence_entities = nltk.chunk.ne_chunk(tagged_sentence)

Fun to Read:
https://www.datacamp.com/community/tutorials/text-analytics-beginners-nltk
https://pythonprogramming.net/lemmatizing-nltk-tutorialhttps://www.datacamp.com/community/tutorials/text-analytics-beginners-nltk
https://medium.com/m/global-identity?redirectUrl=https%3A%2F%2Ftowardsdatascience.com%2Fnatural-language-processing-nlp-top-10-applications-to-know-b2c80bd428cb

unit-testing

mohaned mashaly — Sat, 25 Jul 2020 19:43:00 +0000

unit testing is defined as testing a function or class or a module in a program to check for any bugs before further testing is carried by the software testing team, a lot of start-ups and companies now expect developers to know unit testing methodologies and frameworks in the back-end language they are using to make the testing process done by software testers afterwards more efficient in terms of time and cost, so it's favourable for a developer/programmer to know unit teasing to ease the life of software tester and make sure his/her code works well to a certain extent, personally I tried unit testing in python, Golang in projects I was working on and somehow familiar with it in java but never used it, I will talk about pytest as a framework for testing and how it promotes a cleaner and more efficient use than the standard testing library in python, but first I will talk about some definition and jargons in pytest

@pytest.fixture : to put it simply fixture is a code which is repeated many times in the code or a variable which is used by many methods/functions i.e: database connection or creating object.

@pytest.mark.module1_test : markers is used to specify which functions you're interested in testing when starting the testing process, for example every function with module1_test signature will be tested.

let's look at a function which test a function which check wether a number is even or not

@pytest.mark.even_test
def is_even():
number1 = 2
assert number1 % 2 == 0

the test will run successfully if the number if the number1 is event else it will fails.

make sure as a developer to add unit testing on to learn list because it's a very powerful tool which can promote good coding habits

Fun to Read :-
https://yeti.co/blog/using-pytest-to-write-beautiful-tests-and-a-bulletproof-django-app/

http://guru99.com/pytest-tutorial.html#3

CMD Commands

mohaned mashaly — Tue, 12 May 2020 02:12:56 +0000

When I started Learning Development one of the Courses i Took during my enrolment in the full stack nano degree program offered by Udacity was Command Line and how to use it, Command Line is the CLI (Command Line Interface) Version of how to deal with computer, on the mac it's called Terminal but for windows it's called shell, let's say your trying to create a new Directory you have two option press right click and choose create new folder from the menu(GUI Version)or type "mkdir" command in the Terminal (CLI Version),the GUI version was developed because it doesn't make any sense for normal users to type bunch of commands in a black screen to create a folder or to view a photo or to delete a file, in this article i will show a bunch of Commands which comes in Handy.

Basic Commands :-

.Command: mkdir "FolderName"
Create a new Directory called FolderName, the mkdir abbreviation stands for (make directory )

.Command : cd FolderName
Change your current location to be on Folder, the cd abbreviation stands
for (Change Directory)

.Command : ls
List the Files inside the current Directory you're in, ls abbreviation
stands for List

.Command: pwd
output the path you're currently in, pwd abbreviation stands for print
working directory

.Command: cd ../
Redirect you to the previous directory

.Command: rm File.pdf
Remove File.pdf from the computer (you won't find it in the trash)

Extra's :

if develop in python and want to download any module or library you can
install and use pip a package installer for python

Command : pip install ModuleName

you can also code in python in the Terminal by typing python in the
Terminal if python is installed on your machine

you can use HomeBrew to install any package installer you want
Brew install Packagename

Article Hub (DEV Clone)

mohaned mashaly — Wed, 06 May 2020 22:49:43 +0000

I believe that Learning by Doing is the best learning strategy for me to learn anything, before the lockdown, I though about taking a break from Django and Try new Language where you can be more involved and creative since Django provide a minimal work for the developers or the programmer working with it to do and doesn't encourage creativity, I think that's why it's called a Framework for Deadlines so I decided up to give GoLang a shot since I read an Article about it and how it was designed, I was fascinated by it's elegance and eloquence, GoLang really is one of the cleanest language I had came across during my short span of experience as a student developer or programmer but I really love it and I think I will stick with it for some time, the latest side project I have been working on with GoLang is a Web-app which is a clone to DEV
and here are the link for the repo if anyone interested to see it :

https://github.com/12mohaned/article_hub

I am more than welcome to receive feedbacks

the project is not final yet I believe I will finish within the couple next days, I am thinking about constructing a Recommender System with machine Learning so I am not sure if I am going to do this or not

Go Gorilla

mohaned mashaly — Thu, 30 Apr 2020 17:20:45 +0000

Gorilla is a Go Routing Library used for handling requests and sessions, url routing in GoLang.

We will start with Handling Request with and without using mux(Gorilla Feature)

Handlers

Handler from it's name, it Handles the upcoming requests from the users, the naming convention for creating the function responsible for handling requests should be named to it's functionality if we are creating handler to handle user request to access Home page then it would be named HomeHandler let's go the code of creating HomeHanlder.

func HomeHandler(Response http.ResponseWriter, Request *http.Request){
template,_ := template.ParseFiles("Home.html")
template.Execute(Response,nil)
}

then we need to create the url which this function will map to and the port where the page will be hosted.

func main(){
http.HandleFunc("/Home",HomeHandler)
http.ListenAndServe(":8000",nil)
}

the above example doesn't used mux(request multiplexer) it uses function from library "net/http"

Mux in gorilla store many Handler's in it and map every one to it's url so if we want to use Gorilla we will slightly change the above code to this :-

func main(){
request := mux.NewRouter()
request.HandleFunc("/Home", HomeHandler)
http.Handle("/", request)
http.ListenAndServe(":8000",nil)
}

so what mux basically does it's store more than one route.

Sessions

Sessions is used to store information about user and retrieve this information whenever it's needed for example if I want to create authentication for Facebook so that no one can access certain pages without log in.

Sessions can be used to authenticate whether users logged in or not,
let's apply this our HomeHandler

First we are going to create cookie which will store our wanted attributes in an encrypted form
//creating a new cookie

var (
key = []byte("super-secret-key")
store = sessions.NewCookieStore(key)

)

func HomeHandler(Response http.ResponseWriter, Request *http.Request){
//create a session with the request and cookiename
session, _ := store.Get(Request, "cookie-name")
//return boolean which check whether this session is authenticated or not (true)
auth, ok := session.Values["authenticated"].(bool)
if !auth || !ok {
http.Error(Response, "You Should Log in to Access this page", http.StatusForbidden)
}
template,_ := template.ParseFiles("Home.html")
template.Execute(Response,nil)
}

so when should i set the Authentication to true when should the user have access to the Home Page (Spoiler Alert) when logging in !

So Simply when logging in or Signing up we are gonna set the session.Values["authenticated"] = true

So let's write a log in handler when the user log in

func LoginHandler(Response http.ResponseWriter, Request *http.Request){
session, _ := store.Get(Request, "cookie-name")
session.Values["authenticated"] = true
//name here is going to be the name user write when filling the log in form
session.Values["username"] = name
session.Save(Request, Response)
template,_ := template.ParseFiles("Login.html")
template.Execute(Response,nil)
}

and Voila.

you can do the same for Signup Handler when creating it.

Of-course Gorilla has more features and many of these features i am still exploring with and don't know anything about it you can check them out at Gorilla Github Repo

Algorithms part 1

mohaned mashaly — Wed, 29 Apr 2020 18:13:30 +0000

One of the main reasons why i love computer science or programming is problem solving, back to school i used to memorise everything that come within my eye sight which translated in bad grades because learning or knowledge is not a standalone process it dependent process and if you're going to memorise these prerequisite you will eventually forget them and will suffer in a more advanced levels due to lack of knowledge.

this changed in college when i got to my first computer science class and started to grasp the ideas of deduction and understanding and experimenting for example you don't have to memorise the proof just fully understand it and deduce it based on your understanding.

without going into further details this post contain list for some of my favourite Algorithms this list by no mean is Exhaustive or intended for a certain learning pattern it's just for sharing my own thoughts

Sieve of Eratosthenes:-

this Algorithm was developed by a greek mathematician.

Explanation
it's used to find the number of prime numbers for a given a range (N) between 2 and (N) with a complexity of O(Log(n)N) normally to find prime numbers within a range will requires a complexity of O(N*2)
so how this Algorithm works is that you loop over the Range Log(N) which is the first loop then in the second you will over the N the idea here is mark all non-prime numbers to true then check for false value and count them to be prime.

to do so we can get the range of number which divides i then set it to true.

let take 3 for example when i is 3 then j will be 9, 12, 15,18,etc..
do you see the pattern it will set every multiple of 3 to true so this is the general idea of the Algorithm turning every multiple of i to true(non-prime) because if a number if multiple of a number it means it can be divides by it and prime number only is divisible by itself and 1

Note :- There's a developed version of this Algorithm with a complexity of O(N) feel free to check it
** :power sign

Code :-
func Calculate_prime(n int)int {
var count int
var i,j int
var arr[1000000] bool
var number float64
number = float64(n)
number = math.Sqrt(number)
number1 := int(number)
if(n > 1000000){
fmt.Println("The Number has to be less than 1000000")
return -1;

}
for i = 2; i < number1 ; i++ {
for j = i * i; j < n; j = i + j {
arr[j] = true
}
}
for i = 0; i < n ; i++ {
if(!arr[i]){
count++
}
}
return count
}

Counting Sort

Despite this Algorithm is not widely because of it's not good complexity but the idea behind this Algorithm is both Simple and beautiful i don't know but i find it a beautiful algorithm.

Counting Sort was invented by Harold H. Seward, it's complexity is O(n+m).

Explanation

Counting Sort from it's name is a sorting algorithm, the main idea behind this Algorithm is to count the number of occurrence of a number and put every number in an array with an index similar to it's value.

so if we have an array looks like this {7,5} then the first array which map every number to an index with same value will look like this
Numbers = {0,0,0,0,1,0,1,0} if the array is {7,5,5} then it will look like this
Numbers = {0,0,0,0,2,0,1,0} where the number 2 is the occurrences of 5 in the array Numbers

Code :-

func sort(Array[1000000]int)[] int{
var indices[1000000]int
max := Array[0]
var i,j int
for i=0; i < 1000000; i++ {
if(max < Array[i]){
max = Array[i]

}
}
new_Array := Array[0:max]
for i =0 ; i < max; i++ {
if(new_Array[i] != 0){
indices[new_Array[i]]++
}
}
for i=0; i <= max; i++ {
for x := indices[i]; x > 0; x-- {
new_Array[j] = i;
indices[i]--
j++
}
}
return new_Array
}

Explanation
1) find the maximum number in the array and assign the size of array to it + 1
2) loop over the maximum number and position every element according to it's value
3) loop over the array where elements sorted based on the position

4) loop over the number of occurrences of element till it's zero
5) Elements is sorted in new_Array

Naive Pattern Searching

This Algorithm is used for searching a certain character or string or to put is simply a substring in a string.

let's say we are looking for the word "Algorithms" in sentence "I Love Algorithms"

Explanation

we are going to map every Characters to check if it does exists or not
for Example when the Algorithm start it will compare I with A then it will break but it will shift to the next index so it scans every index, if the index doesn't match it will break from the loop, the reason for loop condition to be text.length() - Pattern.length() is that in the last loop in the second loop the index of i will be equals to the text.length() so to avoid IndexoutofBound Exception.

Note :- This Code is written in java because i have more experience solving problems with java and i couldn't find a way to access character like .charAt() in java(I was Lazy)

Code:-
public void naiveSearc(String text, String Pattern){
for(int i=0 ; i <= text.length()-Pattern.length();i++){
int j ;
for(j=0 ; j < Pattern.length();j++) {
if(text.charAt(i+j) != Pattern.charAt(j)) {
break;
}

}
if (j == Pattern.length()) {
System.out.println("Pattern found at " +i);
}
}
}

Sorry for any Typos

Go Crypto

mohaned mashaly — Mon, 27 Apr 2020 18:12:04 +0000

Go has a Framework for Crypto-currency called Crypto which cover topics to build a secured blockchain, i am not a blockchain developer so i am not going to dive deep into blockchain structures i will speak from a background that i was working on the very early stages of a blockchain network in Go and stopped at Markov Tree (I don't know when I will complete this side project but i hope it will be soon), BlockChain can be understood from it's name, it's a chain of blocks connected together by a secured string(hash)like sha256, sha1 or md5 .

All the above Techniques ensure a secured immutable block but not completely secured, a-lot of techniques is used to spoil the security of the blockchain like rainbow table attack, hashing is used in blockchain to hash the pointer of the block so they reference one another.

for example if the id of the block is a string of a value equals to "I_Love_Tanya_Romanov" the equivalent hash of this string in Sha256 is "1ce39a808b22328f862d730bcc77a542798faaa4f2ebb8eed3185b8272e492fa".

the Algorithm or idea behind this conversion is technique called hashing if you're familiar with the data structure HashTable that every index in the HashTable or every row has a key and this key is generated from a hash function the hash function is measured to be good or bad by something called collision resistance which means how the hash function every time generates a unique key without collision with another hash.
(Collision means two strings can generate the same hash from one hash function)

it's impossible to find another string which will generate the same hash of the hash of the string "I_Love_Tanya_Romanov".

So HashFunction is mathematical function which takes an input a string and generates an output (hash) but it's a one way function which means if i attempted to go trace the output to find the original input i will fail and if i succeed then this is considered a weak HashFunction.

Now back to the Rainbow table how it manages to make this function a two way function, putting it simply by storing the hashes of famous passwords like 123456, i think you figured out know why some websites recommend making the password a mix of characters and number and special characters like (?,.,/, etc...) to ensure you don't use a trivial password which it's hash can be found in a database.

GoLang has a library called crypto which deals with the cryptocurrency problems like storing hash of password and this kind of problems now to it

the following method generates a hash of a string:-

func hash(s string) string{
hash := sha256.Sum256([]byte (s))
hashstring := string(hash[:])
return hashstring
}

this hash is used as a link between the block for example when linking between two blocks in a blockchain the hash is the pointer which link between the two blocks, let's imagine the Blockchain as a linked list(list or array) of blocks

Block1 <- Block2 <- Block3 <- Block4

this sign <- is the pointer which is stored as hash so this why blockchain is hard to break because even if you broke a chain the preceding ones won't be affected because Block2 can reach Block1 but Block1 can't Reach Block2 which makes blockchain a reversed linked list instead of using linking to the next node the next node linked to the previous node.

the first node (Block1) is called genesis and it doesn't reference to any other block.

Fun to Read :-

https://en.wikipedia.org/wiki/Rainbow_table
https://www.researchgate.net/publication/318131748_An_Overview_of_Blockchain_Technology_Architecture_Consensus_and_Future_Trends