DEV Community: Mrinal Walia

Must Try (Open-Source) Google Cloud Platform-GCP Projects on GitHub

Mrinal Walia — Thu, 02 Feb 2023 16:51:42 +0000

Google Cloud Platform (GCP) is a public cloud vendor that builds apps faster, makes smarter business decisions, and connects people anywhere. It provides cloud services for Compute, Storage, Databases, Data analytics, AI and machine learning, Networking, and Developer tools to build, scale, and solve your business problems.

Companies like Goldman Sachs and Spotify are using Google Cloud to empower its team to analyze data, scale computing power, and power the playlists of 381 million listeners every month.

Today’s article will help you get familiar with some of the best and most important GCP open-source projects. All the projects are easily accessible on GitHub, and you can play around and customize them to your requirements. (Remember to review their licensing terms before you use them in your project.)

Note: In this article, we will discuss some fantastic open-source GCP Projects/Repositories that you can utilize in your projects in 2023. To read more about each, I recommend following the link given along the project.

Affiliate Courses from DataCamp

Learning isn’t just about being more competent at your job, it is so much more than that. Datacamp allows me to learn without limits.

Datacamp allows you to take courses on your own time and learn the fundamental skills you need to transition to a successful career.

Datacamp has taught me to pick up new ideas quickly and apply them to real-world problems. While I was in my learning phase, Datacamp hooked me on everything in the courses, from the course content and TA feedback to meetup events and the professor’s Twitter feeds.

Here are some of my favorite courses I highly recommend you to learn from whenever it fits your schedule and mood. You can directly apply the concepts and skills learned from these courses to an exciting new project at work or at your university.

1. Training Data Analyst

GitHub: https://github.com/GoogleCloudPlatform/training-data-analyst

Official Documentation: https://cloud.google.com/training?hl=en

️️GitHub Stars: 6.6K

GitHub Forks: 5.2K

Languages: Jupyter Notebook89.3% , Python6.8%, JavaScript1.5% , Java0.9% , HTML0.9% , Shell0.3% , and Other0.3%

Training data analyst is a library containing labels and demos for courses on GCP training to help you grow your skills in emerging cloud technologies with Google Cloud certifications.

The repo has resources for online learning, skills development courses, and many certifications. You can subscribe to their program, which will give you access to join google online cloud events, join the google Cloud community, become a google cloud innovator and read recent training and certification blogs.

NOTE: The training and certifications are provided by Google and come under different price models. Check the details here.

2. Pulumi

GitHub: https://github.com/pulumi/pulumi

Official Documentation: https://www.pulumi.com/

GitHub Stars: 14.9K

GitHub Forks: 844

Languages: Go66.6% , Python13.0% , TypeScript12.5%, JavaScript7.2% , Shell0.4% , and Makefile0.3%

Pulumi is an open-source software infrastructure as code SDK. It helps you to create and deploy cloud software that uses containers, serverless functions, hosted services, and Infrastructure on any cloud platform, including GCP.

You can start by writing code in your favorite language and then use Pulumi to automatically provision and manage your AWS, AZURE, GCP, and Kubernetes resources using the Infrastructure as code approach.

You can start by deploying a simple application in AWS, Azure, GCP, or Kubernetes using Pumuli here.

You can download Pulumi for Windows, MacOS, and/or Linux here.

Here are a few quick resources to get started with Pulumi quickly.

3. Terraformer

GitHub: https://github.com/GoogleCloudPlatform/terraformer

Official Documentation: NIL

GitHub Stars: 9.5K

GitHub Forks: 1.3K

Languages: Go(99.1%), HCL(0.9%)

Terraformer is an open-source CLI (Command Line Interface) tool by Waze SRE that generates tf/json and tfstate files based on existing infrastructure, i.e., reverse Terraform.

This tool performs the reverse of Terraform and can be thought of as Infrastrustructure to Code (IaC) tool, and you can check a quick demo of Terraformer here.

NOTE: The software is supported on Windows, MacOS, and Linux.

4. Diagrams

GitHub: https://github.com/mingrammer/diagrams

Official Documentation: https://diagrams.mingrammer.com/

GitHub Stars: 27.7K

GitHub Forks: 1.7K

Languages: Python(94.8%), JavaScript(3.7%), and Other(1.5%)

Diagram is an open-source tool that allows you to draw the cloud system architecture in python code and prototype a new system architecture design without any design tools.

It lets you track the architecture diagram changes in any version control system and backs necessary providers, including AWS, Azure, GCP, Kubernetes, Alibaba Cloud, Oracle Cloud, etc.

The library sustains on-premise, SaaS, and noteworthy programming frameworks and languages.

NOTE: It does not control any actual cloud resources nor generate cloud formation or terraform code. It is just for drawing the cloud system architecture diagrams.

5. Online Boutique-Microservice Demo

GitHub: https://github.com/GoogleCloudPlatform/microservices-demo

Official Documentation: https://onlineboutique.dev/

GitHub Stars: 13.6K

GitHub Forks: 4.6K

Languages: Python(30.5%), Go(27.1%), HTML(9.0%), C#(7.5%), Dockerfile(5.6%), Shell(5.1%), and Other(15.2%)

The online boutique is an open-sourced microservice demo of a cloud-first application consisting of 11-tier microservices applications. It is a web-based e-commerce application where you can browse different items, add them to carts, and purchase them.

Note: This website is hosted for demo purposes only. It is not an actual shop. This is not a Google product.

Google uses this software to showcase its technologies like Kubernetes, Istio, Stackdriver, and gRPC. The project is openly available to Google employees and other people who want to learn to use GCP.

Bonus 🌟

✨ Kubernetes The Hard Way

GitHub: https://github.com/kelseyhightower/kubernetes-the-hard-way

GitHub Stars: 34.2K

GitHub Forks: 11.7K

As a bonus, I am providing a handy tutorial that will help you take the long route but will ensure you understand each task required to bootstrap a Kubernetes cluster.
Suppose you plan to support a production Kubernetes cluster and want to understand how everything fits together. In that case, this guide will walk you through bootstrapping a highly available Kubernetes cluster with end-to-end encryption between components and RBAC authentication.

If you enjoy reading this article, we intercommunicate similar interests and are/will be in similar industries. So let’s connect via LinkedIn and Github. Please do not hesitate to send a contact request!

Subscribe 📧 For Weekly Tech Nuggets! 💻

Best 5 Logistic Regression GitHub Projects for Beginners

Mrinal Walia — Wed, 18 Jan 2023 00:14:16 +0000

Logistic regression is one of the most often discussed topics for Data Science interviews.

Learn about the best 5 projects in Logistic Regression on GitHub to add to your resume and ace your following Data Science interview.

Image Source

What is Logistic Regression?

In regression analysis, logistic regression (or logit regression) estimates the parameters of a logistic model (the coefficients in the linear combination).

It is one of the most important and frequently asked topics for Data Science interviews. Logistic regression is easy to understand and is one of the widely adopted machine learning algorithms to categorize incoming data based on historical data.

Today's article will help you get familiar with some of the best and most important Logistic Regression open-source projects. All the projects are easily accessible on GitHub, and you can play around and customize them to your requirements. (Remember to review their licensing terms before you use them in your project.)

**Note:** In this article, we will discuss some fantastic open-source Logistic Regression Projects/Repositories that you can utilize in your projects in 2023. To read more about each, I recommend following the link given along the project.

Courses from DataCamp

Learning isn’t just about being more competent at your job, it is so much more than that. Datacamp allows me to learn without limits.

Datacamp allows you to take courses on your own time and learn the fundamental skills you need to transition to a successful career.

Here are some of my favourite courses I highly recommend you to learn from whenever it fits your schedule and mood. You can directly apply the concepts and skills learned from these courses to an exciting new project at work or at your university.

Coming back to the topic -

1. ytk-learn

GitHub: https://github.com/kanyun-inc/ytk-learn
Official Documentation: https://github.com/clips/pattern/wiki
GitHub Stars: 348
GitHub Forks: 81
Languages: Java(97.8%), Shell(1.7%), Other (0.5%)

Ytk-learn is an open-source distributed machine-learning library written in Java. It can help implement machine learning algorithms and run on single and multiple machines and other major distributed environments such as Hadoop and Spark. This library supports many operating systems like Linux, Windows, and Mac OS.

The different models supported by this library are as follows:

GBDT(Gradient Boosting Decision Trees)
GBRT(Gradient Boosting Regression Trees)
Mixture Logistic Regression
Gradient Boosting Soft Tree
Factorization Machines
Field-aware Factorization Machines
Logistic Regression
Softmax

There are a lot of features that ytk-learn provides:

Local file system and hdfs file system support
Easy and user-friendly codes for online production
Efficiently runs with Java SE Runtime Environment 8 without any complex installation

2. MI Ease

GitHub: https://github.com/linkedin/ml-ease
Official Documentation: https://github.com/linkedin/ml-ease#readme
GitHub Stars: 328
GitHub Forks: 79
Languages: Java(100%)

MI Ease is an open-source and large-scale machine learning library in the Alternating Direction Method of Multipliers (ADMM) based on a large-scale logistic regression algorithm. It was invented by engineers from LinkedIn and is licensed under Apache Version 2.0 with copyright 2014 LinkedIn corporation.

The ADMM or Alternating Direction Method of Multipliers considers the large-scale logistic regression model as fitting as a convex optimization problem but with a constraint that the ADMM algorithm is about to converge but minimizing the user-defined loss function enforces an extra constraint that coefficients from all partitions have to become equal to solve this optimization problem by using an iterative process.

You can start the installation by using maven for compiling, and the command is mvn clean install

3. Zen

GitHub: https://github.com/cloudml/zen
Official Documentation: https://github.com/cloudml/zen#readme
GitHub Stars: 172
GitHub Forks: 75
Languages: Scala(98.9%), Others(1.1%)

Zen is an open-source library written in scala to provide the most enormous scale and efficient machine learning platform on top of spark and in logistic regression, latent Dirichlet allocation, factorization machines, and DNN.

It is inspired by Apache Spark, MLib, and GraphX, with very sophisticated optimizations and newly added features to optimize and scale up the machine learning training process.

Zen has a vision of combining data insights, ml algorithms, and system experience to achieve a successful machine-learning platform.

4. ML Fraud Detection

GitHub: https://github.com/georgymh/ml-fraud-detection
Official Documentation: https://github.com/georgymh/ml-fraud-detection/blob/master/paper.pdf
GitHub Stars: 170
GitHub Forks: 108
Languages: Jupyter Notebook(100%)

ML Fraud Detection is a repository that implements three models trained to label anonymized credit card transactions as fraudulent or actual. The dataset is taken from Kaggle's competition Credit Card Fraud Detection and was gathered in Europe in 2 days in Sept 2013.

The three models implemented are:

Logistic Regression
K-Means Clustering
Neural Networks

In the future, the authors of this project also want to implement an autoencoder or try our hand at an SVM to see the performance. It will be beneficial to keep this repository forked and starred in the future.

5. Mlhadoop

GitHub: https://github.com/punit-naik/MLHadoop
Official Documentation: https://github.com/punit-naik/MLHadoop#readme
GitHub Stars: 52
GitHub Forks: 38
Languages: Java(100%)

MLhadoop is a GitHub repository written from scratch in Java that contains machine-learning MapReduce codes for Hadoop. All the codes are written in basic Maths, including algorithm prediction algorithms like linear and logistic regression - iterative version, clustering algorithms like k-means clustering, classification algorithms like KNN classifiers, MBA, familiar friends, etc.

The IDE used to implement the algorithms in Java is Eclipse IDE with Hadoop Development Tools (HDT) plugin installed.

Subscribe 📧 For Weekly Tech Nuggets! 💻

Best 5 Logistic Regression GitHub Projects for Beginners

Mrinal Walia — Sun, 15 Jan 2023 19:22:06 +0000

Logistic regression is one of the most often discussed topics for Data Science interviews. Learn about the best 5 projects in Logistic Regression on GitHub to add to your resume and ace your following Data Science interview.

What is Logistic Regression?

In regression analysis, logistic regression (or logit regression) estimates the parameters of a logistic model (the coefficients in the linear combination).

Note: In this article, we will discuss some fantastic open-source Logistic Regression Projects/Repositories that you can utilize in your projects in 2023. To read more about each, I recommend following the link given along the project.

Courses from DataCamp

Learning isn’t just about being more competent at your job, it is so much more than that. Datacamp allows me to learn without limits.

Datacamp allows you to take courses on your own time and learn the fundamental skills you need to transition to a successful career.

1. ytk-learn

Ytk-learn is an open-source distributed machine-learning library written in Java. It can help implement machine learning algorithms and run on single and multiple machines and other major distributed environments such as Hadoop and Spark. This library supports many operating systems like Linux, Windows, and Mac OS.

The different models supported by this library are as follows:

GBDT(Gradient Boosting Decision Trees)
GBRT(Gradient Boosting Regression Trees)
Mixture Logistic Regression
Gradient Boosting Soft Tree
Factorization Machines
Field-aware Factorization Machines
Logistic Regression
Softmax

There are a lot of features that ytk-learn provides:

Local file system and hdfs file system support
Easy and user-friendly codes for online production
Efficiently runs with Java SE Runtime Environment 8 without any complex installation

Here are some valuable resources for you to access:

Running Guide
Demo
Model Introduction
Data Format
Evaluation Metrics
Performance Guide
Online Prediction Guide

2. MI Ease

GitHub: https://github.com/linkedin/ml-ease
Official Documentation: https://github.com/linkedin/ml-ease#readme
GitHub Stars: 328
GitHub Forks: 79
Languages: Java(100%)

MI Ease is an open-source and large-scale machine learning library in the Alternating Direction Method of Multipliers (ADMM) based on a large-scale logistic regression algorithm. It was invented by engineers from LinkedIn and is licensed under Apache Version 2.0 with copyright 2014 LinkedIn corporation.

You can start the installation by using maven for compiling, and the command is _**mvn clean install**_

3. Zen

GitHub:**** https://github.com/cloudml/zen
Official Documentation: https://github.com/cloudml/zen#readme
GitHub Stars: 172
GitHub Forks: 75
Languages: Scala(98.9%), Others(1.1%)

Zen is an open-source library written in scala to provide the most enormous scale and efficient machine learning platform on top of spark and in logistic regression, latent Dirichlet allocation, factorization machines, and DNN.

It is inspired by Apache Spark, MLib, and GraphX, with very sophisticated optimizations and newly added features to optimize and scale up the machine learning training process.

Zen has a vision of combining data insights, ml algorithms, and system experience to achieve a successful machine-learning platform.

4. ML Fraud Detection

ML Fraud Detection is a repository that implements three models trained to label anonymized credit card transactions as fraudulent or actual. The dataset is taken from Kaggle's competition Credit Card Fraud Detection and was gathered in Europe in 2 days in Sept 2013.

The three models implemented are:

Logistic Regression
K-Means Clustering
Neural Networks

5. Mlhadoop

GitHub: https://github.com/punit-naik/MLHadoop
Official Documentation: https://github.com/punit-naik/MLHadoop#readme
GitHub Stars: 52
GitHub Forks: 38
Languages: Java(100%)

MLhadoop is a GitHub repository written from scratch in Java that contains machine-learning MapReduce codes for Hadoop. All the codes are written in basic Maths, including algorithm prediction algorithms like linear and logistic regression - iterative version, clustering algorithms like k-means clustering, classification algorithms like KNN classifiers, MBA, familiar friends, etc.

The IDE used to implement the algorithms in Java is Eclipse IDE with Hadoop Development Tools (HDT) plugin installed.

If you enjoy reading this article, we intercommunicate similar interests and are/will be in similar industries. So let’s connect via LinkedIn and Github. Please do not hesitate to send a contact request!
> Subscribe 📧 For Weekly Tech Nuggets! 💻

Top 5 Open-Source Sentiment Analysis Projects in Python Every NLP Engineer Should Know

Mrinal Walia — Wed, 04 Jan 2023 07:05:27 +0000

Learn about top 5 Open-Source Sentiment Analysis Projects in Python used by all Natural Language Processing(NLP) experts.

Sentiment Analysis (Also called opinion mining) is an approach to NLP that computationally identifies and categorizes mixed opinions (positive, negative, or neutral) in a piece of text. It is usually done to identify the writer's tone and attitude toward a particular topic, product, object, etc.

Businesses are applying popular sentiment analysis tools to many use cases, like social media monitoring, customer support management, and analyzing customer feedback.

Note: In this article, we will talk about some excellent open-source Sentiment Analysis Projects/Repositories that you can use in your projects in 2023. To read more about each, I recommend following the link given along the project.

Courses from DataCamp

Learning isn’t just about being more competent at your job, it is so much more than that. Datacamp allows me to learn without limits.

Datacamp allows you to take courses on your own time and learn the fundamental skills you need to transition to a successful career.

1. Sentiment Analysis in Python

2. Sentiment Analysis in R

3. Introduction to Text Analysis in R

4. Analyzing Social Media Data in R

5. Introduction to Natural Language Processing in Python

6. Introduction to Natural Language Processing in R

Coming back to the topic -

1. Pattern

GitHub: https://github.com/clips/pattern
Official Documentation: https://github.com/clips/pattern/wiki
GitHub Stars: 8.4K
GitHub Forks: 1.6K
Languages: Python (87%), JavaScript (13%)

Pattern is a web-mining module in Python for performing Data Mining, Natural Language Processing, Machine Learning, and Network Analysis tasks. The library is comprehensively tested with 350+ unit tests and comes bundled with 50+ examples under the BSD license.

Pattern supports Python 2.5+ versions and requires NumPy installed. It can be directly installed using the PyPi package in Python:

$ pip install pattern

Patterns provide support for the following applications:

Data Mining on web services like Google, Twitter, Wikipedia, Web Crawler, HTML DOM parser
NLP for sentiment analysis, n-gram search, WordNet, part-of-speech taggers
Machine Learning algorithms like Clustering, Classification, KNN, SVM, Perceptron, Vector Space Model
Network Analysis using Graph Centrality and Visualization

2. ABSA-PyTorch

GitHub: https://github.com/songyouwei/ABSA-PyTorch
GitHub Stars: 1.7K
GitHub Forks: 491
Languages: Python (100%)

Aspect-Based Sentiment Analysis (ABSA) is a sub-technique in sentiment analysis that divides text data to define its sentiments based on its aspects. This project is a PyTorch-Python implementation of ABSA.

You need PyTorch, numpy, sklearn, Python (3.6 or 3.7), and transformers to install this library. All the implemented models are listed in this folder and its recommended to try PyABSA for flexible training, inference, and aspect term extraction.

Note: This sentiment analysis project follows the all-contributors specification. Contributions of any kind are welcome!

3. StockSight

GitHub: https://github.com/shirosaidev/stocksight
Official Documentation: https://shirosaidev.github.io/stocksight/
GitHub Stars: 1.6K
GitHub Forks: 386
Languages: Python (97.9%), Shell(1.4%), Dockerfile(0.7%)

StockSight is a stock market analyzer and predictor tool that uses Elasticsearch, Twitter, News headlines, NLP, and Sentiment Analysis in Python. This library helps you find out how much emotions on Twitter and different news headlines affect a stock's price by storing and analyzing the data for stocks.

It stores Twitter and news headlines, analyzes the emotions of what the author writes, and then does sentiment analysis on the text to specify what the author feels about the stock.

You can use this project to find not just the sentiments of stocks but also the sentiments of anything.

Note: You can get support and help from the community members by joining their StockSight Slack channel.

Click here to download the StockSight software here.

4. Twitter Sentiment Analysis

GitHub: https://github.com/abdulfatir/twitter-sentiment-analysis
Project Report: https://github.com/abdulfatir/twitter-sentiment-analysis/blob/master/docs/report.pdf
GitHub Stars: 1.3K
GitHub Forks: 560
Languages: Python (100%)

Twitter sentiment analysis is a project for performing sentiment analysis on tweets using various machine learning algorithms and extracted features. This project uses various machine learning algorithms to generate a model ensemble.

The dataset is preprocessed and prepared for the training of the model. The model is trained on the below classifiers:

Naive Bayes
Maximum Entropy
Random Forest
Decision Trees
XGBoost
SVM
Multi-Layer Perceptron
Convolutional Neural Networks
Recurrent Neural Networks

Some scope to improve the model is handling emotion range, using symbols, and detecting emotions in real-time video.

5. Obsei

GitHub: https://github.com/obsei/obsei
Official Documentation: https://obsei.com/
GitHub Stars: 781
GitHub Forks: 108
Languages: Python (58.5%), Jupyter Notebook (41.1%), Other (0.4%)

Obsei is a low-code AI-powered automation tool employed in mixed business streams like social listening, AI-based alerting/notification, brand image analysis, comparative study, automatic assignment of tags, automated customer issue creation, extraction of more profound insights, and many more based upon your creativity.

Some cool features of Obsei are listed below:

- Automating tasks where cognitive thinking is required
- Collecting and dispatching data across various channels
- ETL tool for unstructured data
- Low code interface to automate mundane tasks

Obsei allows you to observe (collect unstructured data), analyze (analyze unstructured data), and inform (send analyzed data to various destinations).

If you enjoy reading this article, we intercommunicate similar interests and are/will be in similar industries. So let’s connect via LinkedIn and Github. Please do not hesitate to send a contact request!

Subscribe 📧 For Weekly Tech Nuggets! 💻

Top 5 Open-Source XGBoost Algorithm Projects to Study in 2023

Mrinal Walia — Tue, 27 Dec 2022 19:08:41 +0000

This article will teach you the top 5 open-source XGBoost Algorithms and Repositories on GitHub.

Source

The XGBoost (also known as eXtreme Gradient Boosting) is a popular supervised learning algorithm and an efficient open-source implementation of the gradient-boosted decision trees (GBDT) algorithms. Conversely, Boosting means combining a set of weak and robust learners to reduce training errors.

There are many classes of boosting algorithms, but today we will talk about XGBoost Algorithm.

Note: In this article, we will talk about some excellent open-source XGBoost projects/Repositories that you can use in your projects in 2023. To read more about each, I recommend following the link given along the project.

Affiliate Courses from DataCamp

Learning isn’t just about being more competent at your job, it is so much more than that. Datacamp allows me to learn without limits.

Datacamp allows you to take courses on your own time and learn the fundamental skills you need to transition to a successful career.

Coming back to the topic -

1. xgboost

GitHub: https://github.com/dmlc/xgboost

Official Document: https://xgboost.ai/

Stars: 23.6K

Forks: 8.5K

The xgboost is a scalable, portable, and distributed gradient-boosting decision tree library for different programming languages, including Python, R, Java, Scala, C++, etc. It is compatible with Big data platforms such as Hadoop, DataFlow, Spark, Flink, Dask, etc.

This is the official open-source repository of xgboost that has been developed and used by a group of active community members across the globe.

The XGBoost algorithm is an optimized gradient boosting algorithm from machine learning algorithms under the Gradient Boosting Framework, and it is designed to be:

Highly Efficient
Highly Flexible
Highly Portable
Supports Multiple Languages
Distributed on Cloud
High Performance

Note: Your contribution is precious to making the library better for everyone. If you can help them in any form, please check out their Community Page here.

2. deepdetect

GitHub: https://github.com/jolibrain/deepdetect

Official Document: https://www.deepdetect.com/

Stars: 2.4K

Forks: 556

The deepdetect is a deep learning API and server written in C++14 language. The XGBoost algorithm and other popular frameworks such as Caffe, PyTorch, Tensorflow, DLib, TSNE, NCNN, and TensorRT support it.

The deepdetect library supports a web platform for training and managing your machine-learning models. The authors envision making deep learning easy and straightforward to work with. Supporting other backend libraries, including XGBoost, secures them a spot on this list.

Source

Some cool features of this library are:

Easy to Setup
Ready for different applications
Support for Web UI
Fast server
Come with Different Neural Network Templates
Trains in a few hours and with small datasets
Comes with ready-to-use models for multiple tasks
Fully open-source ecosystem

3. AlphaPy

GitHub:https://github.com/ScottfreeLLC/AlphaPy

Official Document: https://alphapy.readthedocs.io/en/latest/

Stars: 827

Forks: 167

Source

AlphaPy is an automated machine-learning library with support for Python, Scikit-Learn, Keras, XGBoost, LightGBM, and CatBoost libraries and algorithms.

The library is written in Python and runs machine-learning models using the Scikit-learn, Keras, XGBoost, LightGBM, and CatBoost algorithms.

AlphaPy allows you to do the following tasks:

Generate blended and stacked ensemble models
Design models for investigating the markets with MarketFlow
Predict sporting events with SportFlow
Develop trading systems and portfolios using pyfolio

4. XGBoostLSS

GitHub: https://github.com/StatMixedML/XGBoostLSS

Stars: 275

Forks: 36

Source

The XGBoostLSS is an extension of the XGBoost framework. It helps predict the entire conditional distribution of univariate and multivariate responses.

XGBoostLSS has support for the following:

Multi-Target Regression
Estimation of the Gamma Distribution
Full Predictive Distribution via Expectile Regression
Automatic Derivation of Gradients & Hessians
Pruning During Hyperparameter optimization

The library is written in Python and estimates all distributional parameters simultaneously. The support for multi-target regression allows the multivariate response and its dependencies to be modeled. To understand the output of XGBoostLSS, you must learn SHapley Additive exPlanations, and you can install the package in Python using the below command:

$ pip install git+https://github.com/StatMixedML/XGBoostLSS.git

5. XGBoost.jl

GitHub: https://github.com/dmlc/XGBoost.jl

Official Documentation: https://dmlc.github.io/XGBoost.jl/dev/

Stars: 244

Forks: 113

Source

XGBoost.jl is a Julia interface of the popular XGBoost algorithm. The package is efficient and can be more than ten times quicker than some current gradient-boosting packages. The users can define their objectives by making the library extensible.

The package uses xgboost_jll to package the xgboost binaries and is the Julia wrapper of the xgboost gradient boosting library.

BONUS

As a bonus, I am adding a link to a curated list of gradient-boosting research papers (until 2022 Oct) with their implementations.

Awesome Gradient Boosting Papers: GitHub (835 Stars & 147 Forks)

Subscribe to my #MEDIUM 📧 For Weekly Tech Nuggets! 💻

Top 5 Open-Source Projects In LSTM Neural Networks To Know in 2023

Mrinal Walia — Sat, 17 Dec 2022 19:00:54 +0000

Long-Short-term memory networks are artificial neural networks capable of learning other dependencies in time series or sequence predictions.

What is LSTM? You might have heard this term in the last interview you gave for a Machine Learning Engineer position, or some of your friends might have mentioned using LSTM in their predictive modeling projects. So the big question that may arise here is what LSTM is, what sort of projects can be created using the LSTM algorithm, etc. Do not worry. This article will cover the top 5 open-source projects in LSTM Neural Networks that everyone should know about in 2023.

Quick Fact: Microsoft uses LSTM models to improve its speech recognition software, selfies, search engine, learn to code, and more.

In today's article, we will talk about five open-source LSTM Neural Network projects/ repositories on GitHub to help you enhance your skills to prepare for the Data Science job market.

Note: In this article, we will talk about some excellent open-source LSTM neural network projects/Repositories that you can use in your projects in 2023. To read more about each, I recommend following the link given along the project.

1. EasyOCR

GitHub: https://github.com/JaidedAI/EasyOCR

Official Document: https://www.jaided.ai/

Stars: 16.5K

Forks: 2.4K

Easy OCR is a ready-to-use optical character recognition tool that supports 80+ languages, including their popular writing scripts. The languages supported include Latin, Chinese, Arabic, Devanagari, Cyrillic, etc.

The library can be installed using, pip install easyocr and you can perform two tasks for development work. For extending the recognition model, read here, and for the detection mode (CRAFT), read here.

You can take a demo on their official website or use the link here.

The two main components of this project are:

It can be operated for handwritten support.
The code can be restructured to support swappable detection and recognition algorithms.

2. Stock Prediction Models

GitHub: https://github.com/huseinzol05/Stock-Prediction-Models

Stars: 5.7K

Forks: 2.2K

Stock Prediction models is a project comparison of machine learning and deep learning models that are majorly used for applying stock price forecasting. The project is also used for training trading bots and real-time simulation platforms.

There is a diverse list of LSTM models used in this project, some of which:

LSTM
LSTM Bidirectional
LSTM 2-Path
LSTM Seq2seq
LSTM Bidirectional Seq2seq
LSTM Seq2seq VAE

A standard jupyter notebook file allows you to forecast using any of the models mentioned above or including other models in the list.

The file for how-to-forecast can be found on this link.

3. LSTM Human Activity Recognition

GitHub: https://github.com/guillaume-chevalier/LSTM-Human-Activity-Recognition

Stars: 3.1K

Forks: 915

Activity Recognition Experiment Using Smartphone

LSTM human activity recognition implements real-time human activity detection using TensorFlow and an LSTM RNN network on a smartphone sensor dataset.

The type of movement is classified into six diverse categories. They are:

Walking
Walking Upstairs
Walking downstairs
Sitting
Standing
Laying

The proposed method in the project uses the state-of-the-art LSTM model with an RNN model to avoid the feature engineering process, which generally requires a significant amount of time.

4. Automatic Speech recognition

GitHub: https://github.com/zzw922cn/Automatic_Speech_Recognition

Stars: 2.8K

Forks: 549

Automatic speech recognition is an end-to-end project for automatic speech identification for the English & Mandarin (Chinese) language. Some examples of generated speech predictions are:

Label:
it was about noon when captain waverley entered the straggling village or rather hamlet of tully veolan close to which was situated the mansion of the proprietor
Prediction:
it was about noon when captain wavraly entered the stragling bilagor of rather hamlent of tulevallon close to which wi situated the mantion of the propriater
Label:
one who writes of such an era labours under a troublesome disadvantage
Prediction:
one how rights of such an er a labours onder a troubles hom disadvantage

This is one of the finest projects for automatic speech recognition, as it is implemented in Tensorflow and supports training with CPU/GPU. It also gives you a choice to pick your network to train your model:

RNN
BRNN
LSTM
BLSTM
GRU
etc.

5. LSTM

GitHub: https://github.com/nicodjimenez/lstm

Official Document: https://arxiv.org/abs/1506.00019

Stars: 1.5K

Forks: 645

LSTM GITHUB REPOSITORY

LSTM repository is a basic implementation of the LSTM network in the python programming language in a few hundred lines of code from scratch. This repository is intended to keep the lstm concept simple by providing clean examples of training neural networks in lstm in python.

You can use this valuable yet helpful resource to learn how to implement an lstm network with minimum effort. You can extend your knowledge by adding more functional abilities and features and trying your LSTM model on different datasets.

The full article explaining the concept can be found here.

Follow the link to a detailed and practical blog on how to use this simple project.

Subscribe to my MEDIUM 📧 For Weekly Tech Nuggets! 💻

Five Things You Should Know About the M2 Chip by Apple

Mrinal Walia — Fri, 22 Jul 2022 15:45:39 +0000

Apple has announced the release of their newest chip, the M2. It is anticipated to be released in July 2022 as part of the new 13-inch MacBook Pro and MacBook Air. The M2 chip is a big step from its predecessor, the M1 chip. Apple-designed it to make the laptops more capable than ever and the most portable they've ever been.

This post will highlight five things about the M2 chip, from power consumption to security.

1. The M2 Chip Is Faster Than the Previous Chip
The M2 chip is up to 1.6x faster than the M1 model and 6x faster than the Intel-based models, making it an excellent option for people looking for a fast and efficient laptop. This is important for people who use their computers for gaming or graphic design work. Now you can see most graphic-intensive games at a smooth 120 FPS and work with more streams of 4K and 8K ProRes video with the high-performance.

2. The M2 Chip Is More Energy Efficient Than the Previous Chip
The M2 chip is made more power-efficient by optimizing the motherboard through low power consumption and thermal management. Because of this, you can work or play for longer without worrying about your battery life. Also, the battery on a single full charge will last longer. You won't have to worry about your laptop overheating, making your MacBook more energy-efficient overall.

3. The M2 Chip Is Smaller and More Powerful
The M2 chip is smaller and more powerful than its predecessor, the M1 chip. It has a 64-bit architecture and a PCIe-based SSD controller. It's designed to have four high-performance cores and four high-efficiency cores to tackle power-intensive single-threaded and multi-threaded performance tasks. The 35% more powerful GPU and 40% faster Neural Engine make M2 faster and more efficient than the original chip.

4. The M2 Chip Is More Secure
Another thing worth mentioning about the M2 chip is that it's more secure than its predecessor. It has a dedicated security processor that helps protect your data and passwords and keeps your MacBook safe from attacks. The A12 Bionic Chip in the iPhone XS and XR has a Secure Enclave coprocessor that provides hardware-level security for Face ID, Touch ID, and other security features. The M2 Chip in the MacBook Pro and MacBook Air also includes a Secure Enclave coprocessor, so your data is protected with the same level of security.

5. The M2 Chip Makes the MacBook Pro and MacBook Air More Capable Than Ever
The M2 chip is the newest chip released by Apple. It makes the 13-inch MacBook Pro and MacBook Air more potent than ever. With the 8-core CPU and 10-core GPU, the laptops can handle more 4K and 8K ProRes video streams and more graphics-intensive tasks. Additionally, they have up to 24GB of faster-unified memory. M2 works in sync with the macOS to bring more speed and responsiveness to all your favourite iOS apps. This chip supports up to 2TB of SSD storage and 40GB/s of data transfer speed on the go.

Conclusion

The M2 chip is a new addition to Apple's lineup of chips. The redesigned MacBook Air and Pro are the first Apple products to have the M2 chip. The company is yet to announce its M2 Pro and M2 Max chips for its professional and focused Mac users shortly. Nevertheless, Apple's M2 chips are another significant step toward technological advancements. If you're in the market for a new Apple laptop, check out models that feature the M2 chip. You won't be disappointed!

> If you liked this post, I believe we have similar interests and let's connect to keep ourselves engaged with the amazing community out there. Send me a HI! 👋 on LinkedIn or Github and I will reach out to you.

Do You Need To Know How To Code To Become A Data Science Professional?

Mrinal Walia — Tue, 19 Apr 2022 04:24:44 +0000

So you've decided that you want to pursue a career in data science. Or maybe you're already in the field but want to make a switch. We're guessing that, like many others, you aspire to wield the power of data to change the world. And you're on the right track!

Data science is one of the considerable in-demand and high-paying professions today. But what if we told you that you don't need to know how to code to become a data scientist?

In this article, we will discuss what coding languages are most important for data science, must have data science skills, a few essential coding questions asked in interviews, and last, we will answer the most common doubt every data scientist has, do you need to learn how to code to become a data scientist?

What is data science?

Data science is a relatively new professional domain that sits at the intersection of statistics, computer science, and business. There is no one-size-fits-all explanation to this inquiry, as the required skill set for data science roles can differ depending on the company and position.

Data science is all about extracting knowledge and insights from data to solve real-world problems. In other words, data scientists use data to answer questions and solve problems.

A data scientist extracts insights from data and then communicates these insights to others in a way that is easy to understand. This often involves working with programming languages such as Python or R, but it is not necessarily required to know how to code to be a data scientist.

Many data science roles also require expertise in statistics and mathematics and solid business skills. As a data science professional, you will be responsible for cleaning and preparing data, building models to analyze data, and interpreting the results.

What coding languages are most important for data science?

According to data from indeed.com, the coding language you will need to learn will depend on the specialization you enter as a data science professional. Here are three of the most essential coding languages for data science careers:

1. SQL: Structured Query Language is used to manage and query databases. As a data scientist, you'll likely use SQL to access and analyze data sets.
2. Python: Python is a universal language used in many different areas of data science, from machine learning to scientific computing.
3. R: R is a robust programming language for statistical analysis and graphical processing. It's used by many data scientists for data mining and modelling.

However, it's crucial to note that these are not the only coding languages you need to know. In fact, many data science professionals use a variety of coding languages to suit their needs.

One data scientist might prefer Python for its readability and wide range of libraries, while another might prefer R for its statistical capabilities. The necessary thing is not to get bogged down in any language and instead focus on becoming proficient in all the coding languages.

Learning different programming languages will allow you to explore and analyze data on your own, and it'll also help you understand the data science process better. It never hurts to be well-rounded and know a few coding languages!

Do you need to learn how to code to become a data scientist?

So the big question is: do you need to know how to code to become a data scientist? The answer is no. You don't need to understand how to code to become a data scientist. However, coding skills are definitely beneficial.

You don't need to be an experienced coder to enter the data science field. It allows you to work with the data directly, rather than relying on someone else to help you. In addition, coding gives you a better understanding of your results and how to improve them.

Other essential skills for data science include statistics, math and machine learning. With these skills, you'll be able to analyze data, find trends and make predictions.

- Math skills: You need to be able to understand and work with complex equations, as well as be able to analyze data to find trends.
- Statistical knowledge: Data science is all about analyzing data and finding patterns. Understanding statistics will help you do this more effectively.
- Critical thinking: To solve complex data problems, you need to be able to think critically and come up with creative solutions.
- Communication skills: Data science is a team sport. You'll need to communicate your findings effectively to other members of your team.

How To Learn To Code For Data Science

Data science positions often require coding knowledge. If you struggle to learn to code for data science projects and positions, this section will present the best practices and resources to help you start coding for data science.

1. Reading Articles: Articles are the best resources available on the internet today to help you get started with learning to code. It takes significantly less time, and you can choose from a pool of articles on the same topic. If you want to dive deep into some concept or plan to learn about a tool, AnalyticsVidhya is the best place for every aspiring data scientist. It has a categorized section of thousands of articles covering all data science concepts. Once you have gained a foundation of the concepts, you can even share your knowledge with others on their platform, and they pay you a good amount of money for each published article. So start reading and sharing your knowledge today.
2. Hackathons: Hackathons are a great place to learn to code. It is a competition where you have to complete software in a short period. It allows you to open your horizons and explore with different skills and tools. AnalyticsVidhya hackathons are a great place to start if you want to take place in an exciting and full of the awards event. Plus, it will help you learn something new.
3. Courses: If you want to learn coding for data science for some particular project or out of your own interest, then taking a course on that language is a great way to learn it. AnalyticsVidhyas Courses are a great resource to start with as they future-proof your career in data science and learn from industry experts with one-on-one mentorship.

Conclusion

The answer, "Do you need to know how to code to become a data scientist?" is complicated. There is no one-size-fits-all answer to whether or not you need to learn how to code to become a data scientist. However, coding languages are an essential part of data science, and learning how to code can build you a robust foundation for a career in data science as it is certainly a valuable skill.

Many coding languages are essential for data science, but Python and R are two of the most common. In addition to coding, it's essential to have strong math, statistics knowledge, and problem-solving skills. However, some companies may require coding knowledge, so it is essential to do your research before applying for jobs in data science.

Thanks for reading my article, and have a good day :)

About Author

I am a Data Scientist with a Bachelors's degree in computer science specializing in Machine Learning, Artificial Intelligence, and Computer Vision. Mrinal is also a freelance blogger, author, and geek with five years of experience in his work. With a background working through most areas of computer science, I am currently pursuing Masters in Applied Computing with a specialization in AI from the University of Windsor, and I am a Freelance content writer and content analyst.
Connect with me on my social media profiles and follow me for a quick virtual cup of coffee.

Anomaly Detection for Industrial IoT Devices

Mrinal Walia — Sun, 06 Feb 2022 05:36:29 +0000

An anomaly, described as any change in usual behavior, seriously affects industrial products' production in Industrial IoT (IIoT).

Anomalies in an IoT sensor's time-series data can imply a failure in a manufacturing unit; hence accurately and opportunely detecting anomalies is becoming increasingly crucial.

This blog will discuss anomaly detection in IoT, challenges faced because of anomaly detection, and possible anomaly detection techniques.

##What is anomaly detection in IoT?
The industrial internet of things (IoT) consists of various intelligent devices adept at data collection, storage, processing, communication, and widespread deployment of edge devices in this paradigm has spawned a variety of emerging applications with edge computing.

For instance, intelligent manufacturing, intelligent transportation, and smart logistics provide powerful computation resources to facilitate real-time, flexible, and quick decision-making for Industrial IoT device examples.

However, the IIoT applications are suffering from critical security risks, and there are several threats and vulnerabilities as emerging protocols—for example, engines with sensors that have abnormal behaviors or abnormal traffic and varying reporting frequency.

An anomaly is a sequence of patterns in IoT networks that significantly deviate from standard behavior and typically collect sensing data from IIoT nodes, especially time-series data, to interpret and capture the behaviors and functional needs of IIoT nodes by edge computing.

##Challenges of Anomaly Detection in the Industrial IoT
The detection time of anomaly detection schemes in the Industrial IoT environment is challenging due to multiple factors such as:

###1. Lack of IoT Resources
There are a lot of data collection methods that perform well. Still, industrial anomaly detection can be restrained by the limitations in storage, processing, communication, and power resources. Furthermore, analytics must be accomplished in real-time and adapted to the fast pace. Hence, companies need dedicated resources.

###2. Dimensionality of Data
Industrial internet of things data for time-series anomaly detection can be univariate as key-value or multivariate as temporally correlated univariate. Determining a specific anomaly detection mechanism in IoT applications hinges on data dimensionality. Multivariate data raises the complexity of processing models, whereas univariate data may not designate finding patterns and correlations that improve machine learning models' performance.

###3. Profiling Expected Behaviour
With some firms gathering over 250 petabytes of data per day, anomalous behaviors might be collected within normal behaviors. With a shortage of datasets defining both IoT normal and abnormal data, structured information becomes a real challenge.

##What are the possible anomaly detection techniques?
Conventional anomaly detection techniques cant keep pace with the extreme data volumes and velocity of today's industrial IoT devices; hence industrial anomaly detection in real-time is becoming more and more difficult.

Utilizing Deep Anomaly Detection (DAD) can help detect abnormal behaviors of IIoT devices by investigating sensing time-series data and learning hierarchical discriminative features from historical time-series data.

A more automated approach is to detect anomalies by configuring static threshold-based alerts. Although this is an improved method still has some demerits like proper domain knowledge is required for the admin to set the appropriate threshold, and static signals are challenging to maintain in ever-changing environments.

Another technique is edge computing which provides a more practical method to leverage ML-based anomaly detection. You can reduce workloads on the cloud by redirecting crucial data processing workloads closer to the data source (IoT devices). An anomaly detection example using edge computing is to monitor machine health in real-time, which otherwise might indicate a failure in a system.

##Conclusion
Anomaly detection is excellently positioned to guard the Industrial IoT network, and it is a crucial tool to identify and alert abnormal activities in the system. The motive behind today's blog was to present a standard survey of anomaly detection for Industrial IoT devices and help you understand the challenges and possible methods available for anomaly detection.

Challenges faced by open source projects in 2022

Mrinal Walia — Sat, 04 Dec 2021 07:38:46 +0000

Open-source community platforms have become so successful not primarily for cost reasons but because of decentralization, speedy development cycles, and availability to everyone. Many people rely on open-source software in one way or another, and numerous people see the benefits of open source from a distinct viewpoint.

However, open-source has had a more profound impression on people, culture, and our society. Open-source software and monetization dynamics have changed drastically, transforming how products and services are developed and delivered.

Today's article will discuss some of the significant problems with the modern trend in open-source development and present you with solutions to mitigate them. Let's look into some basics of open-source software before we get started.

What is Open-Source?

Open Source software, for most of us, is free software to just save a few bucks. People just Google for something like "free photo editing software" and are glad to get something free suited to their needs. But Open Source is much more than that. Open source is just not free software. It is the software with which you have the freedom to change the source code of the Open Source software. This is the greatness of the Open Source Movement. The Open Source movement is about letting the software evolve, grow with people from all over the world contributing to the software rather than keeping the code restricted to a few developers.

For some of us, it would sound alien. Why would anyone work on something and then give it away for FREE? Whether to make the source code open for a software definitely lies with the software developer, but that is essentially a secondary issue. The most important fact is that Open Source is not only free. It's the freedom to evolve and to create.

Read complete article on Aviyel

Aviyel | Challenges faced by open source projects in 2022

Datacamp: A Datascience Portal

Mrinal Walia — Mon, 29 Nov 2021 20:11:11 +0000

#DataCamp -- Black Friday Sale is here!

Save 75% on an annual Premium plan! & Get your first month at just $1.

Are you ready to future-proof your career with data skills—from non-coding essentials to advanced data science.

DataCamp offers unlimited access to the entire catalog, including 360+ courses, 60+ tracks, and 90+ projects. Also, get certified and access career services.

DataCamp has interactive content especially designed by experts, and you can apply your new data skills in the workspace to analyze real datasets, share your insights, and build your data science portfolio.

You can either opt for individual courses or track courses with a set of methods inside them. I prefer track courses because they have all the relevant systems inside them. So I insist the readers go ahead and check them out.

Also, if you are short of Project ideas to develop and apply your skills, DataCamp has impressive Projects. I love doing these, and they are my favorite.

Colour Quantization Using K-Means Clustering and OpenCV

Mrinal Walia — Mon, 19 Jul 2021 13:47:34 +0000

Hi Connections,

I recently wrote an article on "Colour Quantization Using K-Means Clustering and OpenCV" which got published on @Analytics Vidhya!

It would be really great if you could take a few minutes to take a look at it.

Hope you guys will find it useful.

analyticsvidhya #datascience #analytics #machinelearning #algorithms #ai #deeplearning #artificialintelligence #dataanalytics #technology #colorgrading #opencv #opensource #python #kmeans #clustering

https://www.analyticsvidhya.com/blog/2021/07/colour-quantization-using-k-means-clustering-and-opencv/

DEV Community: Mrinal Walia

Must Try (Open-Source) Google Cloud Platform-GCP Projects on GitHub

Affiliate Courses from DataCamp

1. Training Data Analyst

2. Pulumi

3. Terraformer

4. Diagrams

5. Online Boutique-Microservice Demo

Bonus 🌟

✨ Kubernetes The Hard Way

Best 5 Logistic Regression GitHub Projects for Beginners

What is Logistic Regression?

Courses from DataCamp

1. ytk-learn

2. MI Ease

3. Zen

4. ML Fraud Detection

5. Mlhadoop

Best 5 Logistic Regression GitHub Projects for Beginners

What is Logistic Regression?

Courses from DataCamp

1. ytk-learn

2. MI Ease

3. Zen

4. ML Fraud Detection

5. Mlhadoop

Top 5 Open-Source Sentiment Analysis Projects in Python Every NLP Engineer Should Know

Courses from DataCamp

1. Pattern

2. ABSA-PyTorch

3. StockSight

4. Twitter Sentiment Analysis

5. Obsei

Top 5 Open-Source XGBoost Algorithm Projects to Study in 2023

Affiliate Courses from DataCamp

1. xgboost

2. deepdetect

3. AlphaPy

4. XGBoostLSS

5. XGBoost.jl

BONUS

Top 5 Open-Source Projects In LSTM Neural Networks To Know in 2023

Other Links (Affiliate)

1. EasyOCR

GitHub: https://github.com/JaidedAI/EasyOCR

Official Document: https://www.jaided.ai/

Stars: 16.5K

Forks: 2.4K

2. Stock Prediction Models

GitHub: https://github.com/huseinzol05/Stock-Prediction-Models

Stars: 5.7K

Forks: 2.2K

3. LSTM Human Activity Recognition

GitHub: https://github.com/guillaume-chevalier/LSTM-Human-Activity-Recognition

Stars: 3.1K

Forks: 915

4. Automatic Speech recognition

GitHub: https://github.com/zzw922cn/Automatic_Speech_Recognition

Stars: 2.8K

Forks: 549

5. LSTM

GitHub: https://github.com/nicodjimenez/lstm

Official Document: https://arxiv.org/abs/1506.00019

Stars: 1.5K

Forks: 645

Five Things You Should Know About the M2 Chip by Apple

This post will highlight five things about the M2 chip, from power consumption to security.

Conclusion

Do You Need To Know How To Code To Become A Data Science Professional?

What is data science?

What coding languages are most important for data science?

Do you need to learn how to code to become a data scientist?

How To Learn To Code For Data Science

Conclusion

About Author

Anomaly Detection for Industrial IoT Devices

Challenges faced by open source projects in 2022

What is Open-Source?

Read complete article on Aviyel

Datacamp: A Datascience Portal