DEV Community: bhat sumair

K-mean clustering and its real use-case in the security domain

bhat sumair — Mon, 19 Jul 2021 16:14:06 +0000

What is K- Means Clustering ?
We are given a data set of items, with certain features, and values for these features (like a vector). The task is to categorize those items into groups. To achieve this, we will use the k-Means algorithm; an unsupervised learning algorithm.

Document classification cluster documents in multiple categories based on tags, topics, and the content of the document. this is a very standard classification problem and k-means is a highly suitable algorithm for this purpose. the initial processing of the documents is needed to represent each document as a vector and uses term frequency to identify commonly used terms that help classify the document. the document vectors are then clustered to help identify similarity in document groups.
Identifying crime localities With data related to crimes available in specific localities in a city, the category of crime, the area of the crime, and the association between the two can give quality insight into crime-prone areas within a city or a locality.
Insurance fraud detection Machine Learning has a critical role to play in fraud detection and has numerous applications in automobile, healthcare, and insurance fraud detection. utilizing past historical data on fraudulent claims, it is possible to isolate new claims based on its proximity to clusters that indicate fraudulent patterns. Since insurance fraud can potentially have a multi-million dollar impact on a company, the ability to detect frauds is crucial.
Cyber-profiling criminals Cyber-profiling is the process of collecting data from individuals and groups to identify significant co-relations. The idea of cyber profiling is derived from criminal profiles, which provide information on the investigation division to classify the types of criminals who were at the crime scene. Thank You...

Use Case of JavaScript in today's industry.

bhat sumair — Fri, 25 Jun 2021 12:54:40 +0000

Uber and javascript
Uber needs to handle loads of data in real time. They have millions of requests coming in continuously, and that’s not just hits on a page. Uber needs to track driver locations, rider locations, and incoming ride requests. It has to seamlessly sort that data and match riders as fast as possible.

All of that plays to NodeJS’s and JavaScript’s strengths. Node is designed to handle requests and hand off data quickly. It’s asynchronous capabilities are a huge part of that. Node is central to Uber’s user facing stack for just that reason.

Uber’s core trip execution engine was originally written in Node.js because of its asynchronous primitives and simple, single-threaded processing. (In fact, we were one of the first two companies to deploy Node.js in production.) Node.js gives us the ability to manage large quantities of concurrent connections. We’ve now written many services in Go, and this number continues to increase. We like Go for its concurrency, efficiency, and type-safe operations.
The frontline API for our mobile apps consists of over 600 stateless endpoints that join together multiple services. It routes incoming requests from our mobile clients to other APIs or services. It’s all written in Node.js
Highly Available, Self-Healing, Persistent
Having to support the highest availability demands, the Marketplace stack must receive and execute in real time. Even brief interruptions in this area have major consequences for our users and our business. Much of Marketplace’s stack was built by and for Marketplace engineers first.
Ringpop, a library for building cooperative distributed systems, solved some of Marketplace’s problems before its adoption in other teams at Uber and beyond. It gives the high-availability, partition-tolerant properties of distributed databases like DynamoDB or Riak to developers at the application level.
The systems that handle pings from riders and drivers in real-time and then match them are written in Node.js and Go. These teams use Ringpop and Sevnup for cooperation and shifting of object ownership when a node in a hashring goes down, or when another node takes ownership of the keyspace. Riak is their distributed database. Redis provides caching.
Seeing and Using Data
A group within Marketplace turns Marketplace data into useful visualizations to help the team understand and observe the state of the world. We use JavaScript for web front-end applications. React+Flux, D3, and Mapbox are the libraries and frameworks behind this group. For the back end, it’s the same Node.js server used by Uber’s web engineers.

Confusion Matrix and cyber crime

bhat sumair — Sat, 05 Jun 2021 18:10:46 +0000

A confusion matrix is a table that is often used to describe the performance of a classification model (or "classifier") on a set of test data for which the true values are known. The confusion matrix itself is relatively simple to understand, but the related terminology can be confusing.

I wanted to create a "quick reference guide" for confusion matrix terminology because I couldn't find an existing resource that suited my requirements: compact in presentation, using numbers instead of arbitrary variables, and explained both in terms of formulas and sentences.

Let's start with an example confusion matrix for a binary classifier (though it can easily be extended to the case of more than two classes):
What can we learn from this matrix?

There are two possible predicted classes: "yes" and "no". If we were predicting the presence of a disease, for example, "yes" would mean they have the disease, and "no" would mean they don't have the disease.
The classifier made a total of 165 predictions (e.g., 165 patients were being tested for the presence of that disease).
Out of those 165 cases, the classifier predicted "yes" 110 times, and "no" 55 times.
In reality, 105 patients in the sample have the disease, and 60
patients do not.

cybercrime:
Cybercrime is any criminal activity that involves a computer, networked device or a network. While most cybercrimes are carried out in order to generate profit for the cybercriminals, some cybercrimes are carried out against computers or devices directly to damage or disable them, while others use computers or networks to spread malware, illegal information, images or other materials. Some cybercrimes do both -- i.e., target computers to infect them with a computer virus, which is then spread to other machines and, sometimes, entire networks.

A primary effect of cybercrime is financial; cybercrime can include many different types of profit-driven criminal activity, including ransomware attacks, email and internet fraud, and identity fraud, as well as attempts to steal financial account, credit card or other payment card information. Cybercriminals may also target an individual's private information, as well as corporate data for theft and resale. As many workers settle into remote work routines due to the pandemic, cybercrimes are expected to grow in frequency in 2021, making it especially important to protect backup data.

CYBER-SECURITY WITH CONFUSION MATRIX
Support Vector Machines (SVM) are the classifiers that were originally designed for binary c1assification. The c1assificatioin applications can solve multi-class problems. The result shows that pSVM gives more detection accuracy for classes and comparable to the false alarm rate.

Cyberattack detection is a classification problem, in which we classify the normal pattern from the abnormal pattern (attack) of the system.

The SDF is a very powerful and popular data mining algorithm for decision-making and classification problems. It has been using in many real-life applications like medical diagnosis, radar signal classification, weather prediction, credit approval, and fraud detection, etc.

A parallel Support Vector Machine (pSVM) algorithm was proposed for the detection and classification of cyber attack datasets.

The performance of the support vector machine is greatly dependent on the kernel function used by SVM. Therefore, we modified the Gaussian kernel function in a data-dependent way in order to improve the efficiency of the classifiers. The relative results of both the classifiers are also obtained to ascertain the theoretical aspects. The analysis is also taken up to show that PSVM performs better than SDF.
The classification accuracy of PSVM remarkably improve (accuracy for Normal class as well as DOS class is almost 100%) and comparable to false alarm rate and training, testing times.
Thanks for reading👍

Run GUI apps in docker container.

bhat sumair — Sun, 30 May 2021 09:31:52 +0000

Hello learners,
Today i got a new task how to run GUI applications in container.
we known that by default we can't run GUI applications in docker container because of non-availability of xserver which is required to meet this challenge.
let me show you how to overcome with this challenge:-
Follows these instructions carefully
Install docker in your os
-yum install docker-ce
start docker services
-systemctl start docker

Now pull any image you want in my case i pulled centos latest one
-docker pull centos:latest
It is already in my system you can download it from above command.

launch the docker container with following options.
-docker run -it --net=host --env="DISPLAY" centos:latest
here i pass --net=host means i want to use host connectivity and provide an --env to share docker's host display.

After successffully launch of docker container install firefox
-yum install firefox -y

Now run firefox command
-firefox
so you see it runs successfully👍

Now we can run jupyter notebook
install python
-yum install python3
install jupyter notebook
-pip3 install jupyter
After successfully installed launch jupyter notebook
-jupyter notebook
it shows some error so we have to bypass it with root permission
jupyter notebook --allow-root

Finally it is launched successfully👍

Thanks for reading.

Deploy Machine learning(ML) model in Docker container.

bhat sumair — Thu, 27 May 2021 05:50:27 +0000

First we will understand what is machine learning(ML) and Docker.
Machine learning:- Machine learning is an application/program or subset of AI that allows machines to learn from data without being programmed explicitly.
Docker:-It is a tool that provides platform to create, run and deploy the applications by using containers. Docker is a bit like a virtual machine rather than creating a whole virtual operating system, Docker allows applications to use the same Linux kernel as the system that they're running on and only requires applications be shipped with things not already running on the host computer. This gives a significant performance boost and reduces the size of the application.
Here are the complete steps:-
Step 1:
Install docker
#yum install docker-ce --nobest
if you have already installed check for query
#rpm -q docker-ce

step 2:
check status of docker
#systemctl status docker
if services are not enabled run
#systemctl start docker

step 3:
download the latest centos image from docker hub
#docker pull centos:latest

step 4:
deploy a latest centos new container
#docker run -it --name MyOs centos:latest
by passing -it arguments provides terminal and we can interact with it.

STEP 5:
Installing python3 on centos container

Step 6:
Now installing all required packages
#pip3 install numpy
#pip3 install pandas
#pip3 install sklearn

step 7:
copying dataset from base os to centos container
#docker cp /root/salary.csv MyOs:/ML

Step 8:
Code to run for ML model

Final Step:
#python3 model.py
Here is the complete output of our ML model

THANK YOU