DEV Community: Nikhil Dhawan

Machine Learning - Regression- Simple Linear Regression

Nikhil Dhawan — Mon, 08 Jan 2024 08:23:06 +0000

Hi all in this post we will see how we can train Simple Linear regression model. After we are done with preprocessing of data we can direct train our model with help of LinearRegression from sklearn linearmodel. First we train and then predict the testing values that will help to see how good our model is. To train we use fir method on LinearReression and using predict method on the same for testing data.
Before we move to code, let me tell you some BTS for SLR, our data is of shape as below:

so here we have salary for the years of experience and we have to make a mode to be able to predict salary in future based on years we input in model. SLR is usually depicted as:

yh= bo + b1X1

where yh is dependent variable, bo is y-intercept and is constant, b1 is slope co-officient and X1 is independent variable

Model tries to find best slope line such that we have ordinary least squares (sum of (y1-yh)^2 is minimized)

Let's move to code part

Training the model

from sklearn.linear_model import LinearRegression
regressor = LinearRegression()
regressor.fit(X_train, y_train)

Predicting the test results

y_pred = regressor.predict(X_test)

Lets now visualize the plot for training set, for that we use pyplot from matplotlib

plt.scatter(X_train, y_train, color = 'red')
plt.plot(X_train, regressor.predict(X_train), color = 'blue')
plt.title('Salary vs Experience (Training set)')
plt.xlabel('Years of Experience')
plt.ylabel('Salary')
plt.show()

Plot for Test set

plt.scatter(X_test, y_test, color = 'red')
plt.plot(X_train, regressor.predict(X_train), color = 'blue')
plt.title('Salary vs Experience (Test set)')
plt.xlabel('Years of Experience')
plt.ylabel('Salary')
plt.show()

Here if we see closely we are using plot line same for both , that's because the slope line will be same for both as the model we used will use same predictor while predicting values for unknown values, even if you change that to X_test it should be same. Red dots in second graph shows actual values we have in data and values along line will be predicted values, and we can see most of the points are overlapping but some are not. No model can be 100% fitting.

Thanks for reading, hope it was useful ,see you next time

Machine Learning - Data Preprocessing-2 label encoder, Splitting Data and Feature scaling

Nikhil Dhawan — Sun, 07 Jan 2024 07:31:03 +0000

H all, So in this article we will discuss about label encoder, splitting data set and then feature scaling it.
So let's start with label encoder these are basically used for encoding dependent variable which is Y here. so in data set we have values like yes or no , that will be encoded to numeric values of 0 or 1. We will use LablelEncoder from sklearn preprocessing and then transform the data as below

from sklearn.preprocessing import LabelEncoder le = LabelEncoder() y = le.fit_transform(y)

After we are done with this we can now split the data into Training and Test sets. Generally we keep 80% of data as training and rest as test set. It is useful for evaluating model performance once we are done training the model. This can be done via train_test_split from sklearn model_selection

from sklearn.model_selection import train_test_split X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 1)

Once we have got 2 sets of data if our data needs to Feature scaling we will perform that using Standard Scaler from sklearn preprocessing . Here we use mostly Standardization method for Feature Scaling, there is also another method which is called normalization but mostly standardization is used. We will first scale training data and then using same scaling parameters test set will be scaled. It is recommended and general practice to scale after data splitting as test data should be unseen data to the model and if we split after scaling it will have data bias and our model might not perform in the way we intend.

from sklearn.preprocessing import StandardScaler sc = StandardScaler() X_train = sc.fit_transform(X_train) X_test = sc.transform(X_test)

Now we have transformed data after preprocessing and ready to develop our models to predict values. We will cover it in future articles.

Hope it helped.

Machine Learning - Data Preprocessing- 1

Nikhil Dhawan — Sat, 06 Jan 2024 07:04:55 +0000

Hi all, in this article we will discuss about the first step to building our model - Data Preprocessing.

Importing Libraries - In this we import most common used libraries like pandas, numpy . There might be others also but for this example lets keep it as it

import numpy as np import pandas as pd

Importing Datasets - for this example we have a dataset that is as below containing data if a purchase was made or not from a specific person having particular identity

In this data we can see first 3 columns are features and last is the dependent variable . We mostly divide our data using this only, with X as the inputs and y as the output and are loaded using pandas and then divided using iloc operation on dataset

dataset = pd.read_csv('Data.csv') X = dataset.iloc[:, :-1].values y = dataset.iloc[:, -1].values

So now we have our data separated and ready for further handling.

Handling missing data- So if you see the data we have 2 missing data (marked in yellow)

Most broadly used library for data science operations is sklearn. So in this case also we will use it only. Strategy for this will be replacing the empty data with average value from the column and to use it SimpleImputer from sklearn impute is used

from sklearn.impute import SimpleImputer imputer = SimpleImputer(missing_values=np.nan, strategy='mean') imputer.fit(X[:, 1:3]) X[:, 1:3] = imputer.transform(X[:, 1:3])

and it will give us

One more thing to take care here is first column that has text/string values, as models might not be able to interpret these correctly we also need to encode this to digits to be able to feed them to model.
For this ColumnTransformer is used from sklearn compose and OneHotEncoder from sklearn preprocessing

from sklearn.compose import ColumnTransformer from sklearn.preprocessing import OneHotEncoder ct = ColumnTransformer(transformers=[('encoder', OneHotEncoder(), [0])], remainder='passthrough') X = np.array(ct.fit_transform(X))

In this ColumnTransformer takes input as transformers that is tupple which has name of transformer, transformer itself and column to transform and remainder is to specify what to do for other columns, which here is passthrough which mean no change in other columns.
and our output will be

Hope it was helpful. In this next part of preprocessing we will see how to encode labels, Feature scaling and spliting data into training and test set.

Machine Learning - The start

Nikhil Dhawan — Fri, 05 Jan 2024 08:09:46 +0000

Hi all, with article i start with learning process of machine learning and AI from very beginning. Lets start from The Machine learning process:

Data Pre-processing - This is the starting phase of almost all the new models we build. it mostly has the below common processes.

Importing the data
Cleaning the data -> In this we take care of various things like missing data, need of normalization or standardization based on our requirement, encoding categories or labels.
Splitting the data into training and test sets -> Generally we split our data into 2 sets in which 80% is kept as training set on which our model is trained and 20% is test set on which performance or efficiency of our model is measured.

Modelling - In this actual model is build from already present models or ensemble(combination of models) and has the below stages mainly.

Build
Train - Our training data set is fed into the model based on which our model gets trained
Make predictions - Based on training data we are ready to make predictions for unknown data set

Evaluation - In this step performance is measured for the model we have built and once we are satisfied with the results we make future predictions using and if not satisfied we repeat the above steps again as its mostly some models work great on some sets of data and some may not perform well .

Hope it give you simple overview of the process, see you next time.

Search text and navigate -Chrome alike feature with marker locations using Angular

Nikhil Dhawan — Sun, 23 Jan 2022 03:45:18 +0000

Photo by Aaron Burden on Unsplash

Hi All, this post is in continuation with my last post in which we have learned how to highlight text from user input, if you haven't read that you can refer to it at https://dev.to/this-is-angular/search-and-highlight-text-feature-using-angular-l98 as it is our starting part of today's task where we will make Chrome alike search and navigate feature.

Our basic code will remain the same, now we will have a bar on the right on which we will show the location as per scroller for helping to move to highlighted text from scroll bar also with arrows to navigate between those. Let us make 1 marker bar on the right side based upon the container height the location from the top, which we can get as below

marker height: number = 0;
markerTop: number = 0;
@ViewChild('textContainer') textContainer!:ElementRef;

ngAfterContentInit(): void {
    setTimeout(() => {
      const textContainerRect =
        this.textContainer.nativeElement.getBoundingClientRect();
      this.markerHeight = textContainerRect.height;
      this.markerTop = textContainerRect.top;
    }, 100);
  }

<div *ngIf="markerHeight>0 &&markerTop>0 " class="marker" [ngStyle]="{'height':markerHeight+'px','top':markerTop+'px'}" ></div>

Now what we have on the screen is

So now we will find a way to get the location of the highlighted text based upon the class which we attached to each highlighted text which is highlighted-text in our case, which you can choose as per your choice. We will build a function that we can call after highlights are done and save the location in an array with which we can make the ticks on the bar.

getMarkerTicks() {
    setTimeout(() => {

      this.searchElements =
        this.textContainer.nativeElement.querySelectorAll('.highlighted-text');
      const markerTicks = this.searchElements.forEach((element: any) => {


        this.markerTicks.push(
          (this.markerHeight / this.scrollHeight) *
            element.getBoundingClientRect().top
        );
      });
      this.activeIndex = 1;

      this.textContainer.nativeElement
        .querySelectorAll('.highlighted-text')[0]
        .scrollIntoView({ block: 'center', behavior: 'smooth' });
    }, 100);
  }

And corresponding to it we can have html to create ticks as per number of highlights we have.

    <div class="markersTicks" *ngFor="let m of markerTicks" [ngStyle]="{'top':m+'px'}" ></div>

We need now the way to show total number of highlights and navigator buttons, for which i will use disabled input and 2 arrow buttons.

  <input class="input-navigator" type="text" disabled value="{{ activeIndex + ' of '+  markerTicks.length}}" >
    <div class="nav-btns">
      <img class="nav-btn" (click)="moveToNext()" src="../assets/down-arrow.png" alt="">
      <img  class="nav-btn" (click)="moveToBack()"  src="../assets/up-arrow.png" alt="">
    </div>

For buttons functionality we need to have functions to perform the said actions

moveToNext() {
    const highlightedSpans =
      this.textContainer.nativeElement.querySelectorAll('.highlighted-text');
    if (highlightedSpans.length > 0) {
      if (this.activeIndex === highlightedSpans.length) {
        this.activeIndex = 0;
      }
      this.activeIndex++;

      this.searchElements[this.activeIndex - 1].focus();

    }
  }
  moveToBack() {
    const highlightedSpans =
      this.textContainer.nativeElement.querySelectorAll('.highlighted-text');

    if (highlightedSpans.length > 0) {
      if (this.activeIndex === 1) {
        this.activeIndex = highlightedSpans.length + 1;
      }
      this.activeIndex--;
      this.searchElements[this.activeIndex - 1].focus();
    }
  }

Now we are done with a fully functional app for searching and navigating between them.

Full code can be found at GitHub.
You can play with it on https://nikhild64.github.io/highlight-text-navigator/

If you liked it please share it with your friends or if any suggestions reach me out on Twitter or comment below.
Till next time Happy Learning!

Search and Highlight Text feature using Angular

Nikhil Dhawan — Thu, 20 Jan 2022 15:00:07 +0000

Cover Photo by Aaron Burden on Unsplash

Hi all, in today's post we will discuss how we can make an app search and highlight features in our angular app. So the scenario can be we have a long text and we have to give the user functionality to search in a text box and related text to highlight on the paragraph below.

For this we need to have a simple setup, a search box, and a text in which we want to search(here I will take some dummy text) like as below:

For the input lets bind that to ngModel which we will use as search criteria and a div having sample text as an innerHtml

<div class="search-input">
 <label for="">Search here: </label> <input [(ngModel)]="searchText" type="search">
</div>

<div class="text-contaniner" [innerHtml]="text"  >
 </div>

export class AppComponent {
  searchText='';
  text=`somedummy text here`
}

For functionality of highlighting we will need to create an Angular pipe which I am naming as a highlighter, we can create a pipe using Angular CLI using the below command.

ng g pipe highlighter

We can put the below code in the pipe if we want to have border limit on the search

 transform(value: any, args: any): unknown {
    if(!args) return value;
      const re = new RegExp("\\b("+args+"\\b)", 'igm');
      value= value.replace(re, '<span class="highlighted-text">$1</span>');
      return value;
  }

and if we want to have text to searched irrespective of the word boundary , we can use below

 transform(value: any, args: any): unknown {
    if(!args) return value;
      const re = new RegExp("\\b("+args+"\\b)", 'igm');
      value= value.replace(re, '<span class="highlighted-text">$&</span>');
      return value;
  }

Lets add 1 more input where we utilize this partial highlighting also and combine these 2 pipes into 1 pipe based upon purpose, which makes our pipe code as

 transform(value: any, args: any,type:string): unknown {
    if(!args) return value;
    if(type==='full'){
      const re = new RegExp("\\b("+args+"\\b)", 'igm');
      value= value.replace(re, '<span class="highlighted-text">$1</span>');
    }
    else{
      const re = new RegExp(args, 'igm');
      value= value.replace(re, '<span class="highlighted-text">$&</span>');
    }

      return value;
  }

And after adding 2 inputs our UI will look like this.

Now lets integrate our code with pipe and test it out. In HTML we can add the pipe to text we added with the input as input from user and with search criteria.

<div class="text-contaniner" [innerHtml]="text | highlighter:searchText:'full'"  >

Full code can be found at GitHub.
Now let us test it out, we will be able to see the text is getting highlighted in both ways and should be as below, you can also try it out at https://nikhild64.github.io/highlight-text/:

Hope you liked it and if you have some other ways you might have used it, please comment below.
If you liked it please share it with your friends or if any suggestions reach me out on Twitter or comment below.
Till next time Happy Learning!

Blockchain - Writing our first Smart Contract

Nikhil Dhawan — Wed, 22 Dec 2021 06:58:52 +0000

Photo by Pierre Borthiry on Unsplash

Hi all, in this article we will start writing our first smart contract based on Ethereum in Solidity. In the last few articles, we discussed blockchain and its various terminology.

So for writing our smart contract we will be using an online code editor Remix with which we can quickly write our contract and test that out in the web browser itself without any local setup. We will also do some local setup in upcoming articles, But in this let us just try to understand important aspects of smart contract and solidity. So if you are familiar with JavaScript it would be easy for you to grasp solidity.
Once you open Remix, you will be able to see the left panel from that you can navigate to contracts and open Ballot. sol file. Sol is the file extension for the solidity contract.

To start with you can copy-paste the below code in the editor and then I will explain this line by line.

pragma solidity ^0.4.17;

contract Inbox{
    string private message;
    function Inbox(string initialmsg) public{
        message=initialmsg;
    }
    function setMsg(string newmsg) public{
        message=newmsg;
    }
    function getMsg() public view returns (string){
        return message;
    }
}

In this we are using solidity version 0.4.17, you can verify if the remix is using the same version of this or not avoid any complication error as per the image below.

Now let's understand the code and its functioning, this might feel familiar to you with JavaScript.

Now we can deploy this contract inside remix to do that left nav button which is as below.

Now make sure in this tab JavaScript VM is selected, and you might see you have some 100 test ethers for this environment and make sure our file is selected in contract input.

Now you can set some initial value and click on deploy. You will notice a new contract in the below section.

One more interesting thing to notice here is the amount of ether in the account selected above.

Now with this, we can understand that deploying contracts over the network blockchain involves some charges. YOu can now play around with the contract you deployed by retrieving and setting new messages.
Please let me know in case of any doubts.

Blockchain - How are transactions carried out?

Nikhil Dhawan — Mon, 13 Dec 2021 14:53:23 +0000

Photo by Clint Adair on Unsplash
Hi readers, in this article we will discuss how are the transactions started and carried out for the new transaction and the steps involved in it.

Once you submit any transaction, a backend call is made with the address details.
A transaction object is created at the backend using web3.js which includes the below details:

nonce: It is usually called a number only used once.
to: The to address which is the receiver.
value: The amount which is to be sent from sender to receiver.
gasPrice : Amount of Wei user is willing to pay for transaction. Wei is the smaller unit of ether. 1 ETH = 1,000,000,000,000,000,000 wei.
gas limit: Maximum units of gas this transaction can consume.
v,r,s: these 3 pieces of information are the cryptographic data. this can be used to generate the sender's account address by the sender's private key.
So in our case as we will be using the Rinkbey Test network in this series, this transaction object is sent to that network.
Now there is a wait time involved in this request to receive the response, which is block time(you can refer to previous articles to terms).
Once the block is added success message is received.

So this concludes our steps involved in the transaction. In the next article, we will create our first smart contract on the test network.

Starting journey toward building Blockchain App

Nikhil Dhawan — Sun, 12 Dec 2021 15:44:43 +0000

Photo by Zoltan Tasi on Unsplash

Hi all, in this article, we will discuss the path toward building our DAP (Decentralized Applications) using blockchain specifically based upon Ethereum.

So when Ethereum was launched it was kept in mind that it will help make Dap's and nowadays it's a famous cryptocurrency and it can also be used to transfer data based upon blockchain technology. There are many networks involved in the overall ecosystem and each network has 1 or more nodes and each node is running an Ethereum client. Each node has a full copy of an identical blockchain.

For end consumers, there are wallets from which they can access their wallets and carry out transactions e.g. we have chrome extension Metamask and developers use web3.js. You can install Metamask in your chrome form here.

For our journey toward building our App, we will use the Rinkeby Test network of Ethereum which help us to test out our transactions without burning real cash.

If we see here we have an account address that is unique to a user and will be the same across different networks. For getting free ETH in your test account to the address you get in your wallet you can use https://faucets.chain.link/

These tokens don't have any real value but can be used in testing our smart contract codes for free before we make them usable on the main ETH network.
The account address we see on the wallet is our public address(can be referred to as an email address for understanding) which we can use to receive or send ETH tokens and there are 2 other keys involved with it Public and Private key which combined to act as an identifier for our account to carry out the transaction.

Thanks for reading this. In upcoming articles in this series, we will continue our journey to learn more and discuss how is the transaction carried out and important pointers in that.

Blockchain - Commonly used terms

Nikhil Dhawan — Sat, 11 Dec 2021 16:41:29 +0000

Hi all, in this article, we will try to cover some of the terms which are very helpful in understanding blockchain technology better.

Block Time
Block time refers to the time it takes for adding new data to the existing network. This time taken by the blockchain network is mainly the time it takes to validate the hash of that data to some predefined value. Like for example for Bitcoin, this time is around 10 mins and for Ethereum it is around 15 seconds which depicts that it is fast to add data on ETH than BTC. To see the average Block time for Ethereum over time you can refer to https://etherscan.io/chart/blocktime .
Average Block time of ETH over time

Smart Contract
Smart contracts are accounts controlled by the code. They tell how incoming and outgoing requests are to be handled without the intervention of any other party involvement, that is the reason the blockchain cryptocurrencies are referred to as decentralized banking systems. This is the case of Ethereum has below information:

Code: Raw machine code for this contract.
Storage: Data storage of contract.
Balance: Amount of Ether.

Hash Functions
These are the functions that are used to convert input data into encrypted values of a certain length and the same data will always produce the same output. It is not possible to get the original data from the hash outputs which makes it important for blockchains for higher security.

Proof of Work
It is how some computations are done by the network nodes to get to the hash output for the certain matching predefined criteria before the block is added to the network. Doing this makes use of resources of computer i.e. work, hence the person who is lending these resources are called minors and is given some rewards in return for this. Bitcoin uses this method to add new transactions blocks to the network. But this method involves higher energy consumption.

Proof of Stake
Proof of stake was developed as an alternative to the Proof of Work. In this, the miner can only mine once he has put his/her coins on the stake and can only mine the max of the coins that are on stake. So this solves the issue of higher power consumption that is there is a POW. Ethereum as of now uses both POS and POW and is planning to completely move toward POS.

Thanks for reading this article, in upcoming articles we will discuss more on blockchain technology.

Blockchain- An Introduction

Nikhil Dhawan — Fri, 10 Dec 2021 13:59:08 +0000

Photo by Thought Catalog on Unsplash

Hi all, In the upcoming series of articles we will discuss what blockchain is and what are the ways we can get into building our apps based upon this technology.

Blockchain is a technology in simple words helps to make some irreversible transactions or the block of data. So thinking in this way makes it the secure way of storing data than the databases. It works in this way like we have a chain of data 1 behind the other and the front data node has the address of the node which is behind it and the 1 in front and has its hash key which is SHA256 hash and is like a digital fingerprint of that data, so if you make some changes in the data that changes the hash key and makes it as out of sync from the other blocks.

To make it in sync we might think we will change the hash addresses in other nodes but that would be an easy task to do for when there are millions of nodes, moreover here we are talking about only 1 network that has a copy of node data, but in actual blockchain, we have many copies of same data and the data to be modified on network copies is not that easy and if on some network it is found data to be mismatched that will not be modified anyhow as blockchain as the technology works on proof of work concepts.
We will discuss blockchain terms more in upcoming articles alongside learning to build apps based upon blockchain

Resource Helper website for easy access to daily usable links

Nikhil Dhawan — Mon, 06 Sep 2021 14:27:43 +0000

Hi all,

I am happy to announce the Resource Helper website, which I have been working on for the last 2 weeks. The link is https://reshelper.netlify.app/ .
I thought to make an app like this because many a time we need to go through google for simple links again and again and sometimes we even bookmark them but it gets lost in the huge list. You can also add your links on the website which you think might be helpful. I will make this GitHub repo public in a week as I need to check some more things. If you have any suggestions for improvement please let me know.
If you liked the concept please share it in your network so that it is helpful for more people.