DEV Community: Md Manawar Iqbal

Wander: A Screen-Free Voice Companion That Gets You Outside 🌿

Md Manawar Iqbal — Sun, 11 Oct 2026 11:44:13 +0000

Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass Submission 🌿

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

Wander is a screen-free, voice-first walking companion.

You put your phone in your pocket, tap once, and talk.

It narrates nearby places and species, gives you small “touch grass” challenges, keeps a life list of what you spot, plans walking loops, and logs every walk to a map journal.

The entire brain runs locally with Gemma 3 (via Ollama) + faster-whisper.

Only the voice replies use ElevenLabs (with an offline Piper fallback).

Demo

GitHub: https://github.com/Manawariqbal/wander

How it works

Phone (client) ──WebSocket──▶ FastAPI server
│ │
GPS + voice Whisper → WalkAgent → Gemma
│ │
plays audio ◀── mp3 ──────── ElevenLabs / Piper

Key pieces:

Grounded narration – nearby places (OSM + Wikipedia), species (iNaturalist), weather and sunset are prefetched once. Gemma is only allowed to talk about facts that were actually retrieved.
Offline place cache (poi_cache) – after the first fetch, the walk can continue with almost no internet.
Touch-grass challenges every ~7 minutes.
Life list – log birds/plants/animals by voice; it tells you when something is new.
End-of-walk summary – clear spoken recap of distance, species, and challenges.

Why open innovation matters here

A closed cloud model would force every GPS ping and every utterance into someone else’s servers.

Wander keeps the entire reasoning loop local so you can:

Walk in the mountains with no signal and still get grounded narration
Keep your location data on your own machine
Swap models, fine-tune, or change the agent behaviour freely
Run inference at zero cost after the initial setup

The open pieces (Gemma + local Whisper + open map/species data) are what make the “phone stays in the pocket” promise actually work.

Partner categories

This project is submitted under:

Best Use of Gemma
Best Use of ElevenLabs

Built with

Gemma 3 (Ollama)
faster-whisper
ElevenLabs
FastAPI + WebSocket
OpenStreetMap / Overpass, Wikipedia, iNaturalist, Open-Meteo, OpenRouteService
SQLite + Leaflet

What’s next

Hands-free mode with Silero VAD
Better journal polish
Multiple guide voices / Hindi support

Built during the challenge window (5–11 Oct 2026).

Thanks for reading — now go touch some grass.

DataPilot — An Open-Source AI Data Scientist That Investigates Your Dataset

Md Manawar Iqbal — Mon, 05 Oct 2026 16:23:11 +0000

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

What I Built

built DataPilot, an open-source autonomous data scientist for tabular datasets.
I built DataPilot, an open-source autonomous data scientist for tabular datasets.

I built it for a friend who works with data and often starts with the same frustrating workflow:

Upload CSV → inspect columns → check missing values → perform EDA → train a baseline → try another model → compare results → figure out what to investigate next.

A lot of this work is repetitive, and the bigger problem isn't running a model — it's knowing what to investigate next and why.

DataPilot tries to solve that.

You give it a CSV and a target column, and it combines deterministic Python data-science tools with a local open-weight LLM to investigate the dataset.

The goal isn't to build another "chat with your CSV" application.

Instead, DataPilot follows an investigation loop:

Dataset
↓
Profile
↓
EDA
↓
Hypothesis
↓
Experiment
↓
Evidence
↓
Next Investigation
↓
Final Report

The AI scientist decides what it should investigate next, while Python tools perform the actual data analysis and calculate the metrics.

This separation is important:

The LLM reasons about the investigation.
Python produces the evidence.

For example, on my loan-default dataset, DataPilot found:

credit_score → correlation: -0.1472, p-value: 0.00096
income → correlation: -0.1112, p-value: 0.01286
debt_ratio → correlation: 0.0619, p-value: 0.167
age → correlation: -0.0132, p-value: 0.768

It then compared a Logistic Regression baseline against TabPFN.

LogisticRegression
Accuracy: 0.770
Weighted F1: 0.6786

TabPFN
Accuracy: 0.775
Weighted F1: 0.6768

Rather than simply saying "TabPFN is better", DataPilot's evidence evaluator determines that there is no clear overall improvement because the models disagree slightly on accuracy vs. F1.

That evidence-first behavior is the foundation I'm building around.

Demo

The project currently runs locally with an open-weight model through Ollama.

Example:

git clone https://github.com/Manawariqbal/datapilot.git
cd datapilot

python test_agent.py

The agent runs the investigation and produces a scientific report from the generated evidence.

I'm also working toward a Streamlit interface where the workflow becomes:

Upload CSV
↓
Select target
↓
Run DataPilot
↓
Watch investigation
↓
Read evidence-backed report

Code

https://github.com/Manawariqbal/datapilot

The project is intentionally built as a modular open-source system rather than one large agent prompt.

Current structure:

datapilot/
├── app.py
├── requirements.txt
├── README.md
├── data/
│ └── loan_default_sample.csv
├── tests/
│ ├── test_profiling.py
│ └── test_eda.py
└── datapilot/
├── profiling.py
├── problem_detector.py
├── models.py
├── tabpfn_tool.py
├── eda.py
├── experiments.py
├── evidence.py
└── agent/
├── state.py
├── llm.py
├── nodes.py
├── graph.py
└── report.py -->

How I Built It

The core of DataPilot is built around open-source AI and local inference.

Open-weight AI

I'm using:

Gemma 3 4B

running locally through Ollama.

The LLM acts as the scientific reasoning layer.

It decides things such as:

What should I investigate next?
What hypothesis should I test?
Is the current evidence sufficient?
Should I continue or produce a report?

But the LLM does not calculate the statistical metrics itself.

Agent orchestration

I'm using LangGraph to orchestrate the investigation.

The architecture is roughly:

             ┌───────────────┐
             │   Gemma 3 4B  │
             │    Scientist  │
             └───────┬───────┘
                     │
              Next investigation
                     │
   ┌─────────────────┼─────────────────┐
   ↓                 ↓                 ↓
EDA Tool        Baseline Tool      TabPFN Tool
   │                 │                 │
   └─────────────────┼─────────────────┘
                     ↓
              Evidence Evaluator
                     │
                     ↓
             Scientific Report

The Python tools are deterministic.

For example:

Pandas performs data analysis.
SciPy calculates statistical relationships.
scikit-learn trains the baseline.
TabPFN performs the foundation-model experiment.
Python calculates the evaluation metrics.
The LLM interprets the resulting evidence.

This prevents the agent from simply inventing numbers.

Machine Learning

The current stack includes:

Python
Pandas
NumPy
SciPy
scikit-learn
TabPFN
LangGraph
LangChain
Ollama
Gemma 3
Streamlit
Pydantic

One of the experiments compares:

Logistic Regression
vs.
TabPFN

The project is designed so additional experiments can become new tools.

Evidence-driven reasoning

This is probably the part of the project I'm most interested in.

Instead of:

LLM:
"Credit score seems important."

I want DataPilot to produce:

Hypothesis:
Credit score is associated with loan default.

Experiment:
Calculate Pearson correlation and significance.

Evidence:
r = -0.1472
p = 0.00096

Interpretation:
A weak negative association is present.

Next investigation:
Determine whether the relationship
has predictive value beyond linear association.

That creates a much more useful agent:

Hypothesis
↓
Experiment
↓
Evidence
↓
Reasoning
↓
Next hypothesis

Why Does Open Innovation Matter?

This project is specifically designed around the idea that the AI reasoning layer should be replaceable and locally runnable.

Using an open-weight model makes it possible to run the scientist locally instead of sending the dataset to a proprietary API.

That matters for data-science workflows because datasets can contain sensitive information.

With the current architecture, the workflow can look like:

Private Dataset
↓
Local Python Analysis
↓
Local Gemma
↓
Local Investigation
↓
Local Report

No proprietary LLM API is required for the core reasoning loop.

Open models also make experimentation much easier.

I can change the reasoning model without redesigning the data-science tools:

Gemma
↓
Qwen
↓
Another open-weight model

while keeping the same:

EDA
Models
TabPFN
Evidence
LangGraph

architecture.

That's the part of open innovation I wanted to explore: AI agents should not have to be tied to one closed model provider.

What I Want DataPilot to Become

The current version is the foundation.

The next major step is making the investigation genuinely autonomous.

Instead of the agent choosing between a fixed set of tools, I want it to discover increasingly specific investigations:

Dataset
↓
EDA
↓
"Credit score has a significant relationship"
↓
Hypothesis
↓
Feature analysis
↓
Evidence
↓
"Linear relationship is weak"
↓
New hypothesis
↓
Non-linear analysis
↓
Evidence
↓
Model experiment
↓
Final scientific report

Eventually, every run should leave behind reproducible artifacts:

runs/
└── run_20261005/
├── profile.json
├── hypotheses.json
├── experiments.json
├── model_results.json
├── charts/
└── report.md

So DataPilot isn't just an AI that talks about your dataset.

It's an AI that investigates the dataset and leaves behind evidence of what it actually did.

Prize Categories

Gemma

DataPilot uses Gemma 3 4B as its local scientific reasoning model through Ollama.

TabPFN

DataPilot uses TabPFN as one of its model experimentation tools for tabular datasets.

Final Thought

I started this project because I wanted to build something more meaningful than another chatbot wrapper.

A useful data scientist doesn't just answer:

"What does this dataset contain?"

They ask:

"What should I investigate next?"

That's what I'm trying to build with DataPilot.

An open-source AI data scientist that doesn't just answer questions about your data — it investigates them.

🔍 Top 10 Websites Every Developer Should Know 🔍

Md Manawar Iqbal — Tue, 25 Jul 2023 22:22:28 +0000

👩‍💻 Being a developer means staying up-to-date with the latest tools and resources. Here's a thread with ten must-know websites that will boost your skills and productivity. Let's dive in! 👇

1️⃣ GitHub (github.com):
A treasure trove of open-source projects, collaboration, and version control. Contribute, discover, and showcase your work on the world's largest developer platform. #GitHub #OpenSource

2️⃣ Stack Overflow (stackoverflow.com):
The go-to place for developers to seek solutions and share knowledge. Ask questions, find answers, and get involved in the thriving developer community. #StackOverflow #DeveloperCommunity

3️⃣ MDN Web Docs (developer.mozilla.org):
An extensive resource for web developers with in-depth documentation on HTML, CSS, JavaScript, and more. Stay ahead in web development with MDN. #MDNWebDocs #WebDevelopment

4️⃣ CodePen (codepen.io):
A playground for front-end developers to experiment with HTML, CSS, and JavaScript code snippets. Showcase your creations and get inspired by others' work. #CodePen #FrontEndDev

5️⃣ Dev.to (dev.to):
A welcoming platform for developers to share knowledge, experiences, and insights. Engage with the community, learn new concepts, and level up your skills. #DevCommunity #Learning

6️⃣ HackerRank (hackerrank.com):
Sharpen your coding skills through challenges and competitions. Improve problem-solving abilities and get noticed by top tech companies. #HackerRank #CodingChallenges

7️⃣ Smashing Magazine (smashingmagazine.com):
A fantastic resource for web designers and developers. Stay updated with the latest trends, tips, and best practices in the industry. #SmashingMagazine #WebDesign

8️⃣ CSS-Tricks (css-tricks.com):
The ultimate place to learn and master CSS tricks and techniques. Enhance your front-end development skills with valuable tips and tutorials. #CSSTricks #FrontEndTips

9️⃣ LeetCode (leetcode.com):
Crack coding interviews and enhance problem-solving skills with real coding questions. Prepare for technical interviews and land your dream job. #LeetCode #CodingInterviews

🔟 Awwwards (awwwards.com):
For web designers seeking inspiration and recognition, Awwwards showcases outstanding website designs that push the boundaries of creativity. #Awwwards #WebDesignInspiration

💡 Bookmark these websites and explore them regularly to stay at the forefront of the developer community. Happy coding! 🚀 #DeveloperResources #WebDevelopment #Programming

(Note: URLs are fictional and provided as examples. Please use the actual URLs of the mentioned websites.)

Simple Landing Page in HTML

Md Manawar Iqbal — Mon, 09 Jan 2023 06:36:56 +0000

Here is simple html code for your landing page

<html>
  <head>
    <title>Landing Page</title>
  </head>
  <body>
    <h1>Welcome to my landing page!</h1>
    <p>Thank you for visiting. Please take a look around and learn more about what I have to offer.</p>
    <ul>
      <li><a href="#services">Services</a></li>
      <li><a href="#about">About me</a></li>
      <li><a href="#contact">Contact</a></li>
    </ul>
    <h2 id="services">Services</h2>
    <p>I offer a variety of services, including web design, graphic design, and social media management. Let me know how I can help you succeed online.</p>
    <h2 id="about">About me</h2>
    <p>I am a creative professional with years of experience in the industry. I am passionate about helping businesses and individuals achieve their goals through effective online marketing.</p>
    <h2 id="contact">Contact</h2>
    <p>If you have any questions or would like to work with me, please don't hesitate to get in touch. You can reach me by email at hello@example.com or by phone at 555-555-5555.</p>
  </body>
</html>

This code creates a basic landing page with a heading, a paragraph, and a list of links to different sections of the page. The page also includes three sections: "Services," "About me," and "Contact." Each section has its own heading and paragraph of text.

Hopes it helps you
Do follow me on twitter

Clustering Algorithms in Machine Learning

Md Manawar Iqbal — Mon, 09 Jan 2023 06:29:47 +0000

Clustering algorithms are a type of unsupervised learning method in machine learning that divides a dataset into groups (also known as clusters) based on the patterns in the data. The goal of clustering algorithms is to group data points in a way that points within a group are more similar to each other than they are to points in other groups.

There are several different types of clustering algorithms, each with its own unique characteristics and applications. Some common examples include:

K-means clustering: This is a widely used algorithm that divides a dataset into a specified number (k) of clusters. It does this by iteratively assigning each data point to the closest cluster based on the mean of the points in the cluster, and then updating the mean of each cluster until convergence.

Hierarchical clustering: This type of clustering algorithm creates a hierarchy of clusters by repeatedly dividing the dataset into smaller and smaller clusters. There are two main types of hierarchical clustering: agglomerative, which starts with individual data points and merges them into larger clusters, and divisive, which starts with the whole dataset and divides it into smaller clusters.

DBSCAN: This algorithm is designed to find clusters of any shape in a dataset. It does this by identifying points that are in high-density regions of the dataset and grouping them together, while ignoring points in low-density regions.

Clustering algorithms have a wide range of applications, including image segmentation, text analysis, and customer segmentation. They are often used as a preprocessing step for other machine learning tasks, such as classification and regression.

It's important to note that choosing the right clustering algorithm for a given dataset can be a challenge, as different algorithms may perform better on different types of data. It is also important to carefully evaluate the quality of the clusters produced by a clustering algorithm, as this can have a significant impact on the accuracy of downstream machine learning tasks.

Data Science Best Learning Roadmap

Md Manawar Iqbal — Thu, 24 Nov 2022 14:40:51 +0000

Data Science Learning Roadmaps

ML #DataScience #python

When we are start learning data science we some times get confused where to start so in this article thread Try to explain which roadmaps we should follow for learning data science. Just search every topics on internet and learn them you can follow me on github also where I try to upload the source code of projects you can follow me on github
https://github.com/Manawariqbal
01
Getting Started with Python
Python Basics
Loops in Python
Functions in Python
Strings in Python
Data Structure

2
Files and JSON
Getting started with Files
Minor Project:Inventory Management System with Files
Minor Project: Inventory Management System with JSON
3
OS with Python
Getting started with OS
Jupyter Notebook Setup
OS with Python
4
Mastering Numpy Arrays
Mastering Numpy Arrays
05
Data Analysis with Python
Getting started with Pandas
Statistics
Data Preprocessing
Data Analysis
Minor Project: Black Friday Sales Data Analysis
Minor Project: Data Visualization on Heart Disease Dataset
Minor Project: GDP Analysis
06
Tableau
Introduction to Tableau
Understanding Parameters
Basic Plots in Tableau
Fundamentals of Tableau
Designing the Plots
Project 1 - Superstore Sales Analysis Dashboard
Project 2 - Covid-19 World Dashbaord
07
Web Scraping
To Scrape
Minor Project: Wikipedia Scraper
Selenium
Minor Project: Youtube Scrapper
Minor Project: Stock Images Infinite Scroll
Image Dataset Creation
08
Getting Started with AI
Introduction to AI
How Data Science Comes into Play
09
Machine Learning
Linear Regression
Multiple Linear Regression
Polynomial Linear Regression
Support Vector Machine
Decision Tree
Random Forest
Classification Algorithms
Clustering Algorithms
Feature Engineering
10
Image Processing
Pixels with Lists
Image and Gradient Formation
Image Manipulation
Instagram Filters Colour and Patterns
Edge, Contour and Colour Detection
Face Detection with HaarCascase
Webcam using OpenCV
Image Manipulation with Live Webcam
Realtime Face Image Processing
11
Deep Learning
Perceptrons
Multi layer perceptron architecture
Convolutional neural network

12
Natural Language Processing
Getting started with Text
Regular Expression with Python
Getting started with Spacy and PDF
Text Encoding Decoding from Scratch

13
Major Projects
MNIST Handwritten Digit Recogniser
Titanic Survival | EDA
Stock Image Scrapper

What is ERP ?

Md Manawar Iqbal — Tue, 22 Nov 2022 15:07:10 +0000

Many time we heard the term ERP in software Engineering
Today I am going to explain about ERP in brief
what is ERP?
According to wikipedia it's define as

Enterprise resource planning (ERP) is the integrated management of main business processes, often in real time and mediated by software and technology. ERP is usually referred to as a category of business management software—typically a suite of integrated applications—that an organization can use to collect, store, manage and interpret data from many business activities.

For many companies, Enterprise Resource Planning (ERP) systems are a critical element of their success. ERP refers to software and systems that are used to plan and manage all core supply chain, manufacturing, services, financial and other processes of an organization. ERP can considerably streamline an organization’s operations, automating functions such as accounting, procurement, project management, customer relationship management, risk management, compliance and supply chain operations.

While a business can utilize an individual ERP application, a complete ERP application suite brings processes together across a business, connecting every aspect of the enterprise and allowing for improved communication and data gathering. Today’s ERP systems extend processes by providing collaboration and digital solutions to effectively connect with partners outside the organization including, customers, suppliers, carriers, financial institutions and others.
What are the Benefits of ERP?
The benefits of ERP are many. ERP system benefits range from optimizing processes to helping different departments better collaborate to improving the relationship between a business and its customers. The following are several of the most valued benefits to utilizing ERP.
Increased Efficiency
One of the primary advantages of ERP is that it allows companies to automate manual and routine functions.
Improved Collaboration
ERP systems also connect teams, improving communication and employee engagement. With an ERP system, every approved employee has on-demand access to operational data, allowing them to understand all of the company’s moving parts and the role they play.
Real-time Data & Enhanced Reporting
One of the biggest benefits of ERP is that it’s a powerful data hub. An ERP system allows you to collect, store, and analyze data across your operations in one centralized location
Cloud Accessibility
Though ERP systems are available as on-premise solutions, cloud ERP software has become more popular in recent years.
What are the Disadvantages of ERP?
Though the advantages of an ERP system far outweigh the disadvantages, there are a few to consider to make sure moving forward with ERP is right for your business at a given time.

Upfront Costs
There are initial costs to keep in mind when considering an ERP system, particularly an on-premise system, such as the cost of implementation, maintenance, training and potential third-party software add-ins.
Training
To get the most out of ERP software, proper training is a must. Training should cover all of the ERP system’s features, with sessions in line with business processes.
*Leading software companies providing ERP services are *
SAP
Oracle
Microsoft
Workday
Follow me on twitter and Linkedin

Data Preprocessing

Md Manawar Iqbal — Tue, 22 Nov 2022 14:50:55 +0000

What is Data Pre-processing?
"Data Preprocessing is a process of converting your raw data into suitable form."
Data Preprocessing Steps Involves
1:Getting Dataset
2:Importing Libraries
3:Importing Datasets
4:Finding Missing Values
5:Encoding Categorical Data
6:Splitting Dataset into Training & Test Set
7:Feature Scaling

Numpy in Python

Md Manawar Iqbal — Thu, 17 Nov 2022 07:10:43 +0000

NumPy
The NumPy library is the core library for scientific computing in Python. It provides a high-performance multidimensional array object, and tools for working with these arrays.

Use the following import convention:

>>> import numpy as np

*Numpy Arrays *

*Creating Arrays *

>>> a = np.array([1,2,3])
>>> b = np.array([(1.5,2,3), (4,5,6)], dtype = float)
>>> c = np.array([[(1.5,2,3), (4,5,6)],[(3,2,1), (4,5,6)]], dtype = float)

Initial Placeholders

>>> np.zeros((3,4)) #Create an array of zeros
>>> np.ones((2,3,4),dtype=np.int16) #Create an array of ones
>>> d = np.arange(10,25,5)#Create an array of evenly spaced values (step value)
>>> np.linspace(0,2,9) #Create an array of evenlyspaced values (number of samples)
>>> e = np.full((2,2),7)#Create a constant array
>>> f = np.eye(2) #Create a 2X2 identity matrix
>>> np.random.random((2,2)) #Create an array with random values
>>> np.empty((3,2)) #Create an empty array

I/O

Saving & Loading on Disk

>>> np.save('my_array' , a)
>>> np.savez( 'array.npz', a, b)
>>> np.load( 'my_array.npy')

Saving & Loading Text Files

>>> np.loadtxt("myfile.txt")
>>> np.genfromtxt("my_file.csv", delimiter= ',')
>>> np.savetxt( "myarray.txt", a, delimiter= " ")

Asking For Help
>>> np.info(np.ndarray.dtype)
Inspecting Your Array

>>> a.shape #Array dimensions
>>> len(a)#Length of array
>>> b.ndim #Number of array dimensions
>>> e.size #Number of array elements
>>> b.dtype  #Data type of array elements
>>> b.dtype.name  #Name of data type
>>> b.astype(int). #Convert an array to a different type

Data Types

>>> np.int64 #Signed 64-bit integer types
>>> np.float32. #Standard double-precision floating point
>>> np.complex. #Complex numbers represented by 128 floats
>>> np.bool  #Boolean type storing TRUE and FALSE values
>>> np.object #Python object type
>>> np.string_ #Fixed-length string type
>>> np.unicode_ #Fixed-length unicode type

Array Mathematics

Arithmetic Operations

>>> g = a - b. #Subtraction
   array([[-0.5,0. ,0.], [-3. , -3. , -3. ]])
>>> np.subtract(a,b) #Subtraction
>>> b + a #Addition 
  array([[ 2.5, 4. , 6.],[5. ,7. ,9. ]])
>>> np.add(b,a) #Addition 
>>> a/b #Division 
 array([[0.66666667,1. ,1.],[0.25 ,0.4 ,0.5 ]])
>>> np.divide(a,b) #Division 
>>> a * b #Multiplication 
  array([[1.5, 4. ,9.],[ 4. , 10. , 18. ]])
>>> np.multiply(a,b) #Multiplication 
>>> np.exp(b) #Exponentiation
>>> np.sqrt(b) #Square root
>>> np.sin(a)  #Print sines of an array
>>> np.cos(b) #Elementwise cosine
>>> np.log(a)#Elementwise natural logarithm
>>> e.dot(f) #Dot product 
 array([[7.,7.],[7.,7.]])

Comparison

>>> a == b #Elementwise comparison

 array([[False , True, True],
             [ False,False ,False ]], dtype=bool)
>>> a< 2 #Elementwise comparison
   array([True, False, False], dtype=bool)
>>> np.array_equal(a, b) #Arraywise comparison
Copying Arrays 
>>>h = a.view()#Create a view of the array with the same data
>>> np.copy(a) #Create a copy of the array
>>>h = a.copy() #Create a deep copy of the array
Sorting Arrays 
>>> a.sort() #Sort an array
>>> c.sort(axis=0) #Sort the elements of an array's axis

Subsetting, Slicing, Indexing
Subsetting

>>> a[2] #Select the element at the 2nd index
  3
>>> b[1,2] #Select the element at row 1 column 2(equivalent to b[1][2])
  6.0

Slicing


>>> a[0:2]#Select items at index 0 and 1
 array([1, 2])
>>> b[0:2,1] #Select items at rows 0 and 1 in column 1
  array([ 2.,5.])
>>> b[:1] 
#Select all items at row0(equivalent to b[0:1, :])
  array([[1.5, 2., 3.]])
 >>> c[1,...] #Same as[1,:,:]
 array([[[ 3., 2.,1.],[ 4.,5., 6.]]])
>>> a[ : : -1] #Reversed array a array([3, 2, 1])

Boolean Indexing

>>> a[a<2] #Select elements from a less than 2
 array([1])

Fancy Indexing

>>> b[[1,0,1, 0],[0,1, 2, 0]] #Select elements(1,0),(0,1),(1,2) and(0,0)
  array([ 4. , 2. , 6. ,1.5])
>>> b[[1,0,1, 0]][:,[0,1,2,0]] #Select a subset of the matrix’s rows and columns
 array([[ 4. ,5. , 6. , 4.],[1.5, 2. , 3. ,1.5],[ 4. ,5. , 6. , 4.],[1.5, 2. , 3. ,1.5]])

Array Manipulation
Transposing Array

>>> i = np.transpose(b) #Permute array dimensions
>>> i.T #Permute array dimensions

Changing Array Shape

>>> b.ravel() #Flatten the array
>>> g.reshape(3, -2) #Reshape, but don’t change data

Adding/Removing Elements

>>>h.resize((2,6)) #Return a new arraywith shape(2,6)
>>> np.append(h,g) #Append items to an array
>>> np.insert(a,1,5)  #Insert items in an array
>>> np.delete(a,[1])  #Delete items from an array

Combining Arrays

np.concatenate((a,d),axis=0) #Concatenate arrays
array([1, 2, 3, 10, 15, 20])

>>> np.vstack((a,b) #Stack arrays vertically(row wise)
 array([[1. , 2. , 3.],[1.5, 2. , 3.],[ 4. ,5. , 6. ]])
>>> np.r_[e,f] #Stack arrays vertically(row wise)
>>> np.hstack((e,f)) #Stack arrays horizontally(column wise)
 array([[7.,7.,1.,0.],[7.,7.,0.,1.]])
>>> np.column_stack((a,d)) #Create stacked column wise arrays
 array([[1, 10],[ 2, 15],[ 3, 20]])
>>> np.c_[a,d] #Create stacked column wise arrays
Splitting Arrays 

>>> np.hsplit(a,3) #Split the array horizontally at the 3rd index
  [array([1]),array([2]),array([3])]
>>> np.vsplit(c,2) #Split the array vertically at the 2nd index
  [array([[[ 1.5, 2. ,1.],[ 4. ,5. , 6. ]]]),
   array([[[ 3., 2., 3.],[ 4.,5., 6.]]])]

Follow me on twitter and Linkedin

Python os Module

Md Manawar Iqbal — Thu, 17 Nov 2022 05:56:01 +0000

It is possible to automatically perform many operating system tasks. The OS module in Python provides functions for creating and removing a directory (folder), fetching its contents, changing and identifying the current directory, etc.

You first need to import the os module to interact with the underlying operating system. So, import it using the import os statement before using its functions.

Getting Current Working Directory
The getcwd() function confirms returns the current working directory.

>>> import os
>>> os.getcwd()
'C:\\Python37'

Creating a Directory
We can create a new directory using the os.mkdir() function

>>> import os
>>> os.mkdir("C:\MyPythonProject")

Changing the Current Working Directory
We must first change the current working directory to a newly created one before doing any operations in it. This is done using the chdir() function.

>>> import os
>>> os.chdir("C:\MyPythonProject") # changing current workign directory
>>> os.getcwd()
'C:\MyPythonProject'

Removing a Directory
The rmdir() function in the OS module removes the specified directory either with an absolute or relative path.

>>> import os
>>> os.rmdir("C:\\MyPythonProject")

List Files and Sub-directories
The listdir() function returns the list of all files and directories in the specified directory.

>>> import os
>>> os.listdir("c:\python37")
['DLLs', 'Doc', 'fantasy-1.py', 'fantasy.db', 'fantasy.py', 'frame.py', 
'gridexample.py', 'include', 'Lib', 'libs', 'LICENSE.txt', 'listbox.py', 'NEWS.txt',
'place.py', 'players.db', 'python.exe', 'python3.dll', 'python36.dll', 'pythonw.exe', 
'sclst.py', 'Scripts', 'tcl', 'test.py', 'Tools', 'tooltip.py', 'vcruntime140.dll', 
'virat.jpg', 'virat.py']

Introduction to Data Analysis

Md Manawar Iqbal — Tue, 15 Nov 2022 14:51:08 +0000

What is Data analysis?
Data analysis is a process of inspecting, cleansing, transforming, and modeling data with the goal of discovering useful information, informing conclusions, and supporting decision-making.
Tools used in Data Analysis :
Auto-managed closed tools->
Qwiklabs, Tableau,Looker, Zoho Analytics
The programming language used: Python,R,Julia
Why Python for Data Analysis?
*-very simple and intuitive to learn.
-correct language
-powerful libraries

-free and open source
-Amazing community,docs, and conferences
**When to choose R language?
*- When R studio is needed
-When dealing with advanced statistical methods.
-When extreme performance is needed.
**Data analysis Process:
1:Data extraction->
SQL,Scrapping ,File format(CSV,JSON,XML),Consulting APIs,Buying Data,Distributed database
2:Data cleaning ->
• Missing values and
empty data
• Data imputation
• Incorrect types
Incorrect or invalid
values
• Outliers and non
relevant data
● Statistical sanitization

Data Wrangling->

Hierarchical Data
Handling categorical
data
Reshaping and
transforming
structures
Indexing data for
quick access
Merging,combining
and joining data
4:Analysis->
• Exploration
• Building statistical
models
• Visualization and
representations
. Correlation vs
Causation analysis
• Hypothesis testing
● Statistical analysis
• Reporting
5:Actions->
• Building Machine
Learning Models
Feature Engineering
• Moving ML into
production
• Building ETL
pipelines
• Live dashboard and
reporting
• Decision making
and real-life tests
**PYTHON ECOSYSTEM:
**The libraries we can use ...
pandas: The cornerstone of our Data Analysis job with Python
matplotlib:The foundational library for visualizations.Other libraries we'll use will be
built on top of matplotlib.
numpy:The numeric library that serves as the foundation of all calculations in Python.
seaborn:A statistical visualization tool built on top of matplotlib.
statsmodels:A library with many advanced statistical functions.
scipy:Advanced scientific computing, including functions for optimization,linear
algebra, image processing and much more.
scikit-learn:The most popular machine learning library for Python (not deep learning)

Top Sites for Education and Career Enhancement

Md Manawar Iqbal — Sun, 13 Nov 2022 09:35:39 +0000

➢ ► FREELANCING
➢(http://www.fiverr.com/
➢http://www.upwork.com/
➢http://www.odesk.com/
➢http://www.elance.com/
➢http://www.99designs.com/
➢http://www.freelancer.com/
➢http://www.guru.com/
➢http://www.peopleperhour.com/
➢http://www.freelanced.com/
➢http://www.ifreelance.com/

)
➢ ► CAREER
➢https://www.linkedin.com/
➢https://www.indeed.com/
➢https://www.naukri.com/
➢https://www.monster.com/
➢https://www.careercloud.com/
➢https://www.dice.com/
➢https://www.careerbuilder.com/
➢https://www.jibberjobber.com/
➢https://www.glassdoor.com/index.html

➢ ► RESUME
➢https://zety.com/
➢https://www.resumonk.com/
➢https://www.resume.com/
➢https://www.visualcv.com/
➢https://www.cvmaker.com/
➢https://resumup.com/
➢https://resumegenius.com/
➢https://www.resumebuilder.com/
➢http://www.resumebaker.com/
➢https://enhancv.com/

➢ ► INTERVIEW PREPARATION
➢https://www.ambitionbox.com/about-us
➢https://www.acetheinterview.com/
➢https://www.geeksforgeeks.org/
➢https://leetcode.com/
➢http://www.gainlo.co/
➢https://careercup.com/
➢https://coder.com/careers
➢https://interviewup.blogspot.com/
➢https://www.livecareer.com/resources/inter
views/prep/interviewbest-blog
➢https://www.indiabix.com/

➢ ► FREE ONLINE EDUCATION
➢https://www.coursera.org/
➢https://www.edx.org/
➢https://www.khanacademy.org/
➢https://www.udemy.com/
➢MIT OpenCourseWare
(https://ocw.mit.edu/index.htm) (Free
Online Course Materials)
➢https://online.stanford.edu/
➢https://www.codecademy.com/
➢(https://github.com/pamoroso/free-python-books)
Learn Python for free