SUMIT GAVALI

Posted on Sep 2

CLI tool for data-science: 891 downloads in 7dy.

#opensource #python #datascience #machinelearning

The Hidden Cost of "Simple" EDA

Every data science project starts the same way. You download a dataset, open a Jupyter notebook, and write the same 50+ lines of code you've written hundreds of times before:

import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

df = pd.read_csv('data.csv')
df.head()
df.info()
df.describe()
df.isnull().sum()
# ... 30 more lines of boilerplate

For a professional data scientist, this ritual takes 45-60 minutes. For a student or junior analyst, it's easily 2-3 hours of wrestling with syntax, debugging import errors, and googling "how to plot correlation matrix in seaborn" for the 47th time.

I did this 47 times. I lost my mind.

So I built something that does it in one command.

The Problem I Was Solving

As a data science student and ML intern, I realized the friction wasn't in the analysis—it was in the setup. Every project demanded:

Dataset discovery – finding and downloading the right data
Data loading – handling different formats and paths
Profiling – understanding structure, types, missing values
Visualization – creating 5-10 standard plots
Preprocessing – writing boilerplate for scaling, encoding
Notebook setup – creating the analysis environment

This isn't "work." It's tax. A tax you pay before you can do anything interesting.

The professional paradox: senior data scientists spend less time on EDA because they have their own scripts, templates, and muscle memory. Juniors spend more time—just when they need to focus on learning the actual data science.

I was in the second group. So I optimized.

The Solution: One Command, Zero Friction

I built kaggle-prep – a CLI tool that automates the entire EDA workflow from dataset download to production-ready notebook.

Installation

pip install kaggle-prep

Basic Usage

# Full EDA pipeline
kaggle-prep uciml/iris --all

# Quick profile only
kaggle-prep uciml/iris --profile

# Generate starter notebook
kaggle-prep uciml/iris --notebook

What Happens When You Run It

Here's the actual output from running kaggle-prep uciml/iris --all:

PS D:\Projects\kaggle> kaggle-prep uciml/iris --all
No local data found in 'data'. Initiating download...
Downloading 'uciml/iris' via kagglehub (Zero-Config mode)...
Download complete! Files saved to: data
Loaded: Iris.csv (150 rows, 6 columns)

===========================================================
DATA PROFILE SUMMARY
===========================================================

Dataset: uciml/iris
Shape: 150 rows x 6 columns
Memory: 0.01 MB
Duplicates: 0
Missing: 0 (0.00%)
Numeric: 5 | Categorical: 1

Within seconds, you have a complete data profile.

The Output Pipeline

The tool generates:

9 EDA Visualizations (automatically generated):

Distributions (histograms + KDE)
Skewness analysis
Q-Q plots for normality testing
Violin plots
Outlier summary
Correlation matrix
Cardinality analysis
Categorical bar charts

Here's the actual correlation matrix generated:

Key insights from the data:

PetalLengthCm and PetalWidthCm have a near-perfect correlation (0.96)
Id shows unexpected correlation with Petal features (0.88-0.90) – indicating potential data ordering bias
SepalWidthCm is negatively correlated with other features – a classic Iris dataset pattern

Skewness Analysis (automatically calculated):

Preprocessing Pipeline Code (auto-generated):

   # A complete, production-ready preprocessing script
   # Includes scaling, encoding, and split logic

Starter Jupyter Notebook (ready to run):
- All imports pre-configured
- Data already loaded
- Visualization code pre-written
- Modeling boilerplate ready

The Traction: Numbers Don't Lie

I quietly released this on PyPI without any marketing push. Here's the organic growth:

Metric	Value
Monthly Downloads	891
Last 7 Days	282
Yesterday	216
Python Versions	3.10, 3.11, 3.12, 3.13, 3.14
Platforms	Windows, macOS, Linux

Platform Distribution:

OS	Usage
Windows	~45%
Linux	~35%
macOS	~20%

Python Version Distribution:

Version	Usage
Python 3.11	~40%
Python 3.12	~30%
Python 3.10	~15%
Python 3.13+	~10%

User Base Demographics:

Windows users are the largest segment (likely students and corporate data scientists)
Linux users follow closely (Kaggle competitions, cloud environments)
macOS has a significant presence (data science community preference)

Technical Architecture: How It Works

The Pipeline

Dataset URL → Download via KaggleHub → Load & Validate → 
  → Profile Generation →
    → Statistical Analysis →
      → Visualization Generation →
        → Preprocessing Script Generation →
          → Notebook Generation →
            → All Outputs Saved

Key Technical Decisions

Zero-Config Download: Uses kagglehub to handle authentication automatically
Smart Profiling: Detects data types, calculates statistics, identifies outliers using IQR method
Visualization Engine: Generates 9 standardized plots using matplotlib + seaborn
Multi-Format Output: JSON profile, HTML report, PNG plots, Python script, Jupyter notebook

The Feedback Loop

I added a --feedback command that:

Opens a Google Form for structured feedback
Logs usage patterns anonymously
Helps prioritize the next features

kaggle-prep --feedback

Why This Matters (Not Just for Me)

For Students

Time saved: 2+ hours per project → more time to learn actual ML
Consistency: Your EDA quality is high from day one
Portfolio-ready: Auto-generated notebooks are production-quality

For Professionals

Scale: Run the same pipeline across 10+ datasets in minutes
Standardization: Consistent profiling across projects
Collaboration: Share the generated reports and notebooks

For Kagglers

Fast iteration: Test hypotheses faster with instant profiling
Reproducibility: Consistent starter code across competitions
Focus on modeling: Stop wasting time on EDA boilerplate

The Roadmap

I'm actively building the Pro version based on user feedback:

Feature	Status	Expected
Automated PDF Reports	In Development	October 2026
Auto-ML Baseline	In Development	November 2026
Competition Optimization	Planned	December 2026
Custom Visualization Config	Planned	January 2027

Try It Yourself

pip install kaggle-prep
kaggle-prep uciml/iris --all

GitHub: [Link to your repo]

PyPI: [Link to your package]

Issues/Feature Requests: [Link to your issues page]

The Bigger Picture

I built this to solve my own frustration. But the response tells me the frustration is universal.

The last month validated three things:

Automation wins: 891 people downloaded a tool that saves them 2+ hours per project
Quality matters: The generated profiles and visualizations are professional-grade
Distribution is everything: 4,000 impressions on a single LinkedIn post → 891 downloads

This is my first open-source project that actually serves a real user base. And I'm just getting started.