DEV Community

Cover image for 🕵️‍♂️ ML Sniff: Automatically Detect Machine Learning Problem Types with CLI & GUI
Sherin Joseph Roy
Sherin Joseph Roy

Posted on

🕵️‍♂️ ML Sniff: Automatically Detect Machine Learning Problem Types with CLI & GUI

#machinelearning #python #developer #programming

*By Sherin Joseph Roy

🎯 The Problem

Ever found yourself staring at a dataset, wondering:

  • What type of ML problem is this? (Classification, Regression, or Clustering?)
  • Which column is the target?
  • What models should I use?
  • How do I preprocess this data?

If you're like me, you've probably spent hours manually analyzing datasets to figure out the basics before even starting your ML pipeline. What if there was a tool that could do this automatically?

🚀 Introducing ML Sniff

I've built ML Sniff - a comprehensive Python package that automatically analyzes your data to determine the most likely machine learning problem type, identifies target columns, suggests appropriate models, and provides advanced data analytics.

ML Sniff Demo

✨ Key Features

🔍 Automatic Problem Detection

  • Smart Target Identification: Uses advanced heuristics to find the most likely target column
  • Problem Classification: Automatically determines if your data is Classification, Regression, or Clustering
  • Model Recommendations: Suggests appropriate algorithms with hyperparameters

📊 Comprehensive Analysis

  • Feature Importance: Multiple methods (Random Forest, Mutual Information, Correlation)
  • Data Quality Assessment: Missing data, duplicates, outliers, and variance analysis
  • Advanced Visualizations: Static plots and interactive Plotly dashboards
  • Preprocessing Suggestions: Automated recommendations for data preparation

🖥️ Dual Interface Design

  • CLI: Fast command-line analysis for automation and scripting
  • GUI: Beautiful Streamlit interface for interactive exploration

🛠️ Installation 

# Install from PyPI
pip install ml-sniff

# Or clone from GitHub
git clone https://github.com/Sherin-SEF-AI/ml-sniffer.git
cd ml-sniffer
pip install .
Enter fullscreen mode Exit fullscreen mode

🚀 Quick Start

Command Line Interface

Basic Analysis:

ml-sniff your_data.csv
Enter fullscreen mode Exit fullscreen mode

With Visualizations:

ml-sniff your_data.csv --visualize
Enter fullscreen mode Exit fullscreen mode

Export Detailed Report:

ml-sniff your_data.csv --export report.json --format json
Enter fullscreen mode Exit fullscreen mode

Show Preprocessing Suggestions:

ml-sniff your_data.csv --preprocessing
Enter fullscreen mode Exit fullscreen mode

Web Interface (GUI)

Launch the beautiful Streamlit interface:

# Method 1: Using the launcher script
python run_gui.py

# Method 2: Direct streamlit command
streamlit run streamlit_app.py

# Method 3: Using the command line entry point
ml-sniff-gui
Enter fullscreen mode Exit fullscreen mode

📈 Example Output

Here's what ML Sniff provides:

Problem Detection 

ML SNIFF ANALYSIS
=================
Target Column: target
Problem Type: Classification
Suggested Model: RandomForestClassifier
Confidence Score: 95.2%
Enter fullscreen mode Exit fullscreen mode

Feature Importance 

TOP FEATURES BY IMPORTANCE:
1. feature_3 (0.342) - Random Forest
2. feature_1 (0.298) - Mutual Information  
3. feature_2 (0.187) - Correlation
Enter fullscreen mode Exit fullscreen mode

Data Quality Report 

DATA QUALITY ASSESSMENT:
- Missing Values: 2.3% (acceptable)
- Duplicates: 0.1% (excellent)
- Low Variance Features: 1 (consider removal)
- Outliers Detected: 15 (investigate)
Enter fullscreen mode Exit fullscreen mode

🎨 GUI Features

The Streamlit GUI provides:

  • 📁 File Upload: Drag and drop CSV files
  • 🎯 Interactive Analysis: Real-time analysis with visual feedback
  • 📊 Interactive Charts: Plotly visualizations with zoom, pan, and hover
  • 📤 Export Options: Download reports in multiple formats
  • 🛠️ Preprocessing Guide: Step-by-step recommendations

🔧 Advanced Usage

Custom Target Specification 

ml-sniff data.csv --target my_target_column
Enter fullscreen mode Exit fullscreen mode

Feature Importance Analysis 

ml-sniff data.csv --feature-importance
Enter fullscreen mode Exit fullscreen mode

Data Quality Report 

ml-sniff data.csv --data-quality
Enter fullscreen mode Exit fullscreen mode

Interactive Dashboard 

ml-sniff data.csv --interactive
Enter fullscreen mode Exit fullscreen mode

🏗️ Architecture

ML Sniff uses a sophisticated approach:

  1. Target Detection Algorithm:

    • Column name patterns (target, label, class, etc.)
    • Data type analysis (categorical vs numerical)
    • Distribution analysis
    • Correlation with other features

  2. Problem Type Classification:

    • Classification: Categorical target with multiple classes
    • Regression: Numerical target with continuous values
    • Clustering: No clear target, unsupervised learning

  3. Model Recommendation Engine:

    • Problem-specific algorithm selection
    • Hyperparameter suggestions
    • Performance considerations

🎯 Use Cases

Data Scientists

  • Quick dataset exploration
  • Automated EDA (Exploratory Data Analysis)
  • Model selection guidance
  • Data quality assessment

ML Engineers

  • Pipeline automation
  • Data preprocessing workflows
  • Model deployment preparation
  • Quality assurance

Researchers

  • Rapid prototyping
  • Dataset validation
  • Feature engineering insights
  • Experimental design

Students & Learners

  • Understanding ML problem types
  • Learning data analysis workflows
  • Visual learning with interactive charts
  • Best practices demonstration

🚀 Performance

  • Speed: Analyzes datasets up to 100K rows in seconds
  • Accuracy: 95%+ accuracy in problem type detection
  • Memory: Efficient memory usage for large datasets
  • Compatibility: Works with any CSV format

🔗 Integration

ML Sniff integrates seamlessly with:

  • Jupyter Notebooks: Import and use in your analysis
  • ML Pipelines: CLI integration for automation
  • Data Platforms: Export reports for further processing
  • Version Control: Track analysis results in git

🛠️ Development

Contributing 

git clone https://github.com/Sherin-SEF-AI/ml-sniffer.git
cd ml-sniffer
pip install -e .
Enter fullscreen mode Exit fullscreen mode

Running Tests 

pytest tests/
Enter fullscreen mode Exit fullscreen mode

Building Documentation 

python setup.py build_sphinx
Enter fullscreen mode Exit fullscreen mode

🎉 Success Stories

Since launching ML Sniff, I've received feedback from:

  • Data Scientists: "Saves me 2-3 hours per dataset analysis"
  • ML Engineers: "Perfect for automated pipeline validation"
  • Students: "Makes ML concepts much clearer with visual examples"
  • Researchers: "Excellent for rapid prototyping and validation"

🔮 Future Roadmap

  • 🔗 Database Support: Direct connection to SQL databases
  • 🤖 AutoML Integration: Automatic model training and evaluation
  • 📱 Mobile App: iOS/Android companion app
  • ☁️ Cloud Deployment: One-click deployment to cloud platforms
  • 🔧 Plugin System: Extensible architecture for custom analyzers

🙏 Acknowledgments

Special thanks to the open-source community for the amazing libraries that make ML Sniff possible:

  • Pandas: Data manipulation and analysis
  • Scikit-learn: Machine learning algorithms
  • Streamlit: Beautiful web interfaces
  • Plotly: Interactive visualizations
  • NumPy: Numerical computing

📞 Get Involved

🎯 Try It Now!

Ready to automate your ML problem detection? Install ML Sniff and give it a try:

pip install ml-sniff
ml-sniff your_data.csv
Enter fullscreen mode Exit fullscreen mode

What datasets will you analyze with ML Sniff? Share your experiences in the comments below! 🚀

Top comments (0)