SmartDataCleaner is a professional tool to clean, normalize, and prepare CSV data. In this tutorial, we’ll go through building it step by step using Python, Pandas, and Tkinter with ttkbootstrap.
Step 1: Project Setup
We’ll need a few libraries:
pip install pandas numpy ttkbootstrap reportlab
Import the essentials:
import os, sys, threading, json
from datetime import datetime
import tkinter as tk
from tkinter import filedialog
import pandas as pd
import numpy as np
import re
import ttkbootstrap as tb
from ttkbootstrap.constants import *
Explanation:
pandas & numpy for data handling.
tkinter & ttkbootstrap for the GUI.
threading for background data cleaning.
json for exporting results.
Step 2: Globals and Utilities
Define global variables and helper functions:
stop_event = threading.Event()
cleanup_results = {}
log_file = os.path.join(os.getcwd(), "datacleaner.log")
def log_error(msg):
with open(log_file, "a", encoding="utf-8") as f:
f.write(f"[{datetime.now().isoformat()}] {msg}\n")
def clean_column_name(name):
name = name.strip().lower()
name = re.sub(r"[^\w\s]", "", name)
name = re.sub(r"\s+", "_", name)
return name
Explanation:
stop_event allows stopping the cleanup mid-process.
log_error writes errors to a log file.
clean_column_name converts column names to snake_case.
Step 3: GUI – Root Window
Create the main window:
app = tb.Window(themename="darkly")
app.title("SmartDataCleaner v2.0.0")
app.geometry("1100x650")
Explanation:
ttkbootstrap.Window provides a modern look.
darkly theme gives a dark interface.
Step 4: File Selection Section
Allow the user to select a CSV file:
file_path = tk.StringVar()
row1 = tb.Labelframe(app, text="Select CSV File", padding=10)
row1.pack(fill="x", padx=10, pady=6)
tb.Label(row1, text="File:", width=10).pack(side="left")
tb.Entry(row1, textvariable=file_path, width=60).pack(side="left", padx=6)
tb.Button(
row1,
text="📄 CSV File",
bootstyle="secondary",
command=lambda: file_path.set(filedialog.askopenfilename(filetypes=[("CSV Files", "*.csv")]))
).pack(side="left", padx=4)
Explanation:
Users can browse their system to pick a CSV.
The selected path is stored in file_path.
Step 5: Cleanup Controls
Add Start and Stop buttons:
row2 = tb.Labelframe(app, text="Cleanup Controls", padding=10)
row2.pack(fill="x", padx=10, pady=6)
start_btn = tb.Button(row2, text="🧹 CLEAN DATA", bootstyle="success")
stop_btn = tb.Button(row2, text="🛑 STOP", bootstyle="danger-outline", state="disabled")
start_btn.pack(side="left", padx=6)
stop_btn.pack(side="left", padx=6)
Explanation:
start_btn will start the cleaning process.
stop_btn can halt a running cleanup safely.
Step 6: Display Results
Use a Treeview to show column analysis:
row3 = tb.Labelframe(app, text="Cleanup Results & Suggestions", padding=10)
row3.pack(fill="both", expand=True, padx=10, pady=6)
cols = ("column", "original_type", "suggested_type", "cleaned_type", "missing_values", "duplicates_removed", "heuristic_score", "rename_suggestion")
tree = tb.Treeview(row3, columns=cols, show="headings")
for col in cols:
tree.heading(col, text=col.upper())
tree.column(col, width=130, anchor="w")
tree.pack(fill="both", expand=True)
Explanation:
Each column of the CSV is analyzed.
Results include type suggestions, missing values, duplicates, and heuristic scores.
Step 7: Heuristic Cleanup Engine
Define the logic for cleaning and scoring columns:
def heuristic_score(missing, duplicates, type_issue):
score = 0
score += min(30, missing * 2)
score += min(30, duplicates * 2)
score += 40 if type_issue else 0
return min(score, 100)
def assess_and_clean(df: pd.DataFrame):
results = []
for col in df.columns:
if stop_event.is_set():
return df, results
series = df[col]
orig_type = series.dtype
missing = series.isna().sum()
duplicates = series.duplicated().sum()
if pd.api.types.is_numeric_dtype(series):
suggested_type = "float"
coerced = pd.to_numeric(series, errors="coerce")
type_issue = coerced.isna().sum() > missing
cleaned_series = coerced.fillna(coerced.mean())
imputation = "mean"
else:
suggested_type = "string"
cleaned_series = series.astype("string")
mode = cleaned_series.mode()
cleaned_series = cleaned_series.fillna(mode[0] if not mode.empty else "")
type_issue = False
imputation = "mode"
df[col] = cleaned_series
cleaned_name = clean_column_name(col)
rename_suggestion = cleaned_name if cleaned_name != col else ""
score = heuristic_score(missing, duplicates, type_issue)
results.append({
"column": col,
"original_type": str(orig_type),
"suggested_type": suggested_type,
"cleaned_type": str(df[col].dtype),
"missing_values": int(missing),
"duplicates_detected": int(duplicates),
"heuristic_score": int(score),
"rename_suggestion": rename_suggestion,
"imputation_method": imputation
})
return df, results
Explanation:
Computes missing values, duplicates, type mismatches.
Fills missing numbers with mean and strings with mode.
Suggests column renames and calculates a heuristic score for data health.
Step 8: Start & Stop Cleanup
Run cleanup in a background thread:
def stop_cleanup():
stop_event.set()
stop_btn.config(state="disabled")
def run_cleanup():
path = file_path.get()
if not path:
return
stop_event.clear()
start_btn.config(state="disabled")
stop_btn.config(state="normal")
tree.delete(*tree.get_children())
df = pd.read_csv(path)
cleaned_df, results = assess_and_clean(df)
for r in results:
tree.insert("", "end", values=(
r["column"],
r["original_type"],
r["suggested_type"],
r["cleaned_type"],
r["missing_values"],
r.get("duplicates_detected", 0),
r["heuristic_score"],
r["rename_suggestion"]
))
start_btn.config(state="normal")
stop_btn.config(state="disabled")
start_btn.config(command=lambda: threading.Thread(target=run_cleanup, daemon=True).start())
stop_btn.config(command=stop_cleanup)
Explanation:
Uses a thread to avoid freezing the GUI.
Updates the Treeview with cleaned column information.
stop_cleanup allows user to cancel mid-process.
Step 9: Export Options
Export cleanup results as JSON, TXT, or PDF:
def export_json():
path = filedialog.asksaveasfilename(defaultextension=".json")
with open(path, "w", encoding="utf-8") as f:
json.dump(cleanup_results, f, indent=2)
def export_txt():
path = filedialog.asksaveasfilename(defaultextension=".txt")
with open(path, "w", encoding="utf-8") as f:
for r in cleanup_results.get("results", []):
f.write(f"{r}\n")
PDF export uses ReportLab to make a formatted report.
Step 10: Run the App
Finally, launch the GUI:
app.mainloop()
✅ Congratulations!
You now have a fully functional SmartDataCleaner GUI app that can:
Load CSVs
Clean missing/duplicate values
Suggest column renames
Score data health
Export results in JSON, TXT, or PDF
Top comments (0)