🧹 Data Cleaning Challenge with Pandas (Google Colab)

Naveen Gokul — Sat, 08 Nov 2025 05:12:16 +0000

🧹 Data Cleaning Challenge with Pandas (Google Colab)

Data cleaning is one of the most crucial steps in any data science or analytics project. In this challenge, I worked on a real-world dataset from Kaggle with over 100,000 rows, performing various Pandas operations to clean, preprocess, and prepare it for further analysis.

📂 Dataset Details

For this challenge, I selected the E-commerce Sales Dataset from Kaggle containing around 120,000 rows and 12 columns.

It includes data such as:

🧾 Order ID
👤 Customer Name
🛒 Product & Quantity
💰 Sales & Discount
🌍 Region
📅 Order Date

Before Cleaning:

Rows → 120,000
Columns → 12
File format → .csv

⚙️ Tools & Environment

Python 3
Google Colab
Libraries: Pandas, NumPy, Matplotlib


python
from google.colab import files
uploaded = files.upload()

import pandas as pd
df = pd.read_csv('ecommerce_sales.csv')

My Experience on NoSQL Data Analysis Using Dataset From Kaggle

Naveen Gokul — Sat, 08 Nov 2025 05:01:00 +0000

``# 🧩 MongoDB Atlas: Insert, Query, Update, Delete, and Export Data

Author: NAVEEN GOKUL S
Date: November 2025
Topic: Data Engineering Assignment — MongoDB CRUD Operations

🗂️ Step 1: Setting up MongoDB Atlas

Go to MongoDB Atlas.
Create a free cluster (use the Shared Tier option).
Under Network Access, add your IP:

Click Network Access → Add IP Address → Allow access from anywhere (0.0.0.0/0).
1. Create a database user and remember the credentials. Example:

` Username: 22cs098 Password: NAVEEN `

Once your cluster is ready, click “Connect → Connect using MongoDB Shell” and copy the connection string.

💻 Step 2: Connect from Mongo Shell

Open PowerShell or Command Prompt, then run:

`bash mongosh "mongodb+srv://m0.wpjmxqh.mongodb.net/" --apiVersion 1 --username 22cs098_db_user `

Then enter your password when prompted:

` Enter password: NAVEEN `

If connection succeeds, you’ll see:

` Atlas atlas-xxxx-shard-0 [primary]> `

📥 Step 3: Create a Database and Insert Records

Switch to a database (it will auto-create):

`javascript use businessDB `

Insert 10 sample business review records:

`javascript db.reviews.insertMany([ { "business_id": "B001", "name": "Cafe Aroma", "rating": 4.6, "review": "Good food and fast service!", "date": "2025-11-07" }, { "business_id": "B002", "name": "Pizza Palace", "rating": 4.8, "review": "Amazing crust and cheese quality!", "date": "2025-11-07" }, { "business_id": "B003", "name": "Tea Time", "rating": 4.2, "review": "Nice ambience and friendly staff.", "date": "2025-11-07" }, { "business_id": "B004", "name": "Sweet Treats", "rating": 3.9, "review": "Desserts were good but service was slow.", "date": "2025-11-07" }, { "business_id": "B005", "name": "Veggie Delight", "rating": 4.1, "review": "Healthy food with good taste.", "date": "2025-11-07" }, { "business_id": "B006", "name": "Burger Hub", "rating": 4.9, "review": "Best burgers ever!", "date": "2025-11-07" }, { "business_id": "B007", "name": "Ocean Dine", "rating": 4.7, "review": "Fresh seafood and great view.", "date": "2025-11-07" }, { "business_id": "B008", "name": "Spice Route", "rating": 3.8, "review": "Food was okay, but spicy.", "date": "2025-11-07" }, { "business_id": "B009", "name": "Bakers Street", "rating": 4.5, "review": "Good pastries and coffee.", "date": "2025-11-07" }, { "business_id": "B010", "name": "Quick Bite", "rating": 4.0, "review": "Good service and clean place.", "date": "2025-11-07" } ]) `

🔍 Step 4: Queries

🏆 4.1 Top 5 Businesses by Rating

`javascript db.reviews.find().sort({ rating: -1 }).limit(5) `

🔤 4.2 Count of Reviews Containing “good”

`javascript db.reviews.countDocuments({ review: /good/i }) `

🏪 4.3 Get Reviews for a Specific Business ID

`javascript db.reviews.find({ business_id: "B005" }) `

✏️ Step 5: Update and Delete

✏️ Update a Review

`javascript db.reviews.updateOne( { business_id: "B005" }, { $set: { rating: 4.3, review: "Updated: Great taste and fresh ingredients!" } } ) `

🗑️ Delete a Record

`javascript db.reviews.deleteOne({ business_id: "B010" }) `

📤 Step 6: Export Data to JSON/CSV

Exit Mongo shell:

`bash exit `

Then run the following from PowerShell (not inside mongosh) 👇

📄 Export as CSV

`bash mongoexport --uri="mongodb+srv://22cs098_db_user:NAVEEN@m0.wpjmxqh.mongodb.net/businessDB" --collection=reviews --type=csv --fields=business_id,name,rating,review,date --out=reviews.csv `

📦 Export as JSON

`bash mongoexport --uri="mongodb+srv://22cs098_db_user:NAVEEN@m0.wpjmxqh.mongodb.net/businessDB" --collection=reviews --out=reviews.json `

📊 Step 7: View the Exported Files

Open reviews.csv in Excel or VS Code.
Open reviews.json in any text editor.

✅ Step 8: Summary

Operation	Command Type	Example
Insert	`insertMany()`	Add 10 reviews
Query	`find()`, `countDocuments()`	Search data
Update	`updateOne()`	Modify rating/review
Delete	`deleteOne()`	Remove record
Export	`mongoexport`	CSV/JSON output

🎯 Final Thoughts

MongoDB Atlas makes it easy to:

Manage cloud-hosted databases
Perform CRUD operations
Export results in multiple formats

This project demonstrates all essential MongoDB operations — perfect for Data Engineering and Database Management learning tasks.

DEV Community: Naveen Gokul

🧹 Data Cleaning Challenge with Pandas (Google Colab)

🧹 Data Cleaning Challenge with Pandas (Google Colab)

📂 Dataset Details

⚙️ Tools & Environment

My Experience on NoSQL Data Analysis Using Dataset From Kaggle

🗂️ Step 1: Setting up MongoDB Atlas

💻 Step 2: Connect from Mongo Shell

📥 Step 3: Create a Database and Insert Records

🔍 Step 4: Queries

🏆 4.1 Top 5 Businesses by Rating

🔤 4.2 Count of Reviews Containing “good”

🏪 4.3 Get Reviews for a Specific Business ID

✏️ Step 5: Update and Delete

✏️ Update a Review

🗑️ Delete a Record

📤 Step 6: Export Data to JSON/CSV

📄 Export as CSV

📦 Export as JSON

📊 Step 7: View the Exported Files

✅ Step 8: Summary

🎯 Final Thoughts