DEV Community: Hari Venkatesh

Understanding 6 Common Data Formats in Data Analytics

Hari Venkatesh — Wed, 08 Oct 2025 13:44:43 +0000

Understanding 6 Common Data Formats in Data Analytics

In the world of data analytics, data comes in many shapes and formats. Choosing the right one can make your analytics pipeline faster, more efficient, and easier to maintain.

In this blog, let’s explore 6 commonly used data formats in data analytics with a simple example dataset.

📊 Example Dataset

Let’s consider a simple dataset of students and their marks:

Name	Register Number	Subject	Marks
Hari	101	Math	95
Vignesh	102	Science	88
Priya	103	English	92

1️⃣ CSV (Comma Separated Values)

📘 Explanation:

CSV is one of the simplest and most widely used formats. Each row represents a record, and columns are separated by commas. It’s easy to create, read, and import into tools like Excel or Python pandas.

🧾 Example (data.csv):

Name,RegisterNumber,Subject,Marks
Hari,101,Math,95
Vignesh,102,Science,88
Priya,103,English,92

2️⃣ SQL (Relational Table Format)

📘 Explanation:

SQL format represents data stored in relational tables. You can use CREATE TABLE and INSERT INTO commands to define and populate your dataset.

🧾 Example (data.sql):

CREATE TABLE Students (
  Name VARCHAR(50),
  RegisterNumber INT,
  Subject VARCHAR(50),
  Marks INT
);

INSERT INTO Students VALUES
('Hari', 101, 'Math', 95),
('Vignesh', 102, 'Science', 88),
('Priya', 103, 'English', 92);

3️⃣ JSON (JavaScript Object Notation)

📘 Explanation:

JSON is a lightweight, human-readable data format used extensively in APIs and NoSQL databases. Data is stored as key-value pairs, making it flexible and hierarchical.

🧾 Example (data.json):

[
  {
    "Name": "Hari",
    "RegisterNumber": 101,
    "Subject": "Math",
    "Marks": 95
  },
  {
    "Name": "Vignesh",
    "RegisterNumber": 102,
    "Subject": "Science",
    "Marks": 88
  },
  {
    "Name": "Priya",
    "RegisterNumber": 103,
    "Subject": "English",
    "Marks": 92
  }
]

4️⃣ Parquet (Columnar Storage Format)

📘 Explanation:

Parquet is an optimized columnar storage format used in big data frameworks like Apache Spark, Hadoop, and BigQuery. It compresses data efficiently and allows faster analytical queries by reading only the required columns.

🧾 Example Representation (Conceptual):

Columnar Storage:

Name:       ["Hari", "Vignesh", "Priya"]
RegisterNo: [101, 102, 103]
Subject:    ["Math", "Science", "English"]
Marks:      [95, 88, 92]

💡 Note: Parquet files are binary and not human-readable, but this is how they conceptually organize data for fast column-based access.

5️⃣ XML (Extensible Markup Language)

📘 Explanation:

XML stores data using custom tags, similar to HTML. It’s self-descriptive and widely used in configurations and web data interchange.

🧾 Example (data.xml):

<Students>
  <Student>
    <Name>Hari</Name>
    <RegisterNumber>101</RegisterNumber>
    <Subject>Math</Subject>
    <Marks>95</Marks>
  </Student>
  <Student>
    <Name>Vignesh</Name>
    <RegisterNumber>102</RegisterNumber>
    <Subject>Science</Subject>
    <Marks>88</Marks>
  </Student>
  <Student>
    <Name>Priya</Name>
    <RegisterNumber>103</RegisterNumber>
    <Subject>English</Subject>
    <Marks>92</Marks>
  </Student>
</Students>

6️⃣ Avro (Row-based Storage Format)

📘 Explanation:

Avro is a binary row-based storage format developed by Apache. It’s efficient for serialization and supports schema evolution, making it popular for data pipelines and streaming platforms like Kafka.

🧾 Example Schema and Data (Conceptual):

{
  "type": "record",
  "name": "Student",
  "fields": [
    {"name": "Name", "type": "string"},
    {"name": "RegisterNumber", "type": "int"},
    {"name": "Subject", "type": "string"},
    {"name": "Marks", "type": "int"}
  ]
}

Data (conceptually):

Hari, 101, Math, 95
Vignesh, 102, Science, 88
Priya, 103, English, 92

💡 Avro data is stored in binary format, but it always includes the schema definition for decoding.

🔍 Summary Comparison

Format	Type	Human Readable	Best Use Case
CSV	Row-based	✅ Yes	Simple data, spreadsheets
SQL	Relational	✅ Yes	Databases and queries
JSON	Semi-structured	✅ Yes	APIs, NoSQL data
Parquet	Columnar	❌ No	Big Data Analytics
XML	Tagged	✅ Yes	Data interchange, configs
Avro	Row-based (Binary)	❌ No	Data pipelines, Kafka

🚀 Final Thoughts

Each format has its own strengths —

Use CSV or SQL for small datasets.
Use JSON for flexible, nested data.
Use Parquet or Avro for big data analytics and storage efficiency.
Use XML when data needs strong structure and tagging.

Understanding these formats helps you pick the right tool for your data analytics workflows! 💡

✍️ Written by Hari Venkatesh

DataAnalytics #BigData #Cloud #DataFormats #Learning

Building a Simple Yelp-Style Dataset in MongoDB (Step-by-Step Guide)

Hari Venkatesh — Mon, 25 Aug 2025 04:28:14 +0000

When I first started exploring MongoDB, I wanted to try something practical instead of just going through documentation. So, I decided to build a mini Yelp-style dataset where businesses have reviews, and then run queries to analyze them.

This blog walks you through how I created the dataset, inserted it into MongoDB, and ran useful queries like finding top-rated businesses and analyzing reviews. 🚀

📌 Why This Project?

Data is everywhere, and being able to store, organize, and query it effectively is an essential skill for developers. MongoDB’s flexible schema makes it perfect for datasets like business reviews, where each review might have slightly different details.

📌 Deliverables

Here’s what I aimed to achieve in this project:

✅ Create a dataset of 25 businesses with reviews.
✅ Insert data into MongoDB (using mongosh).
✅ Run meaningful queries:

Find top 5 businesses by rating

Count reviews containing “good”

Fetch reviews for a specific business

Update and delete records

✅ Export data if needed for further analysis

📌 Step 1: Creating the Dataset

I prepared a dataset of 25 businesses, each with:

business_id
name
rating
review

Here’s a small sample from the dataset:

{ "business_id": "B1", "name": "Cafe Mocha", "rating": 4, "review": "Good coffee and snacks" }
{ "business_id": "B2", "name": "Pizza House", "rating": 5, "review": "Excellent pizza, very good taste!" }
{ "business_id": "B3", "name": "Burger Point", "rating": 3, "review": "Average burger but good fries" }

📌 Step 2: Setting Up MongoDB

I used MongoDB Compass for visualization and mongosh (MongoDB Shell) for running commands.

mongosh

Switch to a database (I called it yelp):

use yelp

📌 Step 3: Inserting Data

Using insertMany(), I inserted all 25 records:

db.reviews.insertMany([
{ "business_id": "B1", "name": "Cafe Mocha", "rating": 4, "review": "Good coffee and snacks" },
{ "business_id": "B2", "name": "Pizza House", "rating": 5, "review": "Excellent pizza, very good taste!" },
...
{ "business_id": "B25", "name": "Coffee Day", "rating": 3, "review": "Average coffee, good sitting place" }
])

📌 Step 4: Running Queries
🔹 1. Top 5 Businesses by Rating
db.reviews.aggregate([
{ $group: { _id: "$business_id", name: { $first: "$name" }, avgRating: { $avg: "$rating" } } },
{ $sort: { avgRating: -1 } },
{ $limit: 5 }
])

👉 This gave me a leaderboard of the highest-rated restaurants.

🔹 2. Count Reviews Containing the Word “Good”
db.reviews.countDocuments({ review: /good/i })

👉 MongoDB’s regex search made it super easy to analyze customer sentiment.

🔹 3. Get Reviews for a Specific Business
db.reviews.find({ business_id: "B2" })

👉 Perfect to pull all reviews for Pizza House. 🍕

🔹 4. Update a Review
db.reviews.updateOne(
{ business_id: "B5" },
{ $set: { review: "Service improved, food is good now" } }
)

🔹 5. Delete a Record
db.reviews.deleteOne({ business_id: "B25" })

📌 Step 5: Insights & Learnings

MongoDB made it very easy to store flexible data (reviews don’t need a rigid schema).

Queries like regex searches helped in basic sentiment analysis.

Aggregations are powerful for ranking and analytics.

This small project gave me confidence to handle real-world datasets.

📌 Next Steps

Add more fields like location, date, user_id.

Perform sentiment analysis on reviews.

Build a simple frontend to display results from MongoDB.

🚀 Final Thoughts

This project may look small, but it’s a great stepping stone for anyone starting with databases. By simulating a real-world use case (like Yelp), I not only practiced MongoDB commands but also learned how to analyze data effectively.

If you’re new to MongoDB, I highly recommend creating your own dataset and experimenting with queries. It’s one of the best ways to learn! 🙌