In the world of data analytics, information can come in many formats. Each format serves different purposes—some are human-readable, others are optimized for storage or speed. In this article, we’ll explore six popular data formats used in analytics: CSV, SQL, JSON, Parquet, XML, and Avro. We’ll use a simple dataset to demonstrate each format.
Sample Dataset
[{'Name': 'Dhanyaa', 'Register_No': 'KPR23CB007', 'Subject': 'Data Analytics', 'Marks': 92}, {'Name': ’Krishna, 'Register_No': 'KPR23CB009', 'Subject': 'Cloud Computing', 'Marks': 88}, {'Name': 'Aarav', 'Register_No': 'KPR23CB011', 'Subject': 'AI & ML', 'Marks': 95}]
1. CSV (Comma Separated Values)
CSV is one of the simplest and most widely used data formats. It stores data in plain text, where each line represents a record and columns are separated by commas.
Name,Register_No,Subject,Marks
Dhanyaa,KPR23CB007,Data Analytics,92
Krishna,KPR23CB009,Cloud Computing,88
Aarav,KPR23CB011,AI & ML,95
2. SQL (Relational Table Format)
SQL databases store data in tables with defined columns and rows. You can create, read, update, and delete records using SQL queries.
CREATE TABLE students (
Name VARCHAR(50),
Register_No VARCHAR(20),
Subject VARCHAR(50),
Marks INT
);
INSERT INTO students VALUES
('Dhanyaa', 'KPR23CB007', 'Data Analytics', 92),
(Krishna, 'KPR23CB009', 'Cloud Computing', 88),
('Aarav', 'KPR23CB011', 'AI & ML', 95);
3. JSON (JavaScript Object Notation)
JSON is a lightweight data-interchange format that’s easy for humans to read and machines to parse. It’s widely used in APIs and data transmission.
{
"students": [
{"Name": "Dhanyaa", "Register_No": "KPR23CB007", "Subject": "Data Analytics", "Marks": 92},
{"Name": "Krishna", "Register_No": "KPR23CB009", "Subject": "Cloud Computing", "Marks": 88},
{"Name": "Aarav", "Register_No": "KPR23CB011", "Subject": "AI & ML", "Marks": 95}
]
}
4. Parquet (Columnar Storage Format)
Parquet is a columnar storage format optimized for big data processing frameworks like Apache Spark. It stores data by columns instead of rows, making queries faster for analytical workloads.
Example representation (simplified for illustration):
| Column Name | Values |
|--------------|--------------------------|
| Name | Dhanyaa, Krishna, Aarav |
| Register_No | KPR23CB007, KPR23CB009, KPR23CB011 |
| Subject | Data Analytics, Cloud Computing, AI & ML |
| Marks | 92, 88, 95 |
5. XML (Extensible Markup Language)
XML uses custom tags to define and structure data. Although more verbose, it’s useful for hierarchical data representation and data exchange.
Dhanyaa
KPR23CB007
Data Analytics
92
Krishna
KPR23CB009
Cloud Computing
88
Aarav
KPR23CB011
AI & ML
95
6. Avro (Row-Based Storage Format)
Avro is a binary row-based format developed under Apache Hadoop. It stores data along with its schema, which makes it efficient for serialization.
Schema Example:
{
"type": "record",
"name": "Student",
"fields": [
{"name": "Name", "type": "string"},
{"name": "Register_No", "type": "string"},
{"name": "Subject", "type": "string"},
{"name": "Marks", "type": "int"}
]
}
Data Example (in JSON-like representation):
{"Name": "Dhanyaa", "Register_No": "KPR23CB007", "Subject": "Data Analytics", "Marks": 92}
{"Name": "Krishna", "Register_No": "KPR23CB009", "Subject": "Cloud Computing", "Marks": 88}
{"Name": "Aarav", "Register_No": "KPR23CB011", "Subject": "AI & ML", "Marks": 95}
Conclusion
Each data format serves a unique purpose depending on the use case. While CSV and JSON are great for readability, Parquet and Avro are more efficient for large-scale analytics. Understanding these formats helps data professionals choose the right tools for data storage, transfer, and processing.
Top comments (0)