Data in the Cloud: 6 Common Data Formats Every Data Analyst Should Know

In today’s data-driven world, information is stored, processed, and shared across multiple systems and cloud platforms. But have you ever wondered how this data is actually structured?

Different tools and systems prefer different data formats — each designed with a specific purpose. Whether you’re analyzing sales data in Python, querying with SQL, or running big data pipelines in Spark, understanding these formats will make your life much easier.

Let’s explore six of the most commonly used data formats in analytics:

1. CSV (Comma Separated Values)
2. SQL (Relational Table Format)
3. JSON (JavaScript Object Notation)
4. Parquet (Columnar Storage Format)
5. XML (Extensible Markup Language)
6. Avro (Row-based Storage Format)

Example Dataset
To make things concrete, let’s use a simple dataset representing students and their marks:

Name    Register_No Subject Marks
Alice   101 Math    88
Bob 102 Science 92
Charlie 103 English 79

We’ll represent this same dataset in six formats below.

1. CSV (Comma Separated Values)

What it is:
CSV is the simplest and most human-readable data format. Each line represents a record, and values are separated by commas.

Where it’s used:
Spreadsheets, Python data analysis (pandas), Excel, and cloud storage exports.

Example:

Name,Register_No,Subject,Marks
Alice,101,Math,88
Bob,102,Science,92
Charlie,103,English,79

*Pros:
*
Easy to read and create
Supported by almost every tool

Cons:
No data types (everything is text)
Doesn’t handle nested data well

2. SQL (Relational Table Format)

What it is:
SQL is used to store data in relational databases (like MySQL, PostgreSQL, or SQLite). Data is organized into tables with rows and columns.

Where it’s used:
Database systems, analytics platforms, and data warehouses.

Example:

CREATE TABLE Students (
  Name VARCHAR(50),
  Register_No INT,
  Subject VARCHAR(50),
  Marks INT
);

INSERT INTO Students (Name, Register_No, Subject, Marks) VALUES
('Alice', 101, 'Math', 88),
('Bob', 102, 'Science', 92),
('Charlie', 103, 'English', 79);

Pros:
Strong schema and structure
Easy querying with SQL language

Cons:
Not flexible for nested or complex data

3. JSON (JavaScript Object Notation)

What it is:
JSON is a lightweight format for structured data. It represents objects as key-value pairs and is widely used in web APIs and NoSQL databases.

Where it’s used:
APIs, web apps, NoSQL systems (MongoDB, Firebase).

Example:

[
  { "Name": "Alice", "Register_No": 101, "Subject": "Math", "Marks": 88 },
  { "Name": "Bob", "Register_No": 102, "Subject": "Science", "Marks": 92 },
  { "Name": "Charlie", "Register_No": 103, "Subject": "English", "Marks": 79 }
]

Pros:
Flexible and supports nested data
Readable and language-independent

Cons:
Larger file size than CSV
Slower for big data analytics

4. Parquet (Columnar Storage Format)

What it is:
Parquet is a binary, columnar data format optimized for analytical workloads. Instead of storing row by row, it stores data column by column — making queries on specific fields lightning-fast.

Where it’s used:
Big data systems like Apache Spark, Hive, AWS Athena, and Google BigQuery.

Example (conceptual view):

# Parquet is binary, but conceptually looks like this:

Column: Name        → [Alice, Bob, Charlie]
Column: Register_No → [101, 102, 103]
Column: Subject     → [Math, Science, English]
Column: Marks       → [88, 92, 79]

Pros:
Highly compressed and efficient
Ideal for large datasets and analytics

Cons:
Not human-readable
Harder to debug manually

5. XML (Extensible Markup Language)

What it is:
XML uses tags to define structured data, similar to HTML. It’s widely used in configuration files, legacy systems, and data interchange.

Where it’s used:
Web services (SOAP), enterprise applications, configuration files.

Example:

<Students>
  <Student>
    <Name>Alice</Name>
    <Register_No>101</Register_No>
    <Subject>Math</Subject>
    <Marks>88</Marks>
  </Student>
  <Student>
    <Name>Bob</Name>
    <Register_No>102</Register_No>
    <Subject>Science</Subject>
    <Marks>92</Marks>
  </Student>
  <Student>
    <Name>Charlie</Name>
    <Register_No>103</Register_No>
    <Subject>English</Subject>
    <Marks>79</Marks>
  </Student>
</Students>

Pros:
Structured and self-descriptive
Supports nested and hierarchical data

Cons:
Verbose and larger file sizes
Harder to parse compared to JSON

6. Avro (Row-based Storage Format)

What it is:
Avro is a row-based binary format developed under Apache Hadoop. It stores both the data and schema together — great for streaming and serialization.

Where it’s used:
Apache Kafka, Hadoop, and data pipelines.

Example (conceptual view):

{
  "type": "record",
  "name": "Student",
  "fields": [
    {"name": "Name", "type": "string"},
    {"name": "Register_No", "type": "int"},
    {"name": "Subject", "type": "string"},
    {"name": "Marks", "type": "int"}
  ]
}

# Data:
[
  {"Name": "Alice", "Register_No": 101, "Subject": "Math", "Marks": 88},
  {"Name": "Bob", "Register_No": 102, "Subject": "Science", "Marks": 92},
  {"Name": "Charlie", "Register_No": 103, "Subject": "English", "Marks": 79}
]

Pros:
Schema evolution support
Compact and fast for streaming

Cons:
Binary (not human-readable)
Needs libraries to read/write

Final Thoughts

• Each of these formats serves a specific purpose.
• Use CSV when you need simplicity.
• Choose JSON for flexibility.
• Rely on SQL for structured, relational data.
• Go with Parquet or Avro for big data or cloud-based analytics.
• And if you’re maintaining older systems, XML might still be your go-to.
• Understanding these formats helps you move data seamlessly across systems — a crucial skill in modern cloud-based analytics.