DEV Community

Damaa-C
Damaa-C

Posted on

Introduction to Linux for Data Engineers, Including Practical Use of Vi and Nano with Examples

#beginners #dataengineering #linux #tutorial

Overview

Linux is the backbone of most modern data engineering systems. From cloud servers and big data platforms to ETL pipelines and data warehouses, Linux provides the environment where data engineers build, deploy, and manage data workflows. This article introduces Linux from a beginner’s perspective, explains why it is important for data engineers, and demonstrates basic Linux usage with a strong focus on text editing using Vi and Nano.

This guide is written for beginners with no prior Linux experience required.

Why Linux Is Important for Data Engineers?

Most data engineering tools run on Linux. Tools such as Apache Hadoop, Spark, Kafka, Airflow, Docker, and Kubernetes are primarily designed for Linux environments.

Here’s why Linux matters:

  • Server dominance – Most servers in the cloud (AWS, Azure, GCP) run on Linux.
  • Performance and stability – Linux handles large-scale data processing efficiently.
  • Automation-friendly – Powerful command-line tools for scripting and scheduling jobs.
  • Open source – Free, customizable, and widely supported

As a data engineer, you will often:

  • Connect to Linux servers via SSH
  • Edit configuration files
  • Run data processing scripts
  • Monitor logs and system resources

Understanding Linux is therefore a core skill.

Understanding the Linux Terminal

The terminal, also called the command line or shell, allows you to interact with the Linux system by typing commands.
Example of terminal output;

damaris@ubuntu:~$
Enter fullscreen mode Exit fullscreen mode

This shows:

  • Username: damaris
  • Machine name: ubuntu
  • Current directory: ~ (home directory)

Basic Linux Commands for Beginners

1. Check for current directory

To check for current directory in the terminal, use the following command;

pwd
Enter fullscreen mode Exit fullscreen mode

Output: /home/damaris

2. List files and folders

To list files in terminal use this command;

ls # listing files
ls -l ## to list folders
Enter fullscreen mode Exit fullscreen mode

3. Create a directory

Use mkdir command to create a directory.

Example:

mkdir data_projects
Enter fullscreen mode Exit fullscreen mode

4. Navigate Between Directories

Use cd command to navigate through directories.
Let's use the file mkdir data_projects we created to navigate the directory.
For example;

cd data_projects
Enter fullscreen mode Exit fullscreen mode

To exit a directory, use cd ..
Example:

cd data_projects
ls #lists files in the directory
cd ..
Enter fullscreen mode Exit fullscreen mode

5. Create a File

Use touch command to create a file.
Example;

touch sample.txt
Enter fullscreen mode Exit fullscreen mode

6. View File Content

To view contexts or content of a file, use cat command.

cat sample.txt
Enter fullscreen mode Exit fullscreen mode

Why Text Editors Matter in Data Engineering

As a data engineer, you will constantly edit SQL scripts, Python files, Shell scripts and Configuration files (YAML, JSON, .conf)

Linux provides powerful terminal-based text editors. The most common are Vi/Vim and Nano.

Using the Nano Editor in Ubuntu (Detailed Beginner Guide)

What Is Nano?

Nano is a simple, beginner-friendly text editor that runs inside the Ubuntu terminal. Unlike Vi/Vim, Nano does not use modes, which makes it much easier for new Linux users to learn and use.

For data engineers, Nano is commonly used to:

  • Edit configuration files (.conf, .yaml, .json)
  • Write quick notes or scripts
  • Modify ETL pipeline settings
  • Edit files on remote Linux servers

Nano is preinstalled on most Ubuntu systems, so you don’t need to install anything.

Opening the Terminal in Ubuntu

Before using Nano, you need to open the terminal.

You can do this in any of the following ways:

Press Ctrl + Alt + T

Search for Terminal in the Applications menu

You will see something like:

damaris@ubuntu:~$
Enter fullscreen mode Exit fullscreen mode

This means you are in your home directory.
nano data_notes.txt

Creating a File Using Nano

To create a new file using Nano, type:

nano data_notes.txt
Enter fullscreen mode Exit fullscreen mode

Then press Enter.

What Happens Next?

If the file does not exist, Nano creates it.

If the file already exists, Nano opens it for editing.

You will now see the Nano editor screen.

Understanding the Nano Editor Interface

When Nano opens, the screen has three main parts:

1. Main Editing Area (Center)

This is where you type your text.

Example:

This file contains notes for our data engineering project.
Source: MySQL
Destination: Data Warehouse
Enter fullscreen mode Exit fullscreen mode

2. Status Bar (Bottom)

At the bottom of the screen, you’ll see something like:

^G Get Help   ^O Write Out   ^W Where Is   ^K Cut   ^X Exit
Enter fullscreen mode Exit fullscreen mode

The ^ symbol means the Ctrl key.

So:

^O means Ctrl + O

^X means Ctrl + X

This shortcut list is one of Nano’s biggest advantages.

3. File Name Display (Top)

At the top, Nano shows the file name you are editing:

GNU nano 6.2         data_notes.txt
Enter fullscreen mode Exit fullscreen mode

Typing Text in Nano

Nano starts in editing mode immediately.

You can begin typing right away without pressing any special keys.

Example:

ETL Pipeline Notes
------------------
Extract data from PostgreSQL
Transform data using Python
Load data into the warehouse
Enter fullscreen mode Exit fullscreen mode

There is no insert mode or command mode like in Vi.

Saving a File in Nano

To save your work:

Press Ctrl + O (Write Out)

Nano will ask:

File Name to Write: data_notes.txt

Press Enter

Your file is now saved.

Exiting Nano

To exit Nano:

Ctrl + X

If You Have Unsaved Changes

Nano will ask:

Save modified buffer?

Press Y → Save changes

Press N → Exit without saving

Press Ctrl + C → Cancel exit

Opening an Existing File with Nano

To edit an existing file:

nano data_notes.txt
Enter fullscreen mode Exit fullscreen mode

This opens the file so you can modify it.

Editing Text in Nano

Moving the Cursor

You can move around using:

  • Arrow keys ↑ ↓ ← →
  • Page Up / Page Down

Deleting Text

  • Backspace → Delete previous character
  • Delete key → Delete next character

Cutting and Pasting Text

Cut a Line
Ctrl + K

This cuts the entire line.

Paste a Line

Ctrl + U

This pastes the last cut text.

Searching for Text in Nano

To search within a file:
Ctrl + W

Type the word you want to find and press Enter.

Example:

warehouse
Enter fullscreen mode Exit fullscreen mode

Practical Example: Editing a Configuration File

Imagine you are a data engineer editing a pipeline configuration file.

Step 1: Open the file 

nano etl_config.conf
Enter fullscreen mode Exit fullscreen mode

Step 2: Add configuration details 

source_database=mysql
source_host=localhost
destination=warehouse
batch_size=500
Enter fullscreen mode Exit fullscreen mode

Step 3: Save and exit 

Ctrl + O → Enter

Ctrl + X
Enter fullscreen mode Exit fullscreen mode

Viewing the File from Terminal

After exiting Nano, you can confirm the file content using:

cat etl_config.conf
Enter fullscreen mode Exit fullscreen mode

Output:

source_database=mysql
source_host=localhost
destination=warehouse
batch_size=500
Enter fullscreen mode Exit fullscreen mode

Common Nano Shortcuts (Beginner Must-Know)

Shortcut Action
Ctrl + O Save file
Ctrl + X Exit Nano
Ctrl + K Cut line
Ctrl + U Paste
Ctrl + W Search
Ctrl + G Help

Using the Vi Editor in Ubuntu (Detailed Beginner Guide)

What Is Vi?

Vi is a powerful text editor available on almost every Linux system, including Ubuntu. Unlike Nano, Vi works using modes, which can feel confusing at first but make Vi extremely efficient once learned.

For data engineers, Vi is important because:

  • It is always available on servers (even minimal installs)
  • It is fast and lightweight
  • It is widely used for editing configuration files and scripts
  • Many tools default to Vi

If you connect to a remote Linux server, Vi is almost always there.

Opening the Terminal in Ubuntu

Open the terminal using:

Ctrl + Alt + T, or

Search for Terminal in Applications

You will see something like:

damaris@ubuntu:~$
Enter fullscreen mode Exit fullscreen mode

Opening or Creating a File with Vi

To open or create a file using Vi:

vi pipeline_config.txt
Enter fullscreen mode Exit fullscreen mode

If the file does not exist → Vi creates it

If the file exists → Vi opens it for editing

You are now inside the Vi editor.

Understanding Vi Modes (Very Important)

Vi has three main modes. Most beginner confusion comes from not knowing which mode they are in.

1. Normal Mode (Default)

This is the mode Vi starts in

Used for navigation and commands

You cannot type text here

If you try typing, nothing appears — this is normal.

2. Insert Mode

Used for typing text

You must enter this mode manually

To enter Insert mode:

i
Enter fullscreen mode Exit fullscreen mode

You will see something like:

-- INSERT --

at the bottom of the screen.

3. Command Mode

Used to save, quit, or exit without saving

Activated by typing : in Normal mode

Typing Text in Vi (Insert Mode)

Step-by-step example:

Open the file:

vi pipeline_config.txt
Enter fullscreen mode Exit fullscreen mode

Press:

i
Enter fullscreen mode Exit fullscreen mode

Type the text:


source_database=postgres
host=localhost
port=5432
destination=data_warehouse
Enter fullscreen mode Exit fullscreen mode

You are now editing the file.

Exiting Insert Mode

To stop typing and return to Normal mode:

Esc
Enter fullscreen mode Exit fullscreen mode

Always press Esc before saving or quitting.

Saving a File in Vi

Make sure you are in Normal mode (press Esc)

Type:

:w
Enter fullscreen mode Exit fullscreen mode

Press Enter

This saves the file but keeps Vi open.

Saving and Exiting Vi

To save and exit at the same time:

:wq
Enter fullscreen mode Exit fullscreen mode

Then press Enter.

Exiting Vi Without Saving

If you want to quit without saving changes:

:q!
Enter fullscreen mode Exit fullscreen mode

This is useful if you make a mistake.

Navigating Inside a File
Using Arrow Keys

Most Ubuntu versions support arrow keys for movement.

Using Vi Keys (Optional but Powerful)

h → left

l → right

j → down

k → up

Deleting Text in Vi

  • Delete a Character -x
  • Delete a Line- dd
  • Undo a Change- u

Searching for Text in Vi

To search for a word:

/warehouse
Enter fullscreen mode Exit fullscreen mode

Press Enter.

To move to the next match:

n
Enter fullscreen mode Exit fullscreen mode

Practical Example: Editing a Configuration File on a Server

Imagine you are logged into a production server.

ssh user@data-server-ip
cd /opt/etl/config
vi etl_config.conf
Enter fullscreen mode Exit fullscreen mode

Inside Vi, press i and add:

batch_size=1000
retry_count=3
log_level=INFO
Enter fullscreen mode Exit fullscreen mode

Then:

Esc
:wq
Enter fullscreen mode Exit fullscreen mode

This is a real-world daily task for data engineers.

Viewing the File After Exiting Vi

Back in the terminal:

cat etl_config.conf
Enter fullscreen mode Exit fullscreen mode

Output:

batch_size=1000
retry_count=3
log_level=INFO

Common Vi Commands (Beginner Cheat Sheet)

Command Action
i Insert mode
Esc Normal mode
:w Save
:q Quit
:wq Save and quit
:q! Quit without saving
dd Delete line
u Undo
/text Search

Through this article, we explored the importance of Linux in data engineering, practiced essential Linux commands, and demonstrated practical text editing using the Nano and Vi editors on Ubuntu. Nano provides a simple and beginner-friendly way to create and edit files, while Vi offers powerful features that are widely used in professional and production environments. Learning both editors prepares beginners for real-world tasks such as editing ETL configurations, scripts, and log files on local or remote servers.

In conclusion, mastering Linux basics along with Nano and Vi is a strong first step toward a successful data engineering journey. With continued practice, these skills become second nature and form the foundation for working with advanced data tools, automation, and large-scale data pipelines.

Top comments (0)