DEV Community: Gabriel Pacheco

Creating Partitions and Mounting File Systems in Linux

Gabriel Pacheco — Thu, 02 May 2024 12:40:49 +0000

Block devices, partition tables, filesystem formats, fdisk, mountpoints. There's a lot of concepts and commands that are involved and are required in the process of creating a partition in Linux and mounting it to a specific directory. It can be a bit overwhelming, but we are gonna try to dissect the task of creating a partition and mounting it step by step.

First of all, we'll start by adding a storage device (HDD, SSD) to our machine, in my case, I'm gonna be adding a VDI (virtualbox disk image) to my VM, but if you have linux as your primary OS, you can try adding a physical device, the process it's gonna be the same. Once we got that taken care of, we can start getting our hands dirty.

Listing block devices with `lsblk`

We'll start by listing our block devices with the command lsblk, if we only have one storage device plugged in, it's most probably gonna be identified in the system as sda, which is where our /boot partition and / partition are located. We are gonna be looking for sdb, we can also identify our block device by checking the size of it, in my case I added a 5GB VDI.

Partition tables and allocation of space with `fdisk`

Having checked that our storage device has been correctly identified by the OS, we can start by creating the partition table and allocating space to our partitions, in this demo I'll be creating two partitions under the same block device(sdb). To perform the creation of the partition table in the block device and the allocation of space in each partition we'll be using the command line tool fdisk. fdisk by default creates a mbr partion table on our block device, but it also allow us to create a gpt partitin table, so there's really no need to use gdisk, but both utilities are very similar.

What is MBR and GPT?
MBR and GPT are two different ways of structuring our partition tables, and a partition table essentially acts like a map, telling the operating system where partitions (sections for your operating system, data, etc.) are located on the disk.

MBR (Master Boot Record): The older standard partition table scheme with limitations of a maximum of four primary partitions and a 2 TB partition size limit.
GPT (GUID Partition Table): The newer and more advanced scheme offering theoretically unlimited partitions(128 technically, as we'll see later with the fdisk command), support for larger partition sizes, and error correction mechanisms.

Looks like a no brainer when it comes to deciding which one to use, but MBR for example has more compatibility with older operating systems, so it depends on what your needs are. For this demo, we'll choose GPT.

Now that we've discussed a bit of theory let's start creating our partition table and our partition with fdisk. You've probably heard multiple times that In Linux EVERYTHING is a file, well, storage devices are no exception. Our sdb block device file is stored under the directory /dev, so we'll provide that path to the fdisk command to modify it. We'll also need to provide root privileges to execute the command sudo fdisk /dev/sdb.

It will prompt us a welcome message, and then if we press m, it will provide us a list of actions to perform on our sdb disk.
As I said previously, we'll be using a GPT partition table, so we'll press g to create it under the Create a new label section.

Ouput:

We can press p to view our partition table and check if it has been created.

Output:

Now we can create our two partitions with n. First it will ask us the partition number that we want to grant between 1-128, since we don't have any partitions created yet we'll select 1 and 2 consecutively.

Then it will ask us for the first sector(sectors are the most atomic unit of storage in a storage device) where the partition will begin storing data. We can press enter, it will select the default value(the first sector available(2048)), we will do the same for the second partition(this will be a different sector number).

Finally it will ask for the last sector of the partition, since we're gonna create two, in the first partition we can input +2.5G to tell fdisk that we want a partition of 2.5G and then in the second partition we can just press enter and it will allocate the rest of the space left to that partition.

If we press p again we'll see both of our partitions created, sdb1 and sdb2:

After making all this changes on our sdb block device we can press w to write the partition table to the disk and exit.

Informing the OS of partion table changes

Now if we run a lsblk again we'll see that our partitions are created:

But before proceeding with the rest of the demo we'll have to inform the OS of the changes we've made with the sdb block device, and for that we'll use the command sudo partprobe /dev/sdb.

Filesystem formats

Now we are ready to add a filesystem format to each partition. We won't be diving a lot into filesystem formats, but essentially they are responsible for creating a filesystem structure on your storage device mapping your files data to your physical storage space and tracking the location and attributes of files (filename, size, permissions, etc.).

We will be creating both partitions with ext4 and xfs filesystem formats. Again, we won't be diving into the types of existing filesystem formats, but ext4 and xfs are the ones that are mostly used due to their journaling capabilities(they keep a track on disk of every filesystem modification before the actual change is made, in case a crash occurs and the state of the filesystem isn't lost) and also the size of partitions and file sizes they they are compatible with. You don't need to pay to much attention to this information, but knowing a bit more never hurts. My takeaway would be that ext4 is more focused on general purpose use and xfs could be better if you are handling very large files and require high-performance storage and scalability (big data, media transfer).

Now, getting back to the demo, we can make use of the mkfs command to add a filesystem format to our partitions. We will use sudo mkfs.ext4 /dev/sdb1 and sudo mkfs.xfs /dev/sdb2 commands to create our file system formats in each partition. You'll see how the outputs of each command are different depending on the filesystem you chose.

Mounting file systems

Now that we've created our partition table, our partitions and our filesystem formats in each partition we can mount the filesystems or map them to a specific path in our system. For this we'll create two folders in our /mnt directory that will be our mountpoints, /mnt/data and /mnt/backup.

To mount our partitions in these folders we'll make use of the command mount. We'll be using the xfs filesystem for the data directory and the ext4 filesystem for the backup directory, for that we'll use these commands: sudo /dev/sdb1 /mnt/backup/ and sudo /dev/sdb2 /mnt/data/. If the commands don't output anything the filesystem mounting will be done. We can check how our partitions point to these directories running lsblk.

Now we'll be able to store data both on the /mnt/backup and /mnt/data directories, but these mountpoints are temporary if we want to mount them permanently we'll have to modify the /etc/fstab file and add an entry for each mounting point.

Before entering this file to modify it we'll need the UUIDs of each filesystem partition. For this we'll use the command blkid.
We can run sudo blkid /dev/sdb1 and sudo blkid /dev/sdb2 to obtain this parameters.

Now we can modify the /etc/fstab file and add the entries for each partition. The entries are structured on the following way:

1st field: UUID of the partition filesystem, in my case the first UUID of the first entry belongs to the /dev/sdb1 partition. Make sure that this UUID matches the correct mountpoint, you can view this using lsblk if you have forgotten which partition maps to which mounpoint.
2nd field: mountpoint
3rd field: filesystem format
4th field: This refers to a predefined set of commonly used mount options. These typically include:

rw: Allows both reading and writing from the filesystem (read-write access).
user: Allows regular users (not just root) to mount and unmount the filesystem.
auto: Instructs the system to automatically mount the filesystem at boot time.

5th field: This represents the dump frequency and is typically used by the dump command, an older utility for system backups. In modern systems with journaling filesystems (like ext4), this field often holds less significance. A value of 0 here indicates that the filesystem should not be included in routine backups performed by dump.
6th field: This represents the filesystem check order during the fsck process, a utility that checks filesystems for errors. With journaling filesystems, the order might be less crucial as the journal itself helps maintain consistency. A value of 0 here often signifies that the filesystem should not be automatically checked by fsck at boot time.

After adding both entries we can save and exit the /etc/fstab file and run the mount -a command to automatically mount all filesystems specified in the /etc/fstab file. If we have no output after executing it, we've successfully mounted our file systems permanently.

Understanding SSH Tunneling

Gabriel Pacheco — Tue, 30 Apr 2024 12:34:46 +0000

What is SSH Tunneling and why is it useful?

Usually when we think of ssh, we think about logging into a remote machine securely using a public/private key pair to execute commands, change configuration files, setup a web server or perform different actions.

In the case of SSH Tunneling the same thing happens, but with a slight difference. SSH Tunneling also involves establishing a connection between a local machine and a remote server, but the main purpose of this connection is not to directly gain access to the remote server. The remote server acts as an intermediary to forward access to a specific port or host.

Let's say you are working at home using your PC, but you want to gain access, to a web server, a database or service that is running locally on another private network. Maybe this service that you are trying to gain access to is running on the remote machine, but it's not exposed to the internet, it's running on localhost for security reasons.

How can you gain access to it then? This is where SSH Tunneling comes into play allowing you to connect to local resources through a "tunnel". As mentioned before, the remote server takes care of forward your request to a specific host and port. Let's dive more into it.

Local SSH Tunnels and Remote SSH Tunnels

There's two different types of ssh tunnels that we'll be exploring and that can be used depending on what your needs are.

Local/Forward SSH Tunnels

Essentially what we achieve by setting up a local ssh tunnel between two hosts, is mapping a specific port in our local machine to a service that is running in the remote server locally with a reserved port, this can be helpful if we want to for example administer a database that is running in a remote machine, but it's not exposed to the internet. It's worth mentioning, sticking to our database example, that the database doesn't need to be run in the remote machine that we are establishing the ssh connection with, the remote server as long as it has access to other machines inside the same private network it can forward the request to that specifig machine.

In the example above we're mapping port 3333 on localhost to port 3306
on the local machine with ip 192.168.1.55 that is running on the same network as the remote machine(44.65.167.34), which is the one responsible to forward our traffic and act as an intermidiary between both, otherwise traffic between this two local machines wouldn't be possible.

All this process is accomplished by executing the following command in local machine:

ssh -N -L 127.0.0.1:3333:192.168.1.1 44.65.167.34

With the -N flag we are specifying to not log in the remote machine shell and execute any commands and with the -L flag we are specifying that we want to perform a local ssh tunnel.

Remote/Reverse SSH Tunnels

Remote ssh tunnels are slightly complex to grasp in comparison to local ssh tunnels, but deep down, the functionality behind it, still remains the same, you still have a remote server forwarding traffic from A to B, that's basically it.

Let's say you're running a web server locally, that is not exposed to the internet at home and you want to gain access to it, but at this moment you are at your workplace. If you have a home server setup at your place you can still access your web server, since they both belong to your home private network, even though as previously mentioned, the web server is running on localhost.

To Accomplish this we will open a port in our home server that then will be mapped directly to the local machine and port that our web server is running on. This way everytime we access our home server on that specific port we will be redirected to our web server that is running locally. Anyone can access your web server this way, so it's not recommended to do this, this is just an example, but you can also think about establishing an ssh connection to your local machine located at home, the functionality is the same, in this case you would establish a ssh connection to your home server using the port that is mapped to your local machine.

In the example above, the remote machine is sending a request to the remote server ip address on port 8888, and then the server is forwarding the traffic to the host with which it's connected to via ssh.

To setup a remote ssh tunnel and allow this kind of traffic we will have to gain access to the /etc/ssh/sshd_config file and set the parameter GatewayPorts to yes(you can also uncomment it) in the remote server and then run the following command in your local machine in which your web server is running (your web server could be running on another local machine and you can still forward the traffic to it, since they are in the same network, but for the sake of the example we will do it this way):

ssh -N -R 8888:localhost:80 remote-server-ip