Setting up a shared volume for your docker swarm using GlusterFs

Mohammed Ashour — Sat, 10 Apr 2021 13:57:42 +0000

Working on a distributed software product before containers is certainly different than how it's now after the sweet introduction of the world of containerization.

And without saying, for most of the community out there, when is containerization is mentioned, Docker is one of the first things that pop in their heads if it wasn't the first!
I'm not here to argue that others started the idea of containerization before docker, that may be the topic of another blog.

I'm here to talk about a problem I faced -and I'm sure others did- while dealing with Docker Swarm and needing some sort of data sharing between the nodes of the swarm. Unfortunately, this is not natively supported in Docker, you need to rely on a third-party storage service that provides you a good API that your nodes deal with and it will cost you a good deal of money for it, or you can go die-hard and make your own shared storage yourself. and here, I'm sharing the die-hard way that I chose to do it.

Introduction:

What is GlusterFS?

Gluster is a scalable, distributed file system that aggregates disk storage resources from multiple servers into a single global namespace.
Gluster is Open source, Provides replication, quotas, geo-replication, snapshots, and others.

Gluster gives you the ability of aggregating multiple nodes into one namespace, and from here you got multiple options.
You can simply

Have a replicated volume that offers the availability for your data without having to worry about your data loss
Have a distributed volume that can aggregate space for data by dividing your data across multiple machines, you here lose the availability but you gain more storage with the same resources

You can learn about the various setups of Gluster from here
https://docs.gluster.org/en/latest/Quick-Start-Guide/Architecture/

Let's start with our setup

First, let's imagine that we have a swarm setup of 3 manager nodes and 3 worker nodes, and we need the containers on worker nodes to see the same data wherever they are, also, we need consistency for our data, the availability for our data matters the most. So, we need to make a replicated GlusterFs volume, that replicates all the data in multiple replication nodes, and since we don't have many resources to get extra machines that can act as storage pools, we will use our swarm machines to act also as storage pool nodes.

So, our architecture will be something like this

3 Swarm Managers
3 Swarm Workers
GlusterFs Volume connected to the 3 servers

Yes, I know you are wondering now, this design makes our workers act as storage pool and clients at the same time, so they act as a replication for the data collected, and also works as servers that mount this volume and read from it, this may seem weird at first, but if you think about it, this can be a very good setup for a lot of use cases that your application may need. and it doesn't cost you any extra!

let's start the dirty work

let's assume we are working with a Debian based distro, like ubuntu

first, you will need to install glusterfs on all three worker machines
you can do this by

sudo apt update
sudo apt install software-properties-common
sudo add-apt-repository ppa:gluster/glusterfs-7
sudo apt update
sudo apt install glusterfs-server

then, you will need to start and enable the glusterfs daemon service on all the worker machines

sudo systemctl start glusterd
sudo systemctl enable glusterd

then, make sure you generate an ssh-key on each machine

ssh-keygen -t rsa

after that, to be able to deal with all machines with their hostnames, you will need to edit the /etc/hosts for each of these machines and add the hostnames for the other nodes assigned to their IPs, like this format

<IP1> <HOSTNAME1>
<IP2> <HOSTNAME2>

Now, let's create our storage cluster, start from one of the machines and add the others using this command

sudo gluster peer probe <Hostname>

after you add the 2 nodes, run this command to make sure that all of the nodes joined the storage cluster

sudo gluster pool list

now, we will need to make a directory in all of the 3 workers to act like a brick for our volume
But wait, what is a brick??
a brick is basically a directory that acts as a volume unit, GlusterFs uses it in all of its storage pool nodes to know where is the data to store/deal with

so basically you will need to create this directory on each worker node, let's name it brick1 and put it under /gluster-volumes

sudo mkdir -p /gluster-volumes/brick1

Now, we are ready to create our replicated volume (let's call it demo-v )
[run this command only on the main machine]

sudo gluster volume create demo-v replica 3 <HOSTNAME1>:<BRICK_DIRECTORY> <HOSTNAME2>:<BRICK_DIRECTORY> <HOSTNAME3>:<BRICK_DIRECTORY> force

then start the volume

sudo gluster volume start demo-v

and Congrats, you have a replicated volume now that ready to be mounted and used in any machine
Now, let's mount this volume on our 3 workers
let's say that we will mount our volume under /our-application/logs

sudo mount.glusterfs localhost:/demo-v /our-application/logs/

then, to make it permanent, we will need to add it to our fstab file
so open /etc/fstab and add this line


localhost:/demo-v /our-application/logs/ glusterfs defaults,_netdev,acl, backupvolfile-server=<HOSTNAME> 0 0

instead of the <HOSTNAME> add a hostname of one of the worker machines you have, so if you needed to get it out of the storage pool, you can still mount the volume using the other machine.

Now you can try this

touch /our-application/logs/test.txt

and check out the other workers, find the file? Congrats! you have a working replicated volume that is mounted across all of your workers.
Any questions? leave a comment!

The Talkative creatures inside the silent beings' head (The SW Engineer mental health problems)

Mohammed Ashour — Sat, 16 Jan 2021 15:11:13 +0000

You wake up every day, checking your backlog, your tasks, your issues that are waiting for you to solve, it's exciting somedays, boring others, painful others, but most of the time you kinda like it, it's telling the story of you, your effect in the world. The solutions that you thought of, are making an impact somewhere for someone. This is exciting, yeah it is.

Let me introduce myself, I'm a software engineer writing solutions in the Data and ML space, I'm also a researcher who tries so hard to find a problem so I can propose some solution to it, exciting life, isn't it?

Beyond my tech intro, I kinda suck! who else am I? Is there another intro that I can tell others about myself besides my magical solution that I wrote to solve the problem we faced in the production? or that solution that made our product faster? or that open source project that I contributed to? What are my hobbies? What are hobbies? are they some sort of problems we can fix? If they aren't, how I deal with them? something is missing.

Dealing with silent creatures too much makes another one

Every day, I open my eyes, get my dose of caffeine, and start my journey this day with my everyday pal, my laptop, we are used to be silent, actually, we excel at it.
TBH, it taught me how to be good at it. Dealing with a silent machine made me a one, and made me prefer their existence, my laptop, my phone, my books, I like spending my time with them, and they never complain.
This isn't a problem? not for me at least, but for the ones inside my head, I don't think they like it.

Dealing with silent creatures too much created talkative ones trapped in my head

I think this created some conflicts in my own being, being that social creature that prefers to be silent most of the time is kinda contradicting.
Don't get me wrong, I don't mean that being silent is the problem, Personally, I think it's great not to be that annoying talkative person, but the thing I pointing to is being not able to express yourself, not able to talk out loud outside your inner zone, outside your inner world, with people that are outside your head.
Yeah, I know you have these people that you are talking to them in your head right now, I know that because I have similar people of my own, we discuss everything that I can't discuss out loud, and these things are a lot.

The life problem of being trapped

Can you stop a moment and try to think of a good hobby you enjoy and practice?
if you found one, good for you! if you haven't, what do you feel?
you can ask your inner creatures now about how do they think about it? let me share with you what they are discussing now inside my head.
One of them thinks that there is no problem here, asks the question of what are hobbies anyway? are they that important? what does it affect our life?
another one replies that he thinks the main issue that we struggle to define ourselves outside being an SW Engineer, he thinks that there would be a problem if we quit this, a life problem of what are we now, and what do we do?
another one thinks that may be the answer to all of this is to not think about it, maybe we should just keep ourselves busy with what we have, and then we won't have much time or energy to discuss this.
There are other ones, but their ideas can't be shared without being concerned about how badly they can affect you as a reader, so let's skip them

So, What are you outside your commits?

For me, I don't have an answer yet, maybe that is the middle age crisis that everybody is talking about, don't get me wrong I'm not the old, but maybe my folks inside my head are having one now.
But what about you? Is there anything going on in your life outside your commits? so If I have gone through all your commits will I know the most about you and your life? or won't I?

Why?

I don't know, I don't know why we should talk about it, and I don't know why I wrote this, maybe there is a different being in my head, one who is old enough to know much about who I was and wondering now about who am I. I couldn't answer, so I tried to solve it, I searched StackOverflow, found nothing! So I thought maybe should be writing about it.

DEV Community: Mohammed Ashour