One of my most favourite things about open source files on GitHub is the ability to see how others do (what some people might call) mundane things, like set up their .bashrc and other dotfiles. While I’m not as enthusiastic about ricing as I was when I first came to the Linux side, I still get pretty excited when I find a config setting that makes things prettier and faster, and thus, better.
I recently came across a few such things, particularly in Tom Hudson’s dotfiles. Tom seems to like to script things, and some of those things include automatically setting up symlinks, and installing Ubuntu repository applications and other programs. This got me thinking. Could I automate the set up of a new machine to replicate my current one?
Being someone generally inclined to take things apart in order to see how they work, I know I’ve messed up my laptop on occasion. (Usually when I’m away from home, and my backup harddrive isn’t.) On those rare but really inconvenient situations when my computer becomes a shell of its former self, (ba-dum-ching) it’d be quite nice to have a fast, simple way of putting Humpty Dumpty back together again, just the way I like.
In contrast to creating a disk image and restoring it later, a collection of bash scripts is easier to create, maintain, and move around. They require no special utilities, only an external transportation method. It’s like passing along the recipe, instead of the whole bundt cake. (Mmm, cake.)
Additionally, functionality like this would be super useful when setting up a virtual machine, or VM, or even just a virtual private server, or VPS. (Both of which, now that I write this, would probably make more forgiving targets for my more destructive experimentations… live and learn!)
This post will cover how to achieve the automatic set up of a computer running Ubuntu Desktop (in my case, Ubuntu LTS 18.04) using bash scripts. The majority of the information covered is applicable to all the Linux desktop flavours, though some syntax may differ. The bash scripts cover three main areas: linking dotfiles, installing software from Ubuntu and elsewhere, and setting up the desktop environment. We’ll cover each of these areas and go over the important bits so that you can begin to craft your own scripts.
Dotfiles
Dotfiles are what most Linux enthusiasts call configuration files. They typically live in the user’s home directory (denoted in bash scripts with the builtin variable $HOME) and control the appearance and behaviour of all kinds of programs. The file names begin with ., which denotes hidden files in Linux (hence “dot” files). Here are some common dotfiles and ways in which they’re useful.
.bashrc
The .bashrc file is a list of commands executed at startup by interactive, non-login shells. Interactive vs non-interactive shells can be a little confusing, but aren’t necessary for us to worry about here. For our purposes, any time you open a new terminal, see a prompt, and can type commands into it, your .bashrc was executed.
Lines in this file can help improve your workflow by creating aliases that reduce keystrokes, or by displaying a helpful prompt with useful information. It can even run user-created programs, like Eddie. For more ideas, you can have a look at my .bashrc file on GitHub.
.vimrc
The .vimrc dotfile configures the champion of all text editors, Vim. (If you haven’t yet wielded the powers of the keyboard shortcuts, I highly recommend a fun game to learn Vim with.)
In .vimrc, we can set editor preferences such as display settings, colours, and custom keyboard shortcuts. You can take a look at my .vimrc on GitHub.
Other dotfiles may be useful depending on the programs you use, such as .gitconfig or .tmux.conf. Exploring dotfiles on GitHub is a great way to get a sense of what’s available and useful to you!
Linking dotfiles
We can use a script to create symbolic links, or symlinks for all our dotfiles. This allows us to keep all the files in a central repository, where they can easily be managed, while also providing a sort of placeholder in the spot that our programs expect the configuration file to be found. This is typically, but not always, the user home directory. For example, since I store my dotfiles on GitHub, I keep them in a directory with a path like ~/github/dotfiles/ while the files themselves are symlinked, resulting in a path like ~/.vimrc.
To programmatically check for and handle any existing files and symlinks, then create new ones, we can use this elegant shell script. I compliment it only because I blatantly stole the core of it from Tom’s setup script, so I can’t take the credit for how lovely it is.
The symlink.sh script works by attempting to create symlinks for each dotfile in our $HOME. It first checks to see if a symlink already exists, or if a regular file or directory with the same name exists. In the former case, the symlink is removed and remade; in the latter, the file or directory is renamed, then the symlink is made.
Installing software
One of the beautiful things about exploring shell scripts is discovering how much can be achieved using only the command line. As someone whose first exposure to computers was through a graphical operating system, I find working in the terminal to be refreshingly fast.
With Ubuntu, most programs we likely require are available through the default Ubuntu software repositories. We typically search for these with the command apt search <program> and install them with sudo apt install <program>. Some software we’d like may not be in the default repositories, or may not be offered there in the most current version. In these cases, we can still install these programs in Ubuntu using a PPA, or Personal Package Archive. We’ll just have to be careful that the PPAs we choose are from the official sources.
If a program we’d like doesn’t appear in the default repositories or doesn’t seem to have a PPA, we may still be able to install it via command line. A quick search for “ installation command line” should get some answers.
Since bash scripts are just a collection of commands that we could run individually in the terminal, creating a script to install all our desired programs is as straightforward as putting all the commands into a script file. I chose to organize my installation scripts between the default repositories, which are installed by my aptinstall.sh script, and programs that involve external sources, handled with my programs.sh script.
Setting up the desktop environment
On the recent occasions when I’ve gotten a fresh desktop (intentionally or otherwise) I always seem to forget how long it takes to remember, find, and then change all the desktop environment settings. Keyboard shortcuts, workspaces, sound settings, night mode… it adds up!
Thankfully, all these settings have to be stored somewhere in a non-graphical format, which means that if we can discover how that’s done, we can likely find a way to easily manipulate the settings with a bash script. Lo and behold the terminal command, gsettings list-recursively.
There are a heck of a lot of settings for GNOME desktop environment. We can make the list easier to scroll through (if, like me, you’re sometimes the type of person to say “Just let me look at everything and figure out what I want!”) by piping to less: gsettings list-recursively | less. Alternatively, if we have an inkling as to what we might be looking for, we can use grep: gsettings list-recursively | grep 'keyboard'.
We can manipulate our settings with the gsettings set command. It can sometimes be difficult to find the syntax for the setting we want, so when we’re first building our script, I recommend using the GUI to make the changes, then finding the gsettings line we changed and recording its value.
For some inspiration, you can view my desktop.sh settings script on GitHub.
Putting it all together
Having modular scripts (one for symlinks, two for installing programs, another for desktop settings) is useful for both keeping things organized and for being able to run some but not all of the automated set up. For instance, if I were to set up a VPS in which I only use the command line, I wouldn’t need to bother with installing graphical programs or desktop settings.
In cases where I do want to run all the scripts, however, doing so one-by-one is a little tedious. Thankfully, since bash scripts can themselves be run by terminal commands, we can simply write another master script to run them all!
Here’s my master script to handle the set up of a new Ubuntu desktop machine:
#!/bin/bash
./symlink.sh
./aptinstall.sh
./programs.sh
./desktop.sh
# Get all upgrades
sudo apt upgrade -y
# See our bash changes
source ~/.bashrc
# Fun hello
figlet "... and we're back!" | lolcat
I threw in the upgrade line for good measure. It will make sure that the programs installed on our fresh desktop have the latest updates. Now a simple, single bash command will take care of everything!
You may have noticed that, while our desktop now looks and runs familiarly, these scripts don’t cover one very important area: our files. Hopefully, you have a back up method for those that involves some form of reliable external hardware. If not, and if you tend to put your work in external repository hosts like GitHub or GitLab, I do have a way to automatically clone and back up your GitHub repositories with bash one-liners.
Relying on external repository hosts doesn’t offer 100% coverage, however. Files that you wouldn’t put in an externally hosted repository (private or otherwise) consequently can’t be pulled. Git ignored objects that can’t be generated from included files, like private keys and secrets, will not be recreated. Those files, however, are likely small enough that you could fit a whole bunch on a couple encrypted USB flash drives (and if you don’t have private key backups, maybe you ought to do that first?).
That said, I hope this post has given you at least some inspiration as to how dotfiles and bash scripts can help to automate setting up a fresh desktop. If you come up with some settings you find useful, please help others discover them by sharing your dotfiles, too!
Top comments (16)
Next level would be porting it to an orchestration tool like Ansible, Puppet, Chef or Salt. They include modules to integrate with everything under the sun. It would be a good learning experience and the skills would be transferable to DevOps roles. With these tools, a well designed script could rebuild your system on any distro, regardless of whether it's using apt/dpkg, yum/rpm or something else.
I recommend using
stowfor that kind of installation (symlinking to your dotfiles).I’ve tried out
stowbefore, and found even that to be more overhead than I needed to manage my (relatively small, I’ll concede) dotfiles collection. If I recall correctly,stowlooks for the files in the same folder structure in the source that it then links to the destination. I found that just writing my own symlink and specifying the destination once was easier than remembering where the dotfile is in the file structure. I’m sure it makes sense for larger installations or ones with more programs than mine, though!That is a very interesting post. Thank you for writing it.
I have a note where I gather all the tools and installation when I am setting up a new Linux or Mac machine.
I planned to create some script to automate it but never started it.
I definitely must do it.
Some time ago I realized that I can write notes of this nature as scripts, or at least, close to. So now I try to start there whenever possible. Documentation is automation!
Defintely. :-)
I had to laugh when this came across my feed this morning, as I just figured part of this out on my own and started doing it just last week (symlinking my dotfiles). I never thought about the rest though. Thanks for the tip!
Great timing! :D Try not to lose too many hours to it, like I did at first ^^;
Very interesting! It reminds me a bit of a project of my own: github.com/Sebbestune/DebConf where I try to also have a way to automatically configure my Debian machines with just a script. If anyone wants to take inspiration from a vimrc here is mine too: gist.github.com/Sebbestune/40b7392... (I'm planning to put it in a dedicated git repo soon). :)
Very inspirational since I have three servers I need to get set up in the next few days! This is a great reminder to try automating things!
Also, to anyone who tries something like this, remember that your dotfiles can contain sensitive information! You may need to strip information out of your files and have a private script not hosted publicly that you can use for the sensitive information. If you're feeling bold, you could try to set up a build process that grabs the secrets from another file, environment variables, Hashicorp Vault, probably all sorts of solutions I can't think of and puts them back into the files while orchestrating your scripts!
Symlinks on symlinks 😂 Good luck with the server setups!
Brilliant post! Thanks a lot!
Oh no kidding. Although the encryption feature is neat.
Links to your GitHub scripts throw 404 errors
I wonder if it would be because I forgot to make it public.
(Thanks! Glad someone’s paying attention.)
Nice post! I want to do something similar in Ansible, but I like the bash approach you did.