TLDR; Use Docker as lite VM to allow for a portable, sharable dev environment
Inspiration
Before I was the product guy at Binaris, I served as lead architect for our scale-out backend compute product. As you might expect, this involved a lot of sshing <insert sunny reference here>, along with remote debugging and development. I quickly grew tired of recreating my environment from scratch on each node.
Why not dotfiles?
Dotfiles solve a part of this problem, but dotfiles do not usually handle package installation and other boilerplate system configuration. Including scripts is nice, but spotty at best. Scripts are also only useful if you can assume the underlying OS is unchanging. For those wondering why I don't simply bake an AMI, each AMI is optimized for the microservice running on it, not my one-off dev needs. My configuration ended up including dotfiles, shell scripts, and Ansible playbooks, just to create a consistent environment.
Inception
A few months ago, I became curious if it's possible/popular to use Docker as lite VM. I saw many people online running a remote server (specifically VSCode) in the docker, but very few actually developing inside the docker container itself. We use Docker a lot at work, so I've become very comfortable with it's tooling and capabilities. Since I'd been wanting to redo my dotfiles+config for some time, I figured why not use Docker.
MyOS
The Dockerfile that resulted from my efforts has now become the basis of MyOS. The Dockerfile provides a semi-opinionated basis for in-container development using vi, zsh and tmux. It handles a lot of boilerplate work like
- Configuring locale and colors
- Creating a non-root user and setting necessary permissions
- Setups OpenSSH for password-less login
- Enabling X11 Display server
Initially, I was using vanilla Ubuntu as a base image, but it required too much hacking just to get the basic functionality I wanted. Eventually, I found out about Phusion, an amazing project that provides a few key features
- Super light, highly optimized base Ubuntu image
- Mechanism to "safely" run multiple processes
- Init for running your user process as PID > 1
- OpenSSH server out of the box
I've also included some opinionated baseline packages. Some are obvious like xauth for native host copy/paste. Others I included, because the getting the setup correct, entails more than running apt-get. Highlights are
- ZSH
- HTop
- Vim8 with clipboard support
- Latest Tmux built from source
- XAuth and XDisplay packages for clipboard support
Using MyOS
MyOS is pretty minimal. Today there are three major parts
Defines the base MyOS image. Handles builtin package installation and system configuration.
Controls which host volumes and ports are mapped into the container. This file is key because it allows us to mount our editor (vi), shell (zsh), and window management (tmux) config files into the container, without actually storing them inside it.
I needed a convenient way of running commands over and over. Initially I used a mixture of sh and Make but always resent maintaining a CLI written with those tools. For now, it's node based, allowing easy installation with npm (I'm definitely willing to change this)
To use MyOS, first install the CLI globally
$ npm install -g myos
Next, you either need to clone an existing MyOS setup, or create a empty template.
$ myos init myos-config
$ cd myos-config
$ ls
docker-compose.yml tmux vim zsh
Once you're in a directory containing a valid MyOS setup run
$ myos create rysenv
The create command essentially runs docker-compose up -d as of today.
Once the container is started, connect using
$ myos connect
And that's it.
Is it worth it?
Disclaimer: obviously this is complete abuse of the container model
There are definitely some limitations of containers, but using one as your development environment has some signficant benefits.
- Incredibly portable - Docker is the JVM of development environments with native OSX, Windows, and Linux support
- Very breakable - Recreating a container has nearly 0 overhead, so feel free to
rm -rf * - Mostly stateless - Aside from the bare essentials, state comes from mounted Docker volumes. This allows you to easily work on your host files in the container environment, without becoming attached to the container.
- Suprisingly performant - On Linux, expect native kernel performance, and on OSX and Windows VM level performance (Docker on OSX runs in a VM).
- Accelerates env iteration - due to its layered file system, you can make massive adjustments to the runtime environment, without having to manually remove and re-install dependencies.
- Transferrable - Utilizing
docker commitalong withdocker pushand AWS ECR, move from host to host while maintaining file system state (but please just stay stateless)
Sharing
Although I primarily value MyOS for my personal development, I browse /r/vim daily, and interactions on the subreddit made me wonder if MyOS could be used for more. MyOS can facilitate the sharing of ENTIRE terminal environments, not just dot files. Wouldn't it be cool to share your setup and know that it's guaranteed to work for anyone on any system? Assuming everyone relies on a semi-stable base image, sharing requires you to only send the layers that you've added on top of the base image along with your dotfiles and other configuration.
Final remarks
I've made all my work open source, and would love if others use what I've made. I'm open to discussing features, or even changes based on what the community needs, and pull requests are definitely welcome. There are a few items I already know are problematic
- Very vim focused, would like to see if its possible to support other editors (definitely Emacs)
- Needs CLI command that allows you to copy another users MyOS environment
- Multiple MyOS environments should be able to run simelataneously
- SSH key for authorized_keys could be autogenerated
- Potentially move away from NodeJS for CLI
Top comments (10)
Nice post and I share the idea of using containers for dev envs. The only big issue is performance on a non Linux machine. On a mac bind mounting is kinda of a big problem performance wise, which makes very hard to share data with host system. Volumes performance is better, but not nearly as performant as Linux.
Vs code support for containers just makes the dev experience better. No need to share file with the host, just use a volume. My guess is that IDE support for devolpment in a container will increase this practice.
I use OSX as my primary development machine. Volume performance is absolutely amazing if you use the “delegated” mode when mounting the volume. This is the default used by MyOS and therefore shouldn’t be a factor for most users.
Were you previously aware of delegated mode? Also a volume is sharing a file with the host...
Hi! Yes, but when compared to an Ubuntu machine for example, there is quite a bit of a difference. Anyway, I agree that volumes are okish, bind mounts are an issue. So much of an issue that you have to resort to unison or other file syncing to get a biderectional sync between container and host.
This is (also) why I still use Linux for dev. Containers and Linux is a sweat match: no vm thus native performance, reduced mental overhead required to switch from host to container, easy to mount host credentials (mounting kerberos credentials from mac to container is very hackish) and you do not have to deal with the docker flakiness on mac.
Anyway I guess, if you want to try out the latest language/framework/tool version without messing up your machine, then containers for dev are a great solution. On the other hand, devoloping within a container all the time, is a stretch of the docker model as you mentioned. Its effects are obviously more evident when you add a vm in the middle
Had absolutely the same idea. I was doing it before Docker with Vagrant and VirtualBox, but Docker makes it even better. Another idea is to use Cloud IDE (in browser IDE), see my post dev.to/stereobooster/truly-integra.... But I'm pretty sure that vim would be faster than any in-browser IDE.
Potentially interesting candidates for the base image: Neovim, zsh + oh-my-zsh, mosh
I definitely think Cloud IDE's solve some of the problems that MyOS does. The key differentiator is that MyOS uses Docker as the display server itself, instead of the usual rsync setup. Your article is nice.
I'm glad you like the idea, if you have any feedback/issues please feel free to reach out.
Have you looked into vscode remote development. It still in preview but I think you would find useful quiet useful
I actually played around with remote vscode a year or two ago (when it was a 3rd party plugin). I use vi as a primary editor so switching to vscode isn't an option for me.
Also, MyOS can support vscode, and also provides you a full remote runtime.
Thanks for reading the article, and your sincere response.
Brilliant idea, I'm excited to give it a try. Thanks for sharing!
Glad to hear you like it, let me know if you run into any issues. Feel free to make any contributions, just submit a PR.
It’s asking for a password so I can’t open it.