Isolated javascript development environment / 2021

TLDR: If you want to separate your development environments there are few ways to go.

✅ Any virtualizations or containerization solution + VSCode + Remote extension works really nice.
❌ Any virtualizations or containerization solution + shared folders + local VSCode/Webstorm works really badly due to various reasons.
✅ VirtualBox/Paralles (ssh + headless) + WebStorm + Projector works very well.
✅ Parallels (GUI) + WebStorm/VSCode works also very well.

Intro

So, in simple words, I'm working at an agency and I'm using a single personal computer for work, personal stuff, and for learning. Because of this, I have some security, performance, and organization concerns regarding the software I need to install daily and to work with.

🥷 As for security, I don't want to install every package which some developer decided to add the project long, long, long time ago on my local machine where I have all banks-related info.

⚡️ As for performance, if it's possible to organize some isolated environment, then it should be possible to do the same for the remote environment. This means I may try to set up some easily upgradeable server as a workstation and only use my Mac as a thin client, so I don't need to upgrade it often.

📦 As for organization, as a developer, I play with different technologies a lot, the last time I've cleaned my mac I needed to remove some unneeded Haskell files, Go files, Clojure files,... Not mentioning brew packages which I needed only once. So I would like to have a playground to play with technologies.

So the possible criteria to compare different setups:

1. Is it easy to install? How long does it take to bootstrap the setup?
2. Does it support GUI apps or headless mode only?
3. Does it work well? It's probably not worth it to sacrifice development experience.

As an example, let's try to run this basic React-based app inside every environment and see how it goes. I have picked this one because it doesn't require a specific CLI to run it.

Workstation

I have some not-so-old-not-so-new mac pro, nothing special here.

The only upgrade I have is an external Samsung 1Tb SSD, it was a pain to find free space on my internal 256GB SSD. But now I can move Docker, VirtualBox, and Parallels images to external drive 👌 Free space, finally.

Docker for Mac

Looks like react-boilerplate doesn't have Docker and docker-compose files. So we need to add them, I guess, it won't be hard.

Docker

FROM ubuntu:latest

ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update && apt-get upgrade -y
RUN apt-get install -y nodejs npm

RUN mkdir /project
WORKDIR /project

COPY package.json package-lock.json ./
RUN npm install

COPY . ./

EXPOSE 3000

CMD [ "npm", "start" ]

// I know that I don't need docker-compose for one Dockerfile, but it's simpler for me to run docker-compose up --build than remember all docker cli commands for ports and volumes 🤷‍♂️

docker-compose.yml

services:
  web:
    build: .
    ports:
      - "127.0.0.1:3000:3000"
    volumes:
      - ./app:/project/app
      - ./server:/project/server

After we add and run these docker-related files the app runs smoothly 👌

So, we have built an app in an isolated environment, we've done half of the way to the goal. Now we need to figure out the way to edit the code.

The only issue with Docker is that we won't be easily able to run GUI apps using it, e.g. if we decide to write e2e tests using cypress, we will need to figure out how do we do that. But anyway, let's see how it goes.

UPD: Looks like you still can run cypress in regular mode inside the docker and just do X11 forwarding. Link

Docker + VSCode + Remote extension

I would assume most of you know what VSCode is, but I'm going to introduce you to the official Remote Development extension. This extension works with SSH, Docker, and even WSL. It can automatically find configurations, connect and install a temporary VSCode server on the target machine, so you only need to pick the target connection and after that magic happens ✨

This is a connected VSCode window (it's almost exactly the same as the local run).

I've changed App.js and here it is! It works!

📝 So overall development experience is almost exactly the same to the local one, 5 out of 5 ✨

Docker + WebStorm

But what do I do if I got used to WebStorm and would like to continue developing using it?

If we just open source files, WebStorm won't be able to load all libraries...Hmmm...

I've searched on the internet but it looks like it's possible only to configure Node.js remote interpreter. It will only be able to run a given application inside the docker and attach a debugger to it, but not to resolve node_moduels inside the container (at least I haven't managed to set it up properly).

There is a question on StackOverflow about it, but the solution didn't work for me, not sure if I did something wrong, but it just didn't work out 🤷‍♂️

There is one more trick left to make it run, we can install node_module inside the container and somehow map it back to the host machine. We can try this:

...
    volumes:
        - ./node_modules:/project/node_modules
... 

But if we just map a volume via docker-compose, then local node_modules will be empty, because of the Docker build & run steps order:

1. We install node_modules during the Docker build.
2. We map volume over the existing folder during the Docker run.

This means the local node_modules folder will be empty 😔

There is a solution with some double volume mapping. Again, it didn't work for me.

We can try to update Docker file to copy node_modules.

Docker

FROM ubuntu:latest

ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update && apt-get upgrade -y
RUN apt-get install -y nodejs npm

# NEW: building app in some different folder
RUN mkdir /project-build
WORKDIR /project-build

COPY package.json package-lock.json ./
RUN npm install

# NEW: copying actual source files to the initial folder
RUN mkdir /project
WORKDIR /project

COPY . ./

EXPOSE 3000

# #!/bin/bash
# # This will copy node_modules to the shared volume
# cp -r /project-build/node_modules /project
# npm run start
CMD [ "./start.sh" ]

If we just run it, it works fine 👌 Let's try to add volume to docker-compose.

docker-compose.yaml

services:
  web:
    build: .
    ports:
      - "127.0.0.1:3000:3000"
    volumes:
      - ./app:/project/app
      - ./server:/project/server
            # NEW: node_modules mapping
      - ./node_modules:/project/node_modules

Aaand... Running it... Running it... Running it...It...Is...Slooooooooooooow...

It took 5 minutes instead of a few seconds to copy node_modules on volume. But why? ... After a quick research, looks like it is some known Docker for Mac issue, which is 5 years old.

📝 I would say it almost doesn't work, development experience is much worse than on local 2 out of 5.

Docker + WebStorm + Projector

Actually, there is still a way to run WebStorm on the remote machine, which is similar to VSCode Remote Extension but requires few more manual steps. The solution is called Projector, here is an initial Jetbrains post about it.

First, we will need to update our Docker file to install projector.

Docker

FROM ubuntu:latest

ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update && apt-get upgrade -y
RUN apt-get install -y nodejs npm

RUN mkdir /project-build
WORKDIR /project-build

COPY package.json package-lock.json ./
RUN npm install

RUN mkdir /project
WORKDIR /project

COPY . ./

EXPOSE 3000

### Install projector deps & projector itself
RUN apt-get install python3 python3-pip -y less
RUN python3 -m pip install -U pip 
RUN pip3 install projector-installer
### TODO: automate projector installation
EXPOSE 9999

CMD [ "./start.sh" ]

^^^ Notice that you will still need to connect to docker manually and run ~/.local/bin/projector/install. It obviously can be automated, but I'm too lazy to do so right now.

Also, we need to update the Docker file to run Projector on the start.

start.sh

#!/bin/bash
projector run &
npm run start

After building docker docker-compose up --build, connecting to it remotely docker exec -it $ID bash, installing projector projector install (yeah, yeah, we can automate it later), it works!

📝 I would say it works, but it's a bit awkward to reinstall the whole dev environment every time you update docker, 3 out of 5.

VirtualBox (ubuntu)

I would assume this tool is widely known, but in simple words it allows you to run the guest OS on top of the host OS. I won't waste your time guiding you through the setup, so let's just jump right to results.

VirtualBox + VSCode/WebStrom

I've tried to install WebStorm on the virtual machine itself, but. It. Is. Just. Slow.

I've installed all VirtualBox guest tools, gave it full resources, but still. It. Is. Sloooooow.

📝 Nah, I can't work like this, 1 out of 5.

VirtualBox (headless) + Vagrant (ssh)

In simple words Vagrant helps you automating bootstrapping virtualization via config files, you can do everything on your own via the VirtualBox interface itself.

After the setup, you can actually set port forwarding via VirtualBox interface, or you can just port forward it via ssh cli 🤷‍♂️

vagrant ssh -- -L 3000:127.0.0.1:3000

VirtualBox (headless) + Vagrant (ssh) + VSCode + Remote extension

Headless VirtualBox actually works fine, so we can just do exactly the same as we did for Docker with VSCode. The only issue is that it won't be possible to run any GUI like this.

📝 Works fine if we just use ssh, 5 out of 5 ✨

VirtualBox (headless) + Vagrant (ssh) + WebStorm

This setup is the most straightforward of all. Once, you need to add the folder with your project as a shared folder via vagrant config or via VirtualBox interface. Every time later you just need to connect to the virtual machine via ssh, go to the target folder, install dependencies, run the project.

Now you can open the project via locally installed WebStorm, change the title and see how it changes.

...Wait, the title is the same!

The issue is that when you change something on host OS, guest OS doesn't get properly notified about changes, so if you are using some build watcher like webpack, it won't work. There are some ways to fix it, but they look like hacks to me.

📝 Change detection doesn't work, so 2 out of 5.

VirtualBox (headless) + Vagrant (ssh) + WebStorm + Projector

📝 Basically the same setup as with Docker, but you don't need to reinstall Projector every time, so 5 out of 5.

Parallels (ubuntu)

Basically, Parallels is VirtualBox specializing in running on Mac. I've asked few friends and looks like Parallels is kind of popular, so I've bought it. I will try to keep every paragraph short and simple since everything is super similar to VirtualBox.

Parallels + VSCode/WebStrom

It works muuuuuuuuch faster than VirtualBox, so it is possible to run GUI apps directly. There are a few minor issues with keybindings and hotkeys, but they are fixable at least for WebStorm, you can just install Mac keybindings.

📝 Works well, from time to time it is a bit laggy, but overall it is solid 4 out of 5.

Parallels (headless) + Vagrant (ssh) + VSCode + Remote extension

📝 As usual, works like a charm, 5 out of 5.

Parallels (headless) + Vagrant (ssh) + WebStorm

📝 Same issues with change detection, the same grade of 2 out of 5.

Parallels (headless) + Vagrant (ssh) + WebStorm + Projector

📝 This was my final choice, works really solid, 5 out of 5.

Other notes

Recently I've noticed few new issues with a setup like this.

Git & githooks

If you git on your host OS, then you will have issues with git hooks, because hooks like pre-commit or pre-push will be run on your host OS instead of guest OS, which is obviously bad from a security standpoint.

If you use git on your guest OS you need to deliver your ssh key to a guest system or use your login/password pair which is also bad from a security perspective. But there few ways to mitigate these issues.

1. You can share your local ssh key with guest os without exposing it via ssh -A host. Git will use your ssh key without actually having direct access to it. The issue is that it is still insecure.
2. You can create a Github account per every environment and have different ssh keys. This approach is the most troublesome, but the most secure, I will try to go for it at some point.
3. You can still use git on your host os, but disable git hooks 🤷‍♂️

p.s.: Will post more fun stuff here and on Twitter👋
p.s.1: If you tried to do the same, please share your results!