How to Run Jupyter In Any Cloud With One Command

Casper da Costa-Luis — Thu, 28 Apr 2022 16:31:02 +0000

Two pain points often faced by scientists and engineers are:

Installing and maintaining your entire software stack (including GPU drivers and dependencies) is difficult and time-consuming, and
Being limited by the hardware you own (laptop or desktop specs).

Here's a solution!

Requirements

Download the terraform CLI tool and say goodbye to ClickOps (free)
Get an ngrok account for port forwarding convenience (free)
Have cloud credentials, of course! (AWS, Azure, GCP, or K8s.)

Lift off! 🚀

Using this GitHub repository and TPI to do all the heavy lifting:

git clone https://github.com/iterative/blog-tpi-jupyter
cd blog-tpi-jupyter
export NGROK_TOKEN="..."  # Signup for free at https://ngrok.com
terraform init    # Setup local dependencies
terraform apply   # Create cloud resources & upload workdir
terraform refresh # Get URLs (rerun if blank)

It takes a few minutes to launch cloud resources and upload your working directory, so you might have to wait a bit before running terraform refresh again. Soon, you'll see:

Outputs:
urls = [
  "Jupyter Lab: https://{id}.ngrok.io/lab?token=...",
  "Jupyter Notebook: https://{id}.ngrok.io/tree?token=...",
  "TensorBoard: https://{id}.ngrok.io",
]

Click on those URLs to get to your dev environment! ✨

Download & Cleanup 📥

When done experimenting, download the shared workdir, delete the cloud storage, and terminate the cloud instance with one simple command:

terraform destroy

Benefits 🎩

It uses Terraform Provider Iterative (TPI) under-the-hood. There are a few distinct advantages to TPI:

💰 Lower cost: use your preferred cloud provider's existing pricing, including on-demand per-second billing and bulk discounts.

🔄 Auto-recovery: spot/preemptible instances are cheap but unreliable. TPI reliably and automatically respawns such interrupted instances, caching & restoring the working directory in the cloud even when you are offline.

👓 Custom spec: full control over hardware & software requirements via a single main.tf config file - including machine types (CPU, GPU, RAM, storage) & images.

Happy coding!

DEV Community: Casper da Costa-Luis

How to Run Jupyter In Any Cloud With One Command

Requirements

Lift off! 🚀

Download & Cleanup 📥

Benefits 🎩