Two pain points often faced by scientists and engineers are:
- Installing and maintaining your entire software stack (including GPU drivers and dependencies) is difficult and time-consuming, and
- Being limited by the hardware you own (laptop or desktop specs).
Here's a solution!
Requirements
- Download the terraform CLI tool and say goodbye to ClickOps (free)
- Get an ngrok account for port forwarding convenience (free)
- Have cloud credentials, of course! (AWS, Azure, GCP, or K8s.)
Lift off! π
Using this GitHub repository and TPI to do all the heavy lifting:
git clone https://github.com/iterative/blog-tpi-jupyter
cd blog-tpi-jupyter
export NGROK_TOKEN="..." # Signup for free at https://ngrok.com
terraform init # Setup local dependencies
terraform apply # Create cloud resources & upload workdir
terraform refresh # Get URLs (rerun if blank)
It takes a few minutes to launch cloud resources and upload your working directory, so you might have to wait a bit before running terraform refresh
again. Soon, you'll see:
Outputs:
urls = [
"Jupyter Lab: https://{id}.ngrok.io/lab?token=...",
"Jupyter Notebook: https://{id}.ngrok.io/tree?token=...",
"TensorBoard: https://{id}.ngrok.io",
]
Click on those URLs to get to your dev environment! β¨
Download & Cleanup π₯
When done experimenting, download the shared workdir
, delete the cloud storage, and terminate the cloud instance with one simple command:
terraform destroy
Benefits π©
It uses Terraform Provider Iterative (TPI) under-the-hood. There are a few distinct advantages to TPI:
π° Lower cost: use your preferred cloud provider's existing pricing, including on-demand per-second billing and bulk discounts.
π Auto-recovery: spot/preemptible instances are cheap but unreliable. TPI reliably and automatically respawns such interrupted instances, caching & restoring the working directory in the cloud even when you are offline.
π Custom spec: full control over hardware & software requirements via a single main.tf
config fileβ-βincluding machine types (CPU, GPU, RAM, storage) & images.
Happy coding!
Top comments (0)