DEV Community πŸ‘©β€πŸ’»πŸ‘¨β€πŸ’»

IKEUCHI Yasuki
IKEUCHI Yasuki

Posted on • Updated on

How to use Google Colaboratory with Local Runtime

TL;DR

  • I created docker image look-alike Google Colaboratory
    • It is using the same OS version(Ubuntu17.10) as Google colaboratory.
    • It is installed the same python library as Google colaboratory.
  • You can use this docker image with β€œLocal Runtime” feature of Google Colaboratory

Step to connect

  1. Setup nvidia-docker2
  2. $ docker run --runtime=nvidia -it --rm -p 8081:8081 --cap-add SYS_ADMIN --device /dev/fuse --security-opt apparmor=unconfined ikeyasu/colab-local:latest
    • --cap-add SYS_ADMIN, --device /dev/fuse, --security-opt and apparmor=unconfined are necessary for mounting Google drive.
    • You need to change port number, please change -p 8081:8081. For example, -p 8082:8081 if you want to use 8082.
  3. Click β–Ό on top right of Colaboratory(next to "CONENCT" button), and click
  4. Input backend port to your docker instance like 8081. Then, click Connect button.

If a host running jupyter notebook is not localhost

Colaboratory requires localhost to connect local instance. If you would like to run jupyter notebook on another host, you need to use port-forwarding as follows.

$ ssh user@example.com -L 8081:localhost:8081

If you are using windows, you can use PuTTY.

What is benefit?

Google Colaboratory has many limitation, especially it could be ran by 12 hours only. In other hand, Google Colaboratory is good for first step of deep learning.

Thus, you can take a following step to move forward your project.

  1. Use Google Colaboratory for rough investigation and testing. It has limitation of 12 hours.
  2. If your model is looks good, launch Local Runtime. It does not have limitations. You can run a learning process for more 12 hours.

Docker image introduced this paper is looka-like Google Colaboratory. You don't need to concern a difference of environment between host runtime and local runtime.

Top comments (1)

Collapse
adumont profile image
Alexandre Dumont aka Adlx

Hi, great article :).

Do you know if the reverse would also be possible? I mean, connect a locally installed Jupyter Notebook to the runtime on Colab? (even if it has the same restrictions).

🌚 Life is too short to browse without dark mode