DEV Community: Ahmad Abdullah Tariq

Configure Jupyter Global Imports

Ahmad Abdullah Tariq — Mon, 18 Jan 2021 08:40:17 +0000

Open a terminal and go to the following directory
```
    cd ~/.ipython/profile_default/startup
```
Now create a python file. You can name it anything you want:
```
    touch start.py
```
Now open the file you can do inside the terminal or open up your favorite IDE.
```
    nano start.py
```
Place all the libraries that you want to get imported by default.
No press CMD+X and press Y to save.
If you have jupyter instance running you need to restart them.
To check if you successfully included those libraries you can check them in globals() in a notebook:

Ref: How to Automatically Import Your Favorite Libraries into IPython or a Jupyter Notebook

Dataframe to Google Sheet

Ahmad Abdullah Tariq — Fri, 15 Jan 2021 13:43:03 +0000

Setup in Google Console

Create a new project in Google Console.
You have to enable google services in order to get access to google sheet. Under API & Services Click on Enable API and Services
- Enable Google Drive API
- Enable Google Sheets API
Now go to Credentials menu and click on Manage service accounts.
- Click on Create Service Account
- Optionally you can also add roles but we are going to skip that since we need to access files for different users.
- Inside Service Account Details click on Add key. Download it and store it we will need it later while setting up functionality in jupyter.

Setup in Jupyter

Place the service key in the root of your jupyter folder. (You can change the path too, but you have to modify it inside the function.)
To get access of google drive and google sheet we need to install some libraries. Install the following libraries:

    pip install pandas gspread google-api-python-client

The following function takes dataframe and two optional arguments sheet_name and email. sheet_name will rename the sheet to a given name and email will add permission to add the created file into the given account's google drvive. Copy the following function:

    import pandas as pd
    from numpy.random import randint
    import gspread
    from google.oauth2 import service_account
    from googleapiclient.discovery import build

    def df_to_sheet(df, sheet_name, email_address):
        """
        Connects with google drive. Creates a sheet and copy data from dataframe, stores and  create a link.

            :param df: dataframe you want to store.
            :param sheet_name: Name of the sheet you want to save in google drive.
            :param email_address: of the user of google drive.
            :return: returns URL of the file saved 
        """

        scopes = [ "https://www.googleapis.com/auth/spreadsheets", "https://www.googleapis.com/auth/drive"]
        credentials = service_account.Credentials.from_service_account_file("./credentials.json",scopes = scopes)
        permission = { "role": 'writer', "type": 'user', "emailAddress": email_address}

        try:
            # connecting to sheets and google drive
            sheet_service = gspread.authorize(credentials) 
            drive_service = build("drive", "v3", credentials=credentials)

            # creating sheet and copying data in worksheet
            sh = sheet_service.create(sheet_name)
            worksheet = sh.get_worksheet(0)
            worksheet.update([df.columns.values.tolist()]+df.values.tolist())
            drive_service.permissions().create(fileId=sh.id, body=permission).execute()

        except Exception as e: 
            print('Error creating google sheet: ',e)
            return None
        return "https://docs.google.com/spreadsheets/d/%s" % sh.id

Now you can create and store any dataframe you want. Note: make sure there are no null values inside dataframe df.fillna('',inplace=True).

    ## creating a dataframe
    df = pd.DataFrame({'x':randint(5, size=10),'y':randint(5, size=10)})

    ## storing dataframe to google sheet 
    df_to_sheet(df, sheet_name='New Sheet', email_address='ahmadabdullah247@gmail.com')

Repurpose Old Laptop to Host Jupyter Lab

Ahmad Abdullah Tariq — Sun, 10 Jan 2021 11:03:47 +0000

If you have an old laptop sitting around in your home, instead of giving it away it is a great idea to make it a remote server to practice your development skills. You can use it to deploy the host website (PS: it is advised to use cloud service to productionize. Since it provides 100% uptime and will save you the hassle of securing the server). Neither the less it provides a really great opportunity to learn setting up a remote server.

Theoretically, you can use any operating system you prefer. In this tutorial, we will be setting up remote access on an Ubuntu OS (version 20.04 precisely). Hosting a Jupyter Lab instance on a remote laptop and accessing it via SSH.

Setup on Remote Laptop

Open a terminal and make sure everything is up to date on your remote laptop:

    sudo apt-get install update
    sudo apt-get install upgrade

Now to enable your laptop as a remote server you need to install ssh server tools via following command.

    sudo apt-get install OpenSSL-server

Now you just need to copy the IP address from this machine which we will need to access the remote laptop. You can find that by running the following command.

    ip addr

(Optional) you can install htop which gives you information about your server resources. free command gives you the stats about memory.

    sudo apt-get install htop
    free

Connecting from Laptop

Fire up the terminal and connect via following command. On entering the following command you'll be prompted to enter the password of your remote laptop. Type your password and hit enter. Wola you are now connected to your remote desktop. Note: change remote to your remote laptop name and 127.0.0.1 to IP that you copied from the other system.

    ssh remote@127.0.0.1

Once you are on your remote desktop you can use it for any application you like. We are going to start a Jupyter lab host. The following command will start jupyter lab on your remote laptop on the default port which is 8888. Copy the URL.

    jupyter lab -no-browser

Open another terminal and write following command. This will again prompt you to enter the password and if there is no error this means that you have successfully connected to your remote machine.

    ssh -N -f -L localhost:8888:localhost:8888 remote@127.0.0.1

Now paste the copied URL from the other terminal in your browser and you have jupyter lab connected from your remote laptop. To understand what all the flags in the above command mean you can visit this website.
Bonus: If you want to operate your remote laptop with the lid down you can take following steps

     sudo nano /etc/systemd/logind.conf

Uncomment HandleLidSwitch and change it to 'ignore' if it isn't already. (ctrl+x to save and exit)

    HandleLidSwitch=ignore

Now you need to restart the system daemon and you can use your remote laptop with the lid down.

     sudo systemctl restart systemd-logind