DEV Community: kougou

How to Connect VSCode to GitHub

kougou — Tue, 08 Apr 2025 14:50:51 +0000

Let’s assume you already have:

Visual Studio Code installed
A GitHub account set up
Git installed on your system I'm working on Ubuntu, but the steps should be similar across other operating systems.

Configure Git (First Time Only)
If you haven’t configured Git with your name and email yet, do it now using the following commands:

git config --global user.name "YOUR NAME"
git config --global user.email "EMAIL ADDRESS"

💡 Tip: If your name or email doesn't contain spaces, the quotes are optional.

Sign in to GitHub from VSCode

Open Visual Studio Code.
Click the account icon in the bottom-left corner.
Select “Sign in to GitHub” from the menu.
A browser window will open with the GitHub authentication page.
Log in and authorize access.
Once authentication is successful, VSCode will be connected to your GitHub account!

Clone an Existing GitHub Repository
If you want to clone a repository that already exists on GitHub:

Go to the repository page on GitHub and copy the URL.
In VSCode, press Ctrl + Shift + P to open the Command Palette.
Select “Git: Clone”.
Paste the copied URL when prompted.
Choose a local directory to save the repository.

Download MODIS files Using PyModis Part2

kougou — Tue, 19 Nov 2024 05:11:04 +0000

Using JSON to Simplify MODIS Product Path Management

In a previous article, I explained how to download MODIS files using PyModis.

One of the challenges with PyModis is determining the server path for a given product. While PyModis includes a script called productmodis.py to address this, the script was designed for Python 2 and is incompatible with Python 3. Additionally, it does not cover all MODIS products, which may limit its usefulness for some users.

To resolve this issue, I created a JSON file that maps MODIS product names to their corresponding server directories. You can download the JSON file here.

JSON File Structure

Here is a sample of the JSON file:

{
    "AG100.003": "ASTT",
    "AG1km.003": "ASTT",
    "AG5KMMOH.041": "ASTT",
    "...": "...",
    "MOD09A1.061": "MOLT",
    "MOD09CMG.061": "MOLT",
    "MOD09GA.061": "MOLT",
    "...": "...",
    "MCD12C1.061": "MOTA",
    "MCD12Q1.061": "MOTA",
    "MCD12Q2.061": "MOTA"
}

Each entry consists of:

Key: The MODIS product name.
Value: The corresponding server directory.
For example:
"AG100.003": "ASTT" indicates that the product AG100.003 is in the ASTT directory.
"MOD09A1.061": "MOLT" means that MOD09A1.061 is located in the MOLT directory.
This JSON structure allows for easy and efficient searching of server paths for MODIS products.

This JSON structure allows for easy and efficient searching of server paths for MODIS products.

Python Script for Searching the JSON File

The following Python script demonstrates how to use the JSON file to search for MODIS product paths, both for exact and partial matches.

Loading the JSON File

import json
import os
from typing import Dict

def load_json_data(file_path: str) -> Dict[str, str]:
    """
    Load and return JSON data from a file.

    Parameters:
    file_path (str): Path to the JSON file.

    Returns:
    dict: A dictionary containing the JSON data.

    Raises:
    FileNotFoundError: If the file does not exist.
    ValueError: If the file contains invalid JSON.
    """
    if not os.path.exists(file_path):
        raise FileNotFoundError(f"JSON file '{file_path}' not found.")
    try:
        with open(file_path, "r") as file:
            return json.load(file)
    except json.JSONDecodeError as e:
        raise ValueError(f"Invalid JSON format in file '{file_path}': {e}")

Searching for Products

from typing import List, Tuple

def search_product_directory(
    product: str, data: Dict[str, str], case_insensitive: bool = True
) -> List[Tuple[str, str]]:
    """
    Search for exact or partial matches of a product in the provided data dictionary.

    Parameters:
    product (str): The product name or keyword to search for.
    data (dict): A dictionary where keys are product names and values are their corresponding directories.
    case_insensitive (bool): If True, perform a case-insensitive search.

    Returns:
    list: A list of tuples containing matching product names and their corresponding directories.
          Returns an empty list if no matches are found.
    """
    if case_insensitive:
        product = product.lower()
        matches = [(key, value) for key, value in data.items() if product in key.lower()]
    else:
        matches = [(key, value) for key, value in data.items() if product in key]

    return matches

Example usage

if __name__ == "__main__":
    json_path = "modis_dir.json"
    mapping = load_json_data(json_path)

    # Case of exact match
    product = "MOD11A1.061"
    result = search_product_directory(product, mapping)
    print(result)
    # Output: [('MOD11A1.061', 'MOLT')]

    # Case of partial match
    product = "MOD11A"
    result = search_product_directory(product, mapping)
    print(result)
    # Output: [('MOD11A1.061', 'MOLT'), ('MOD11A2.061', 'MOLT')]

    # Case of no match
    product = "NO_MATCH"
    result = search_product_directory(product, mapping)
    print(result)
    # Output: []

Using this JSON file and script, you can easily integrate the product's server path into your download code. This simplifies the process and reduces manual effort when managing MODIS data.

Download MODIS files Using PyModis Part1

kougou — Mon, 04 Nov 2024 14:39:15 +0000

To download MODIS files directly from NASA Earthdata, we’ll use the PyModis library, which allows us to handle the entire process in Python, from downloading to analyzing data. You can find PyModis here:
https://github.com/lucadelu/pyModis/

If you don’t already have a NASA Earthdata account, please register first at:
https://urs.earthdata.nasa.gov/

Installing Required Libraries

We’ll start by setting up an Anaconda virtual environment and installing the necessary libraries.

conda install -c conda-forge pymodis
conda install -c conda-forge libgdal

When we install PyModis with conda, GDAL should be included automatically as a dependency. However, if GDAL is missing, we can install it manually. If HDF4 support is also required, installing libgdal will resolve any related errors, such as:

warnings.warn("GDAL installation has no support for HDF4, please update GDAL", ImportError)

Using the downmodis Class in PyModis

The downmodis class from PyModis enables us to download MODIS files. Here’s a brief overview of its arguments, with a partial excerpt from the source code:

# This code is part of the PyModis library by Luca Delucchi and Logan C Byers,
# licensed under the GNU General Public License v2.0.

class downModis:
    """A class to download MODIS data from NASA FTP or HTTP repositories

       :param str destinationFolder: where the files will be stored
       :param str password: the password required by NASA authentication system
       :param str user: the user namerequired by NASA authentication system
       :param str url: the base url from where to download the MODIS data,
                       it can be FTP or HTTP but it has to start with
                       'ftp://' or 'http://' or 'https://'
       :param str path: the directory where the data that you want to
                        download are stored on the FTP server. For HTTP
                        requests, this is the part of the url between the 'url'
                        parameter and the 'product' parameter.
       :param str product: the code of the product to download, the code
                           should be idential to the one of the url
       :param str tiles: a set of tiles to be downloaded, None == all tiles.
                         This can be passed as a string of tileIDs separated
                         by commas, or as a list of individual tileIDs
       :param str today: the day to start downloading; in order to pass a
                         date different from today use the format YYYY-MM-DD
       :param str enddate: the day to end downloading; in order to pass a
                           date use the format YYYY-MM-DD. This day must be
                           before the 'today' parameter. Downloading happens
                           in reverse order (currently)

       :param int delta: timelag i.e. the number of days starting from
                         today backwards. Will be overwritten if
                         'enddate' is specifed during instantiation
       :param bool jpeg: set to True if you want to download the JPG overview
                         file in addition to the HDF
       :param bool debug: set to True if you want to obtain debug information
       :param int timeout: Timeout value for HTTP server (seconds)
       :param bool checkgdal: variable to set the GDAL check
    """

    def __init__(self, destinationFolder, password=None, user=None, token=None,
                 url="https://e4ftl01.cr.usgs.gov", tiles=None, path="MOLT",
                 product="MOD11A1.006", today=None, enddate=None, delta=10,
                 jpg=False, debug=False, timeout=30, checkgdal=True):

We can authenticate using either a username and password or a token. To use a token, generate one.
For details, see:
https://urs.earthdata.nasa.gov/documentation/for_users/user_token

Locating the Correct Directory

Accessing the default url in the arguments, https://e4ftl01.cr.usgs.gov, will show a directory structure like this:

ASTT/
COMMUNITY/
ECOSTRESS/
MEASURES/
MOLA/
MOLT/
MOTA/
VIIRS/

To download specific files, we need to locate them within this structure. For example, MYD11A2 files are stored in the MOLA directory. We can access them at:
https://e4ftl01.cr.usgs.gov/MOLA/MYD11A2.061

MOLA/
├── MYD09A1.061/
│   ├── 2002.07.04/
│   ├── 2002.07.12/
│   └── ...
├── ...
├── MYD11A2.061/
│   ├── 2002.07.04/
│   ├── 2002.07.12/
│   └── ...
└── ...

To specify the location in PyModis, we set the path and product arguments.
For MYD11A2.061, we set:
path: "MOLA"
product: "MYD11A2.061" (where .061 is the version)

Example Code

The following example downloads MYD11A2.061 files for the h29v06 tile between 2024-01-01 and 2024-10-23.

from pymodis import downmodis

# Define parameters
destination_folder = "data"
tiles = "h29v06"
path = "MOLA"
product = "MYD11A2.061"
today = "2024-01-01"
enddate = "2024-10-23"

# Initialize downloader
modis_downloader = downmodis.downModis(
    destinationFolder=destination_folder,
    password="your password",
    user="your username",
    tiles=tiles,
    path=path,
    product=product,
    today=today,
    enddate=enddate,
)

# Connect and download files
modis_downloader.connect()
modis_downloader.downloadsAllDay()

This script authenticates and downloads the desired files directly to the specified folder. Adjust parameters as needed for other MODIS products or time frames.