DEV Community: Gareth M.

Commosta: share computing resources

Gareth M. — Thu, 15 Jan 2026 02:18:38 +0000

Learn more here

Displaying air quality data in the terminal

Gareth M. — Sat, 21 Oct 2023 22:46:37 +0000

Intro

My local county reports data on the level of several pollutants throughout the area. However, the only way to see this data is via an online dashboard. My project air-monitor is a Python script that uses the backend APIs of the dashboard to display the same data in the terminal.

Obtaining the data

The dashboard makes separate requests for each pollutant type, which are specified in the request parameter where, along with the desired time span.
The service measures the following pollutant types:
- sulfur dioxide
- carbon monoxide
- Particulate Matter (10 and 2.5)
- ozone
- nitrogen oxide

The data is returned in an array of JSON objects, each of which contains a value as well as the date and time the data was collected

The are also several sensors placed throughout the county's area, which each generate their own sets of data points. In the web dashboard, these are shown in different colors/lines. In the future, ANSI codes could be used to show multiple data sources. For now, I've opted to use an averaging function that combines pollutants with multiple data sources, which is handled by the chart library.

Graphing the Data

To graph the data, chart.py uses a numpy matrix that is allocated based on the possible values ranges of a pollutant, since each pollutant is measured with different unit types (ppb, ppm, ug/m^3). Each pollutant type has it's own array size and conversion factors to display. For example, here are the values for CO.

  "CO":(40, 2, -0.01, "ppm", 100, 100, "{:.2f}", 100), # ppm

The first value is used as the size for the outer array (the columns), and each of the sub-arrays is allocated according to the number of hours the data covers. The rest of the values are used to round and display the numbers. Then, the values are printed for each interval where a data point exists.
In order to average data from several locations, the data is first arranged into a dictionary with each time slot as a key. Then, any duplicates are averaged together and re-packaged with the same format as the original JSON data.

Challenges

The main challenge with this project is getting the data to fit inside the terminal, while also making it legible enough. My goal was for the user to visually be able to match the shape of the graph to the one in the web dashboard, but also fit on smaller displays/terminals with fewer columns. I wanted to make an initial version of this project that just used regular terminal output, without depending on any special ANSI codes. Future updates would include adding colors to the output and adding connecting lines/dots to better show the curve of the graph.

Project repo: https://github.com/gbafana25/air-monitor
VODs of coding streams: https://youtube.com/@gbafana25

Building a version control system

Gareth M. — Sat, 20 May 2023 05:12:26 +0000

Intro

As developers, we use version control systems everyday. Most are probably with Git or Subversion (SVN). Since it's easy to use these tools without thinking about how they work, I decided to try and make my own version control system.

How version control works

A version control system keeps track of the changes (versions) of your code. Whenever you make a commit, the current version of the file(s) are compared to the previous version, and a new "commit" is made. Each control system has various ways of doing this. For example, Git uses a snapshot system, where a completely new copy of every modified file is stored in a commit object [1]. Other systems like SVN (and my project) record the differences between files over time.

About zren

When a zren repository is created, a base directory is made with a copy of every file at that moment. When the first commit is made, it compares the current version of each file with the base version. For every commit after that, the "image" of the current file is first built off of applying each commit to the base file chronologically. Then, the current file is compared to the reconstruction of the previous version.

Changefiles

To generate the changefiles, the most recent version of the file is compared with the current version being committed. Every character that is different is stored in a linked list. Places that are the same are given a placeholder, but will not be included in the file itself. When it comes time to write the actual changefile, the linked list is iterated through, and only the different characters and the number of spaces in between each change is recorded. Hex codes, 0x16 for characters and 0x15 for space counts are used so the program knows how to read it back and reconstruct the linked list.

Other features

Since this is just a project I started a few weeks ago, it obviously won't have all the features of a system like Git or SVN. Most importantly, this program currently only keeps track of files locally, since there is no server component for a remote repository. Some of the basic features seen in Git, such as logging and an ignore-file, are present.

Branching

New branches are created with the rollback command, along with its own directory for changefiles and a separate log file. Branch names are autogenerated with the prefix sub- plus the first 8 characters of the source commit id. Users can switch between branches using the checkout command.

Project repo: https://github.com/gbafana25/zren

[1] https://www.git-scm.com/book/en/v2/Getting-Started-What-is-Git%3F

Undocumented APIs in websites

Gareth M. — Sat, 08 Apr 2023 19:23:52 +0000

Intro

A website needs to be able to retrieve data, such as information on a set of products or users. To do this, many websites make requests to APIs, which in turn access data from a database or some other backend service. Today I'm going to be talking about how I was able to scrape JSON data from an undocumented API, in order to integrate that data into my own application.

The website

Currently I am working on an application that allows users to create a grocery list (here), as well as compare items. For this I need several pieces of data, such as the name and price of the product(s), as well as nutrition information, ingredients, and images of the product and/or its packaging. Conveniently, I found all of this information in the API at https://www.bakersplus.com/atlas/v1.

Retrieving the data

Preliminary request to get product ids

Secondary request that returns product data

Upon looking at the web traffic, there are two web requests here that are important. The first one access the endpoint /search/v1/products-search, which takes parameters filter.query and page.size. This endpoint returns product id numbers in a JSON object.


h = {"User-Agent": user_agent, "Accept":"*/*", "Host":"www.bakersplus.com", "x-laf-object":json.dumps(x_laf_obj)}

def getProductIds(query, num):
    #query = "rice"
    ids = []

    req = requests.get(base_url+"/search/v1/products-search?filter.query="+query+"&page.size="+str(num), headers=h)

The second request is made to the /product/v2/products endpoint. An array of the product numbers (filter.gtin13s) is passed as a url parameter. What we get as the response is all the information we need, although, including some extra info we don't want that is filtered out via a helper function.


def getProductInfo(ids):
    p = {"filter.gtin13s":[]}
    for i in ids:
        p["filter.gtin13s"].append(i)   

    req = requests.get(base_url+"/product/v2/products", headers=h, params=p)

Rate limiting/Request restrictions

APIs, especially when publicly exposed, will limit the amount of consecutive requests. In my experience with the above example, it is more likely to timeout if the same search is made consecutively. In addition, this API also required certain HTTP headers to be included, which others did not require. One of these was called x-laf-object, which appeared to be some kind of location tracking object.

Conclusion

Many websites don't make it this easy to scrape their data and effectively bypass their frontend. However, if the data these services provide is public anyway, and is effectively rate-limited and protected from abuse, then there isn't really a reason to not build a solution that is modular and easier to fix/debug.

Livestreams/VODs of dev: Youtube

Code: Github

Tuxsav - backup and save text files

Gareth M. — Sat, 18 Mar 2023 19:06:09 +0000

TLDR: backs up text files during and after editing w/ Vim swap files. https://github.com/gbafana25/tuxsav

Intro

Hello, this project is still very much in the beginning stages, but the basic idea I had in mind is operational (Link is at the bottom).

Design

The repository consists of a Django web application and a C++ client. The Django application exposes endpoints that the client can use to create new documents and update existing ones.

On a the local machine, the C++ client must be started before opening a file. While the file is still open, it reads the Vim swap file and sends that data to the server. Once the file is closed, it reads the original file and sends its contents one final time. It currently only backups up one file at a time, which is specified in the command line arguments.

What's Left

I haven't done any styling on the website's frontend, and there are only the bare minimum amount of pages for it to function. I also potentially want the program to backup multiple files at once.

https://github.com/gbafana25/tuxsav

hw-manager: Canvas assignments and todo list application.

Gareth M. — Fri, 20 Jan 2023 21:55:17 +0000

About

hw-manager is a Flask application that helps organize class assignments and has a basic todo list feature. It uses the Canvas API.

Usage

The API token and URL for your school's Canvas page can be entered on the setup page.

gbafana25 / hw-manager

View Canvas assignments and create to-do list.

hw-manager

Flask-based frontend to view Canvas assignments. Try here

Stored Cookies

token: Canvas API token
url: Canvas page url
todo: array of todo list items
removed-classes: list of course ids to hide

TODO:

handle server errors when token not found
remove to do list not found print statement

View on GitHub

typetester - typing practice for the command line

Gareth M. — Thu, 28 Jul 2022 23:35:00 +0000

Typetester lets you to practice typing from the command line. Text passages are loaded into a text file, and statistics are generated after every run

Add a passage into a.txt, compile and run ./main. Once completed, the results will appear like so:

Terminal Modes

In order to read input, typetester chnages terminal settings using termios.h. This library contains flags that control various functions of the terminal, such as the input processing, output, etc. A full list of them can be found here. For this program, canonical mode and output echo were disabled. Canonical mode is the default mode of most terminals. Input is available line-by-line, usually after the user hits Enter. This also allows the user to edit the line, which is useful for typing commands.

In non-canonical mode, user input is available as soon as the character is typed. In the main loop of the program, all we have to do is check if each character equals the current character in the passage.

void setup_terminal()
{
        struct termios options;
        tcgetattr(STDIN_FILENO, &options);
        // enable canonical mode
        options.c_lflag &= ~(ICANON);
        // disable echo
        options.c_lflag &= ~(ECHO);
        tcsetattr(STDIN_FILENO, TCSAFLUSH, &options);


}

ANSI Cursor Controls

When a session starts, the cursor is placed at the beginning of the passage. In order to get the correct cursor position, we save it by sending ESC 7 to stdout. Then after printing out the text passage, we restore the cursor position with ESC 8.

// save current cursor position
write(STDOUT_FILENO, "\e7", 3);
char *text = read_from_file("a.txt");
printf("%s\n", text);
// restore saved cursor position
write(STDOUT_FILENO, "\e8", 3);

Summary

I made this project to experiment with terminal settings, in addition to making something that I normally have to access through the browser. This project can be found at this github repository.

passtwo - mobile password manager

Gareth M. — Sat, 09 Jul 2022 02:48:00 +0000

Update: release apk now available here

Passtwo is my second attempt at making a mobile password manager app. It is designed to integrate with passman, a python script that uses GPG to store passwords. You can see how it works in this DEV article. This app relies on BouncyCastle to decrypt files.

OAuth Device Flow

This app uses the beta Device Flow for OAuth apps. First, when the user presses Request Device Token, the app sends a request for a device token containing the OAuth app's Client ID. The response contains an 8-character authorization code, and the browser automatically opens to type it in (The code is displayed on the app's settings page). The user will be prompted to login before entering the code. Finally, if the user has agreed to allow the application, the Get Token button must be pressed in order to finish the authentication process.

User Info

After authenticating, the user simply inputs their github username, the private repo holding their passwords, and their gpg passphrase. The only thing required in the repo are the folders containing the encrypted passwords, and a gpg private key. The app currently doesn't support adding new accounts (directly from the app), however I hope to add that in the future since it would be improvement over the previous version. Each time the user tries to access the Passwords page, they will be prompted to enter their fingerprint or pin before proceeding

Installation

A release apk can be found here.

Passman - a password manager for the command line

Gareth M. — Mon, 19 Jul 2021 18:03:42 +0000

Introduction

Passman is a python script that encrypts your passwords with gpg. The inspiration for this project came from , a shell script that does the same thing.

Dependencies

gpg and xclip should be installed. No external python libraries were used.

Structure

Passman stores credentials in the following file structure:

-- Website name
       |
       -- username_1.enc
       |
       -- username_2.enc

Passwords are copied to the clipboard with xclip, and are cleared from the clipboard after one paste. Configuration is done via a json file, which must be named config.json, and be in the same directory as passman.py. Since there are no external dependencies, passman.py can be moved around and placed wherever, as long as the store_path variable in config.json is set correctly.

Anyone here use Solus Linux

Gareth M. — Mon, 15 Jul 2019 01:25:23 +0000

Just wanted to see if anyone used Solus Linux for anything, and what your opinions are on it.

THE SCRIPT KIDDIE HOLE OF DOOM (aka the ESP8266 honeypot)

Gareth M. — Sun, 28 Oct 2018 19:41:54 +0000

This project is designed to make a simple honeypot that runs a Telnet server. The server is then port forwarded so it can be accessed by anyone. I wanted to see if people would still bother scanning or hacking Telnet devices since most scanning tools are automated now. Please feel free to suggest any additions or changes, v1 is now out (Honeyboi 1). Project link here