DEV Community: Namatuzio

Custom Themes and Dark Mode Release

Namatuzio — Tue, 12 Dec 2023 02:54:07 +0000

Well, this is it! Exciting stuff I know. Both issues on the integration of dark mode and custom themes have been completed! This marks the end of the 3 post series outlining my experience implementing these 2 functionalities.

However, it doesn't feel right to not add a little more information on my experience with the custom theme functionality, so here are some notes I wrote down while reimplementing it which didn't make it into the previous post:

Unreleased Notes

My progress on the custom themes has been going great! Right now I'm planning on adding 2 new themes, a Gameboy theme, that will work off the limited palette of the original Gameboy and a Pokemon Home theme, based on the colours used in the Pokemon Home app.

Gameboy palette example

Pokemon Home reference images

My prediction was correct, using the scraped data would've been a lot for the web app, even though I spent quite some time gathering all the portraits for the Pokemon, I deleted the data and found an alternate way to handle everything: states and ID tracking.

See the entirety of the styling right now is based on a dropdown, so essentially, when a style is changed:

<select id="themeSel">
    <option value="">Light</option>
    <option value="dark" selected={theme == "dark"}>Dark</option>
    <option value="gameboy" selected={theme == "gameboy"}>Gameboy</option>
    <option value="home" selected={theme == "home"}>Pokemon Home</option>
</select>

Tracking the value of the select component "themeSel" along with a state variable "theme" enabled me to track the selected theme directory-wide by simply referencing the ID of the dropdown. This lets me do fun things like dynamically change the styling of the background and other areas like the cards easily when using a custom theme. This is a pretty big change over the simple toggle I had previously implemented for dark mode.

Using a reference to the dropdown also allows me to dynamically make API calls, as opposed to storing all that data locally. If the Gameboy theme is selected, I want to be able to change the official art portraits to the pixel-based sprites seen in most of the games. Of course, if the home theme is selected, I want to use the home model to show off all the Pokemon. I already have an idea as to what needs to be done for the home theme because of some reference pictures taken from the Nintendo store's site. That's all for the update!

Anyways...

So yes, both functionalities are implemented and fully functional. It was super fun implementing the new themes as it almost required me to create a skeleton theme manager that can be used to implement any number of new themes! I ran into a few problems, mainly with how I handled storing all the themes, which was promptly fixed up by reducing the scale of what I was doing. For example...

    const toggleDarkMode = () => {
      const newTheme = !isDarkMode;
        const newTheme = !isDarkMode;
        setIsDarkMode(newTheme);
        setIsHomeTheme(false);
        setIsGameboyTheme(false);
        localStorage.setItem('isDarkMode', JSON.stringify(newTheme));
        localStorage.setItem('isHomeMode', JSON.stringify(false));
        localStorage.setItem('isGBMode', JSON.stringify(false));
    };

this chunk of code handled everything JUST for toggling dark mode... yeah it's ugly, and it made me realize almost immediately after creating this abomination that I should scale back on how I handle it. Now it's handled in a much neater fashion;

    useEffect(() => {
        const savedTheme = localStorage.getItem('theme');
        if (savedTheme !== null) {
            setTheme(savedTheme);
        }
    }, []);

    const handleThemeChange = (newTheme) => {
        setTheme(newTheme);
        localStorage.setItem('theme', newTheme);
    };

these 2 do it all! the useEffect is mainly just for extracting the theme when loading the site so that it remembers what theme you had stored in a previous session, whereas the handleThemeChange, works its magic when you change the theme while using the site. There's a lot more to it, but the bread and butter is right here, and it was a ton of fun to figure out and implement!

Completing my goals

It took a lot of time (mainly because of my overcomplication with the implementation) but it was a fantastic experience to work on these features. Dark Mode is interesting because working on it made me realize that it can be as easy or as hard as you want it to be to implement. It's one of these features where there are almost a million ways to implement it but luckily I realized a bit after overdeveloping it that there was a much better and neater solution. Custom Themes are almost the same, I spent a good couple of days crafting a comprehensive list using a mixture of web scraping and manual formatting only for me to realize how benign an implementation like this is for such a feature. Of course, it would've worked, but at what cost? Working on the dark mode feature gave me a good idea as to how I would need to handle the custom themes, and working on custom themes gave me an excellent idea as to how custom theme frameworks are built. Tailwind CSS made all of this a breeze by allowing me to create custom colours that could be easily referenced.

The PRs and issues can be found below, as well as pictures of the final look for all the themes!

Issues

add dark mode (my eyes hurt) #12

lstuma posted on Oct 16, 2023

View on GitHub

Custom Themes #22

Namatuzio posted on Dec 04, 2023

Hey again @TheShiveshNetwork, I would like to add custom themes to the site, if you want I could also package the dark mode stuff into there!

View on GitHub

PRs

Feature: Dark Mode Toggle and Styling #23

Namatuzio posted on Dec 08, 2023

@TheShiveshNetwork

Fixes #12

Added a dark mode toggle
Adjusted theme for when dark mode is enabled
Followed guidelines from the official Tailwind CSS Dark Mode guide

I'll post some screenshots of the UI updates but I want to know if you would like anything adjusted in terms of colours, placement, etc, for the implementation before any merge.

Current toggle placement:

Homepage colours:

Pokemon info styling:

I'll mark this as a draft for now and update it until it meets your standards!

View on GitHub

Feature: Custom themes #26

Namatuzio posted on Dec 11, 2023

@TheShiveshNetwork

Fixes #22

Updates:

Added new custom themes: Gameboy & Pokemon Home
Gameboy utilizes the limited colour scheme of the original Gameboy and utilizes sprites instead of official art for all 1017 Pokemon
Pokemon Home is based on the colours used in the app reference pictures can be found below
Made both themes storable
Updated the way tracking themes is handled to support any number of additional themes
Added a drop-down for the themes
Updated SelectComponents.jsx to support ids for tracking the theme changes (default is null).

I had a lot of fun with this, I hope all the changes are up to spec! Here are some pictures of the themes in action:

Gameboy:

Pokemon Home:

References for the home theme:

taken from the official Nintendo store page

Of course, feel free to request changes, I will add them ASAP!

View on GitHub

EDIT: 12-12-2023 Something seemed to have gone wrong with the merging of the previous PR according to the repo owner, so I made another identical one found here:

Feature: ReAdded Custom themes #28

Namatuzio posted on Dec 12, 2023

@TheShiveshNetwork

Fixes #22

Created a new PR as requested in #26

Updates:

Added new custom themes: Gameboy & Pokemon Home
Gameboy utilizes the limited colour scheme of the original Gameboy and utilizes sprites instead of official art for all 1017 Pokemon
Pokemon Home is based on the colours used in the app reference pictures can be found below
Made both themes storable
Updated the way tracking themes is handled to support any number of additional themes
Added a drop-down for the themes
Updated SelectComponents.jsx to support ids for tracking the theme changes (default is null).

I had a lot of fun with this, I hope all the changes are up to spec! Here are some pictures of the themes in action:

Gameboy:

Pokemon Home:

References for the home theme:

taken from the official Nintendo store page

Of course, feel free to request changes, I will add them ASAP!

View on GitHub

The final not final TIL

I learned a good amount from the implementation of these features; how to create and style a dark mode (in more ways than one) using both stylesheets and CSS frameworks, creating custom themes using these frameworks, saving styling themes to a local cache and loading it, and lastly, dynamic API calling through referencing IDs. I felt like I poured my heart into these final features, which was reflected in the code review conducted by the repo owner. It was truly a wonderful experience.

Closing thoughts

It's been an incredible 4 months since my open-source journey began and this is not the end of it. I want to deeply thank you all for reading my posts and keeping up! It was a great semester and I'm so happy I got to learn and experiment so much with open-source projects, something that means the absolute most to me. I hope you all enjoy your holiday season and once more, thank you for following the start of my open source journey!

Custom Themes & Dark Mode Progress

Namatuzio — Mon, 11 Dec 2023 04:59:39 +0000

Hey everyone, as outlined in my previous post, I've spent the last couple of weeks working out dark mode and some custom themes for an open source pokedex app. This is the update on the progress that has been made so far.

Progress

The issue dealing with dark mode has been completed! It was super fun learning how to weave this into an app and I went through a few different iterations before I realized how much more complicated I was making it.

My initial idea for dark mode was to create multiple style sheets that would be updated based on a toggle. For example, take this stylesheet;

/* index.css */
::-webkit-scrollbar {
    width: 7px;
}

::-webkit-scrollbar-track {
    background: #f1f1f1;
}

::-webkit-scrollbar-thumb {
    background: #e0e0e0;
}

::-webkit-scrollbar-thumb:hover {
    background: #a0a0a0;
}

These colours change the styling of the scrollbar and are predominately the standard colours for the most part. The track is white, the roller is grey, and hovering makes it more grey.

My initial implementation had multiple CSS files, for example:

/* index_dark_mode.css */
::-webkit-scrollbar {
    width: 7px;
}

::-webkit-scrollbar-track {
    background: #2f2f2f; /* Darker color for the track */
}

::-webkit-scrollbar-thumb {
    background: #4f4f4f; /* Darker color for the thumb */
}

::-webkit-scrollbar-thumb:hover {
    background: #6f6f6f; /* Slightly lighter color for the thumb hover */
}

This would be great for a standard static app, but I realized this might add extra fluff and cache-related storage when loading the site, especially because I was planning on adding more themes, so I scrapped the implementation and went back to the drawing board.

As I stated before I looked at some libraries for adding dark mode, which is when it hit me... The project already uses Tailwind CSS, surely there's something in there that will help right? So I did some digging and landed upon just what I needed, a dark mode page! Tailwind made adding dark mode a ton of fun! I handled everything through a toggle and one variable. To enable the use of dark mode, all I had to do was add the following line to the tailwind.config.js file:

// tailwind.config.js
export default {
  darkMode: "class",
  // ...

then from there, I could prefix some styling components with dark: which would utilize them when dark mode is enabled.

<div
    className={
        `bg-slate-200 dark:bg-slate-900`
    }>

The div is normally white (bg-slate-200) but if dark mode is enabled, it will be closer to a navy blue or black (dark:bg-slate-900).

Also before I forget, Here are some screenshots of the updated theme!

I love the sleekness of it all, tailwind does have some nice colours.

Either way this works great, and gave me some ideas for the implementation of my second issue: custom themes. I did start on that and have some json data from scraping the pokeapi that contains portraits, but after the realization I had with the CSS files, I may very well try to find an alternative for this.

Until Next Time

So that's pretty much all for this post, I figured out dark mode and got some brilliant ideas for fulfilling my final issue, custom themes. Thank you for reading and see ya next time!

Custom Themes and Dark Mode

Namatuzio — Thu, 07 Dec 2023 04:13:23 +0000

In the next couple of weeks, I'll be tackling a functionality that I've never really messed with before but always wanted to do, that being Custom Themes and Dark Mode!

The Schematic

Considering I've never worked with these functionalities before, I'll have to do some research. For dark mode, I believe I'll use a library as it will be relatively easy to incorporate into the code. So far, I've looked at darkmode-js which seems like a pretty nice package for me to use. As for the custom themes, I believe I may have to crack out a Python web scraper mainly because I have some big plans for that, of course, I have a backup in case I'm heavily time-constrained or problems arise.

Updates

I'll be posting a couple of updates throughout the development process, I hope you can stick around and watch my progress through these brand-new implementations! Until next time, see ya.

Digging Deeper with TypeScript: ESlint

Namatuzio — Tue, 28 Nov 2023 00:23:02 +0000

Last week I undertook another TypeScript-related issue which dealt with adding ESlint to the codebase with some guidelines given by the owner of the repo. I figured the issue would be easy enough, I've used ESlint before - how much different could it be? Well, it turns out it was a lot different than what I'm used to.

The Process

To begin, I installed ESlint by following their official quickstart guide. I was pretty surprised by how simple it was to set up for a TS project, but it ran me through some questions and in no time, ESlint was working!

I added the 2 rules that related with the repo owner's lint request which both related to importing files and types.

  "rules": {
    "@typescript-eslint/consistent-type-imports": ["error"],
    "no-restricted-imports": ["error", {
      "patterns": ["*.ts"]
    }]
  }

So, after adding the rules, I went through the standard procedure and linted everything by using npx eslint . with only 11 errors showing up... which was great, it should be an open-and-shut issue. Pretty much every linting error had to do with a type of any being used. I decided to look up the error and found out that typescript has support for a type called unknown. The official documentation even states "unknown is the type-safe counterpart of any." which is exactly what I needed!

So I got around the errors by changing all the any types to unknown. It was then that I realized this was going to be a whole lot harder than I initially thought. A lot of the object property calls were throwing errors because technically, it would be unknown if anything would exist on there. For example:

typeof value.name !== 'string'

.name would throw an error because, to the compiler, it doesn't exist on value.

Unfortunately for me, I didn't think like that when I first encountered the error, though it did give me some practice with casting and assertions in TS / JS so there's always a bright side.

I read in the documentation and in a lot of discussions on stack overflow that unknown types must first be funnelled into a more concrete typing before use. In order to fix the error, I decided to cast value almost every single time it was called to the expected object type. While not necessarily wrong, it was being done in functions that would check the type of an abstract object. If two objects had different structures, the function would still return false if it didn't match but it's overall just poor code conduct.

Feedback

After submitting the initial PR, the repo owner pointed this out to me.

I ended up overcomplicating a relatively easy fix both computationally and semantically. Thankfully, the repo owner was nice about it and gave some feedback for the requested changes. I applied those changes, opting for null checks which was in line with standard ESlint rules. I then had to change a function heading to take a generic type which after doing so, everything worked perfectly fine.

The PR still has yet to be merged and I'll definitely update the article once it has but for the time being, everything is done.

Progression since Hacktoberfest

This PR was the first time I've used ESlint with TypeScript and also the first time I've ever used the unknown type. It was a painful but necessary experience and it felt good to finally understand the ins and outs of this foreign type. It really goes to show that issues themselves might sound easy, but you'll almost never know what you'll run into until you take the first step.

Closing thoughts

These PRs were a fantastic learning experience and I'm glad to have chosen them as the ones I would tackle this month. Moving forward there's still so much I want to learn and contribute to. For now, I'd love to take a look at adding custom themes or dark modes to websites, as I've never really added such a functionality before. I'm more excited to try custom themes as I would assume they aren't as solved as dark mode considering how many npm packages must exist that do just that. Thank you all for reading and until next time, see ya!

Type-strict coding with TypeScripts "strict" mode

Namatuzio — Mon, 27 Nov 2023 22:25:14 +0000

Around 2 weeks ago, I decided to work on an issue that was unrelated to a lot of what I've done previously. For some context, I've worked extensively in C and C++, which are generally type-strict languages. Though, when thinking about the copious amounts of time I've spent developing sites and such, I've never really utilized TypeScript nor its strict mode. I became so used to the untyped freedom and tedium that came with JavaScript that I decided to dive into the complete opposite end of the spectrum. It was a fun experience working in TypeScript and strict mode reminded me a lot of working in VS with C/++. Anyhow, here's the process I went through while working on this issue.

The Process

Thankfully, the issue itself gave me a direct link to the documentation for strict mode. I took a look and sifted through some of the checks and requirements for enabling and maintaining a strict-mode codebase, and then shortly got to work.

I browsed the folder that contained the blockly plugin and went straight into the tsconfig.json for it to enable strict mode, simple and easy. After doing so, I ran npm run build which showed many of the errors I had to fix because of strict mode type checks.

Many of these errors were simple enough, TS2531: Object is possibly 'null'. simply refers to a check for null that must be done to ensure a null value isn't utilized when it's not expected. Other errors like TS2564: Property 'minimapWrapper' has no initializer and is not definitely assigned in the constructor. simply mean something is declared in a class but is not nullable, therefore they must be assigned in the constructor. It's funny seeing a lot of exceptions and errors that I would run into in C++ appear in code that is almost JavaScript, almost uncanny to me, but this also meant I had a decent idea as to how to fix them.

For example, a lot of the cases were solved with the following:

  // old
  protected minimapWrapper: HTMLDivElement;
  // new 
  protected minimapWrapper: HTMLDivElement | null = null;

This simple change means that the variable can now expect to be null by default, meaning there's no need to assign anything in the constructor. However, this creates its own slew of additional issues. Now that we know we can expect minimapWrapper to be null, what happens when we are actually expecting a value? Well, that's where null checks come into play. Luckily there are many different ways to handle this. Whenever the code requires minimapWrapper to have a value, strict mode will yell at us UNLESS we include this simple check:

if(!minimapWrapper) {
  // Do something...
}

This was essentially the majority of the changes that were needed for everything to work, though there was a strange issue that occurred in the testing files.

For some odd reason, a reference to the DOM was not being linked, meaning that tests on the specific functionality of the plugin could not be done due to needing a DOM reference to grab the positions of objects. After doing some digging, I found that adding

const jsdom = require('jsdom');
const { JSDOM } = jsdom;

const { document } = (new JSDOM('')).window;
global.document = document;

to the top of the file fixed everything, most likely because we're forcing the reference to the DOM to be made by assigning it through an empty HTML doc.

Changes Requested, Changes Made

After submitting my initial PR, explaining what was changed, and some info on the npm run test issues I was facing, various changes were requested. A lot of them were very simple changes, whether it be to make something not nullable and instead assign it in a constructor or fixing up the formatting of a certain function. The PR still hasn't been merged, but it's most likely due to the issue with running the tests, the reviewer said they would extensively look into it.

Progression since Hacktoberfest

This was the first time I've made pretty big changes to an existing, and complex codebase. It came with a lot of surprises and I was pleasantly surprised at how easy it was to understand everything right from the get-go.

Closing thoughts

It was actually pretty fun working with TypeScript and I did enjoy making all the comparisons between it and C++. Seeing certain errors that I never would've expected to see in JS was pretty awesome and it really felt like I was programming in a very familiar yet different language. I look forward to trying my hand at some more TS projects. Thanks for reading and until next time, see ya!

Building and Releasing a Python CLI

Namatuzio — Sat, 25 Nov 2023 04:30:17 +0000

So this week I packaged and released the final version of my CLI, Tiller. It is now officially downloadable through PyPI as well!

This was something totally new to me so there was lots to be learned!

Package and Release method

One of the first things to consider when releasing any kind of project is where users will be able to find it. Considering I used Python, I decided to go with a popular package hosting site called PyPi. The best part is that there's a very useful guide on how to do just what I wanted to do!

This guide helped tremendously with getting everything uploaded and working, but don't worry if you decide to go with other package distribution sites or platforms, there are bound to be many tutorials on how to use it!

So to get started, I refactored my file structure within my repo to separate test and functional code. I also added an empty __init__.py file, which I was unsure of its use until today. __init__.py is a handy file that tells Python what to do when a package is first installed. I also learned that without an init file, Python packages are treated as namespace packages, which for 99% of use cases, is completely unnecessary, especially for my CLI which should be run in a self-contained package.

Another concept I learned was about build backends, an import step which is used to initialize and install any dependencies of the app you're packaging. Since the tutorial went with using Hatch that is also what I went with, though it didn't provide a lot of useful details especially because it didn't show how to add any dependencies, so I took a look at the docs which were very nice and simple to follow.

Then I set all the metadata required for my app such as version numbers, names, authors, etc. and moved on to building my application. This was done using py -m build which generated 2 types of files; a tar.gz which is a source distribution file and a .whl which is a built distribution file.

A source distribution file is a file that contains raw information about a package and contains only the essential files required for installing from tools like pip or being turned into a built distribution.

A built distribution file is a pre-built package file that simply needs to find the right home for the package to run as expected. Both were concepts that were somewhat new to me. I've seen built distribution files previously but had no idea what the file extension really meant.

Anyways, I then installed twine which would allow me to finally upload my CLI to PyPi. Running py -m twine upload --repository testpypi dist/* took everything from the previously made dist file and uploaded it onto PyPi to be openly downloaded. After some bug fixing and a few patch release later, everything was running smoothly, and tiller was finally ready to be used.

Problems

During the packaging process, I ran into a few problems, it's the reason why my tags are all over the place in terms of versioning. Some of the changes were so minuscule I overlooked adding a new patch version.

A lot of my problems came from my pyproject.toml file which after uploading everything to PyPi, would not properly load everything which made me lost on how to run my app. It was at this point that I actually decided to take a deeper look at the hatchling docs to find out I was missing the dependencies part. I also decided to add an entry point into my app, allowing users to simply the call to running tiller by adding the following to my pyproject.toml

[project.scripts]
tiller = "src.namatuzio_tiller_package.main:app"

This lets users call all my functionality by calling tiller as opposed to ./main.py when the app is installed.

Another issue was forgetting to update certain aspects of my code. For example, when I tested my -v functionality, I had the check set to a default string, when I could've simply imported my __version__ variable from the main file to have it be the same in multiple locations.

Testing other CLIs

The CLI I decided to test was Txt2StaticHTML a package of similar functionality, though done in C#. It was interesting testing it mainly because of how different NuGet packages are from Python packages. I had to open up VS2022 and install the NuGet package. Everything was inside of the namespace the repo owner had developed using the namespace allowed all of the functionality to become available. Calling

Converter.Run(<DIR>);

Ran the default functionality, and even more could be called using a string array. I didn't have much feedback to give especially since the README drew everything out for me. The functionality was all there and it was pretty simple to use and a great refresher on C#.

Having my CLI tested

The owner of Txt2StaticHtml also tested my package out. He installed everything from the PyPi page using pip and followed the rest of my README to test the functionality. There wasn't a lot of feedback from the tester because I tried to make my README as descriptive as possible.

Try it for yourself!

Did you want to try out Tiller? Well if that's a yes then worry not! It's extremely simple to download and use.

Installation

Please note that Tiller requires Python 3.11 or higher to run due to the use of TOMLib.

pip install -i https://test.pypi.org/simple/ namatuzio-tiller

Package Usage:

tiller [OPTIONS] DIR

Options:

--version, -v  Print the current version of Tiller.
--help, -h     Print the help message.
--config, -c   Specify the path to a TOML config file to be used.
--output, -o   Change the output directory of the .html files.
--lang, -l    Specify the language of the generated HTML file.

Display the current version of Tiller:

.\main.py --version (or -v)

Tiller Version: 1.1.0

Transform a file through a relative path:

tiller .\example.txt
Converted example.txt to example.html

.\example.txt

Hello

World\\

How are you?

.\til\example.html
<!DOCTYPE html>
<html lang="en">
    <head>
        <meta charset="UTF-8">
        <title>Hello</title>
        <meta name="viewport" content="width=device-width, initial-scale=1">
    </head>
    <h1>Hello</h1>
    <body>
        <p>Hello</p>
        <p>World</p>
        <p>How are you?</p>
    </body>
</html>

Generate HTML file with a different language:

tiller --lang fr .\example.txt

Converted example.txt to example.html

.\example.txt

Hello

World

How are you?

.\til\example.html
<!DOCTYPE html>
<html lang="fr">
    <style>
    body {
        background-color: rgb(0, 116, 145);
        text-align: center;
        color: white;
        font-family: Arial, Helvetica, sans-serif;
        font-size: xx-large;
    }
    </style>
    <head>
        <meta charset="UTF-8">
        <title>example</title>
        <meta name="viewport" content="width=device-width, initial-scale=1">
    </head>
    <h1>example</h1>
    <body>
        <p>Hello</p>
        <p>World</p>
        <p>How are you?</p>
    </body>
</html>

Transform file(s) using a TOML config file

tiller --config (or -c) .\examples\TOMLExample.toml .\examples

Converted example.txt to example.html
Converted example2.txt to example2.html
Converted example3.md to example3.html
example4.html is not a .txt file or a .md file. Skipping...

TOMLExample.toml is not a .txt file or a .md file. Skipping...

CI/CD Part 2: GitHub Actions

Namatuzio — Sat, 18 Nov 2023 04:33:51 +0000

Namatuzio ・ Nov 10

#testing #python #opensource #beginners

Hey everyone, in a continuation post from my original CI/CD guide/exploration/TIL blog, here's part 2, and as described last time, today is all about GitHub Actions. As always, here is the PR for this week.

What are GitHub actions?

GitHub actions are simply put, a CI/CD pipeline, with some extra spice added in the mix. For today, I'm going to focus specifically on pipeline creation, which is thankfully, incredibly simple.

The process

So, to start off, I browsed through the Build and Test section of the Actions wiki page to get to the Python GitHub actions setup. This page was extremely useful and provided everything I needed to get a basic CI setup.

Everything that is required for setup can be done through the action tab, by creating a new workflow

Upon clicking New Workflow, a ton of options are displayed on the screen. 99% of the time, your focus will lie predominately in the Suggested for this repository section.

For my app, I went for the Python Application configuration which opened up a yaml file which can be configured however you see fit for automatically testing and linting your commits and PRs.

This file can of course be updated whenever you want, which is crazy useful in the event that the scale of your app increases significantly. In a previous project, I had a CI/CD pipeline yaml file that would automatically deploy my app to a docker daemon while linting and testing all the functionality, which really drives home the concept I learned in my previous post of integrating CI/CD as early as possible to reduce headaches later on.

Anyway, because I did use some different libraries I had to change my file up a bit. Firstly, I'm using Python 3.12.0 instead of 3.10.0. Secondly, I'm using Ruff as my linter of choice as opposed to flake8 and thankfully GitHub has that covered in their documentation. I simply replaced lint with flake8 with the following code and everything worked perfectly:

- name: Lint with Ruff
  run: |
    pip install ruff
    ruff --output-format=github .

My finalized workflow file looked like this:

# This workflow will install Python dependencies, run tests and lint with a single version of Python
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python

name: Python application

on:
  push:
    branches: [ "main" ]
  pull_request:
    branches: [ "main" ]

permissions:
  contents: read

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v3
    - name: Set up Python 3.12.0
      uses: actions/setup-python@v3
      with:
        python-version: "3.12.0"
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
    - name: Lint with Ruff
      run: |
        pip install ruff
        ruff --output-format=github .
    - name: Test with pytest
      run: |
        pytest

The on: flag at the top of the file controls when this workflow fires. So pretty much, on a commit to my main branch, or a PR to my main branch, the workflow will trigger, lint my code and ensure there are no errors, and then it will test all the functionality.

So like every great programmer, I had to break my code to ensure this worked. In a series of commits to my repo, I decided to break some linting rules within my test file to see if it would trigger.

As shown by the red X next to some of the commits, it worked! Clicking on the red X lets us investigate the reason for the failure:

AHA! So the piece of code that breaks the rule is in my test file on line 158, and look at that, it even explains the rule that the line broke. Browsing into the PR gives us an even more useful view of the error:

It even outlines each line that features an error too!

So after getting everything all fixed up, I merged my changes and finished adding a basic CI/CD pipeline to my app!

That isn't all for this week though, I decided to take a look at someone else's app and add some more test cases to their workflow. I posted an issue to the repo and added a super simple test case for when a file didn't exist. This repo is made entirely in C# which makes it intuitive to work with and easy to develop for. The final test for that repo looked like this:

[Test]
public void FinalizeOutput_NonExistentFile_ThrowsException()
{
    string nonExistentFile = Path.Combine(TestDirectory, "nonexistent.txt");
    Assert.Throws<FileNotFoundException>(() => Helper.FinalizeOutput(nonExistentFile, OutputDirectory));
}

And the same person who owned this repo also added a super simple case to my repo that I had completely overlooked which was pretty much the same case I did for their app. Their issue can be found here

So that about wraps up this week.

TIL

Today I relearned how GitHub actions work and how to easily develop them for any kind of project. I really did forget just how easy it was to integrate CI/CD pipelines which is why this was a great refresher and an awesome learning experience. I loved how easy some of the Python libraries I used made it for me to test, and I adore how intuitive GitHub Actions was to use.

I hope my posts helped out other newcomers or those just simply looking for a refresher on how to create these powerful pipelines! That's about it for now, thank you to everyone who read the first part and to those who read this part as well, see ya later.

CI/CD Part 1: Unit/Integration Testing

Namatuzio — Fri, 10 Nov 2023 23:48:06 +0000

This week, in a setup for a CI/CD pipeline, I added unit and integration testing using Pytest to my Python CLI and utilized pytest-cov for generating a coverage report. As always, the merged commit for changes to the repo can be found here.

What is CI/CD?

CI stands for continuous integration and CD stands for continuous delivery. CI focuses on the foundations of creating new features and testing them rigorously, whereas CD focuses on what to do after code is added and tested. If you've ever worked in a DevOps environment, chances are CI/CD pipelines were something you worked closely with.

The Process

So, I knew CI/CD was going to be an inevitable beast to be tamed. Having done it previously, Unit/Integration testing is one of, if not the most important steps to a solid foundation for continuous deployment. Having never done it in Python this was a new experience, though because of the scale of my app, I figured it wouldn't be as bad as the first time I worked with it.

After looking into various libraries for testing, I decided to use Pytest because it seemed very easy to use and was somewhat similar to other testing libraries I've used previously.

I feel like I lucked out in choosing Typer to create my app because it has an entire testing library within it which came in great use with Pytest, which I only found out about after I attempted writing my first test.

It turns out that the way I was handling my Typer app made it difficult to actually call any CLI arguments programmatically. It was funny because it was a single line of code that prevented me from progressing, more specifically this:

app = typer.Typer(add_completion=False)

I can't remember why I even added add_completion=False but it was breaking my tests, so I removed it and got to work.

The main idea behind unit and integration testing is adding as many small tests as possible to ensure every corner of your app is tested and 100% works as intended. So for my first test, I tested the -v flag.

runner = CliRunner()

# test_main.py
def test_main_version_arg():
    result = runner.invoke(app, ["-v"])
    assert "Tiller Version: 0.1.0" in result.output

As shown in the first test, I wanted to have clear function names for tests (which will come in handy later). This will let me easily organize tests for different functionalities of the app. The runner variable is a CliRunner which calls the CLI programmatically, which is extremely useful for what I need. Calling runner.invoke() will call the app (my main function) and will pass the -v flag to invoke the version flag. The 2nd line of the function uses the bread and butter of unit testing in Python, the assert keyword. assert essentially acts as an if statement, if the condition after it is true, the function worked as intended, but if it's false, it will raise an AssertionError and a custom exit message could be set. This is essentially how all the tests will be developed.

The next problem arose when I tried to create a test for my output flag.

## Test Output Flag
def test_main_output_arg():
    with runner.isolated_filesystem():
        examples_dir = os.path.join(os.path.dirname(os.path.realpath(__file__)), "examples")
        shutil.copytree(examples_dir, "examples")
        result = runner.invoke(app, ["-o", "output", "examples"])
        assert os.path.isdir("output") is True
        assert len(os.listdir("output")) > 0

The above code is the final version of the test, but I ran into a couple of issues when trying to create it. First off, when using the runner.isolated_filesystem() function I had issues asserting parts of the output, specifically if the files were actually being copied over. After trying to use shutil.copytree() I realized it was just throwing over a blank examples folder, so using good ol' os I was able to string together the absolute path that contains the script with the examples folder to create a way to copy the exact contents of the examples folder to out virtual isolated environment. isolated_filesystem() was another life saver for testing as it let me create a temporary place to run my tests which wouldn't reflect in the actual folder of my app and is actually a part of the backend that fuels Typer, Click.

Pytest was great to work with and extremely simple to use, all I had to do was create a file that had a test_ prefix and then call pytest in my CMD which ran every test and threw errors whenever one of them broke.

I wrote around 18 tests, made sure they all passed as expected, then installed pytest-cov and was happy to see that I had 90% coverage, only missing out on a few lines that weren't functional code. After that, I updated CONTRIBUTING.md to give users info on how to test and generate coverage reports, added both pytest and pytest-cov to my requirements.txt folder, and merged my branches.

A learning experience

In all honesty, I'm blown away by how fluently everything worked together. I've used Jest before for testing a microservice js app, and writing those tests was a pain. Sometimes it would just break for no reason, or some small config line was wrong, but with Pytest setting up CI/CD was amazing. I loved how simple it was to setup and I loved how the library I was using had its own library for helpful testing functions.

However, I guess the complexity that came with using Jest was mainly because of how late into development I ended up adding it. The app was already a beast, and in comparison, my CLI is still simple, which made me realize an important lesson. The earlier you add a continuous deployment pipeline, the better. In part 2, I'll be looking towards setting up this testing framework with GitHub Actions. Thanks for reading, and see ya next time.

Code Standards and Consistency with Ruff Linter and Formatter

Namatuzio — Fri, 03 Nov 2023 09:47:48 +0000

This week, I decided to add a Linter and Formatter to my codebase. I've used them previously with a project called fragments, which was handy for keeping JavaScript and HTML code standards in check. However, using it for Python is completely foreign to me, especially considering how much easier it is to write stronger code using Python.

Adding CONTRIBUTING.md

Outside of adding a linter and formatter, I finally added a CONTRIBUTING.md file to my project, in hopes that anyone who wishes to make it better than it is, can do so easily. I learned of a very awesome generator for these md files and decided to use it for mine.

Adding Ruff

Ruff is a dual linter and code formatter made in Rust for Python. Very helpful for what I needed and a huge change from previously using eslint and prettier for everything. Also unlike eslint and prettier, ruff comes pre-packaged to work right off the installation. Another interesting aspect of ruff's formatter is that it's actually compatible with Black another code formatter used in Python. For ease of use and limiting the amount required to install, I just decided to use Ruff.

The formatter is extremely easy to use - simply running ruff format . will fix formatting for a file - and highly customizable and can easily be adjusted by creating a pyproject.toml file, though, the only customization I needed was to exclude certain files and folders from being formatted done as shown below:

[tool.ruff.format]
exclude = ["examples/", ".venv/", ".ruff_cache/", "*.md", "*.txt", "LICENSE", ".gitignore"]

The linter was also just as easy to use - simply running ruff check . --fix linted a directory and instantly fixed standard errors - and came with a slew of options for customization and configuration. For the ruff-lint configuration, I used one of the templated setups found in the official configuring ruff page to handle all the linting needs.

Another benefit to using ruff is that I simply added it to my requirements.txt file allowing anyone who may want to run the app to install it and be able to utilize it. VSCode has an official plugin for it as well which allowed me to turn on automatic linting:

Ruff features a neat feature where it will check for specific comments to skip over re-formatting a specific chunk of code. Adding # fmt: off and # fmt: on around a code block would prevent it from getting formatted.

How effective were they?

Well to be quite honest, linting my file didn't do too much, mostly because I tried to keep my implementation as bare-bones as possible but there was one error, which was an unused import:

main.py:14:31: F401 [*] `typing_extensions.Annotated` imported but unused
Found 1 error.
[*] 1 fixable with the `--fix` option.

When formatting my file, it changed a good deal, mainly in the way it handled my main function header and the formatting for it. Other than that, it seemed to have changed mostly small indentation errors here and there.

What's next?

I want to try and implement a pre-commit hook onto my code, so that it can auto-lint and format everything before each commit. I wasn't able to get it this time around due to time constraints but it's definitely something I'd love to look into since it's very foreign to me.

TIL

Today I learned about Ruff, a very awesome library that contains not only a source code formatter, but a linter as well! I learned the basics of utilizing the formatter and the linter as well as basic setup for both. It's awesome stuff and really goes to show just how amazing Python and the community surrounding it is.

Post-Hacktoberfest Recap

Namatuzio — Tue, 31 Oct 2023 20:47:08 +0000

Welp, that's it, Hacktoberfest is coming to a close, all 4 of my PRs have been completed, and most importantly, it's Halloween!

Early Contributions

The most important but scary thing about Hacktoberfest for me was finding issues that are not only good for my growth but can also be great contributions to a community of people. It was worrying having to scramble around looking for such issues, but luckily GitHub has you covered with their comprehensive search functionality. I found the issues I wanted to work on by scrolling through the Hacktoberfest label on GitHub. During the earlier days of October, it is completely swamped with issues both big and small, complex and simple. Making it super easy to find the perfect one for any skill level. Furthermore, for my very first issue, I added the good first issue tag which was groups of issues that are relatively low in scope and easy to understand. The great thing about GitHub searches is that you can also further filter these issues by looking for specific languages. If you feel more comfortable with Python, simply filter all your results to display repos with their main language being Python.

PRs

I had a fantastic time working on all of my PRs and truly loved how kind all the repo owners were when giving feedback. If Hacktoberfest taught me anything, it's that the open-source community is absolutely great! I loved the idea of contributing to someone's project and getting real experience working with others to deliver a solid product.

PR1 Had me create a Python timer class to get the runtime of functions and entire scripts. I shared my thoughts and process here

PR2
Had me implement an emoji keyboard into a web app which was done with a library and a whole lot of blood, sweat, and CSS. I shared the process of developing and finding a proper library here along with my battle with CSS.

PR3 Had me update an existing functionality to accommodate for a larger amount of data, and weave in loading techniques to not break the app. I shared my process and hardships here

PR4 Had me eager to return to the previous functionality and add a system of advanced filtration to the data, allowing users to easily sift through data they may deem important. I shared my thoughts and the overcomplex process here

TIL

I learned a whole lot from Hacktoberfest, honestly, probably more in a single month than I have in the last 3. I learned a lot of new concepts and implementation techniques, and a lot more about myself and the mistakes I make during the implementation of a functionality. I feel like I've made a lot of progress in both knowledge and hands-on skills with a lot of concepts from the 1st PR to the last and it really is a great feeling. Most importantly I learned how great Hacktoberfest is, with this being my first one, I will definitely try to not make it my last. The open-source community around Hacktoberfest is truly something awesome. Thank you for reading, and I hope your Halloweens are fantastic.

Contributing to the Community: Hacktoberfest PR #4

Namatuzio — Tue, 31 Oct 2023 20:20:30 +0000

This is it, the final issue of Hacktoberfest 2023 and it's one that I made! So, since the last PR, I had a couple of ideas for the repo that I was working on. Considering the last issue was all about the search functionality, I decided I wanted to create a more advanced search function. As noted in the issue I filed, I decided to add a generational filter for all the Pokemon, meaning Pokemon that appear in Generation 1 will only be visible, same with filtering by Generation 2, 3, etc... Which I figured would bring a lot more challenges this time around than the previous PR. So I got to work.

Process

At first, I created a checkbox, but quickly realized checkboxes would not be nice if more filters were to ever be added. So I created a simple dropdown menu in Home.jsx and loaded it with the different Generations, 1-9. So now, I had to figure out a way of grabbing the Pokemon from all the different generations. I figured I would most likely have to use a req URL, so I began my implementation. While developing this feature I found out PokeAPI has a generations endpoint - GET https://pokeapi.co/api/v2/generation/{id or name}/ - which displays the following data:

"This is great!" I thought, "This is just what I need to implement this!". As hindsight is 20/20 I know this is absolutely not the case for what I was going for. After countless nights of broken filters and multiple API calls, I was stuck. I decided to dial back my implementation, clearly what I was going for was too complex for the app to work flawlessly. So, I decided to create values for every dropdown item and make them the number of Pokemon added in that generation and the number of Pokemon that were previously available. After some simple filtering, it worked perfectly. A common problem of mine is trying to overcomplicate functional design which is why I was happy that the solution was so simple.

Wandering Mind

Then, my brain decided to wander off... "What if I added a filter that could sort Pokemon by each game and it would show the version exclusives present in each, the Pokemon added in that game including alternate forms, and the domestic Pokedex".

Well, I thought it would be really cool so I got to work. I once again added another dropdown this time filled with almost every mainline Pokemon game from red and blue to scarlet and violet. I also increased the API call from 1017 to 1292, which encompasses every single Pokemon that the PokeAPI has numbered which will come into play later.

Process Part 2

So after contemplating how something like this would be possible, I went to the documentation page for PokeAPI and went back to the generation endpoint. I noticed the version-group JSON object and figured this might be just what I needed to grab all the information about the Pokemon from each version. I realized at this point that I would most likely need to utilize Python in order to scrape the API for every single Pokedex, this is when I dug deeper and came across the region endpoint:
GET https://pokeapi.co/api/v2/region/{id or name}/
which would produce version-groups of each game that features this pokedex entirely.

This was huge because it meant making an API scraper in Python would be so much easier, I would have to iterate a for loop 9 times grab the version-group data from the JSON response and add it to a JSON file. This also gave me an opportunity to use code from my 1st PR to test the runtime of my scraper! So after all was done, I had a generations folder that looked like this:

the code used to gather the information was also rather simple.

def get_data(url):
    response = requests.get(url)
    data = response.json()
    data = json.dumps(data, indent=4)
    return data

I would take the data and simply filter it out for the version-endpoint JSON object.

So with all that done, it was time to iterate through the generations.

The version-groups endpoint luckily gave me references to Pokedex, though they're not as game-specific as they could be, but that's fine.

So, I ran through each game's Pokedex, grabbing all the Pokemon under pokemon_species. The data for that looked like this:

# black-2-white-2.json
[
    {
        "entry_number": 0,
        "pokemon_species": {
            "name": "victini",
            "url": "https://pokeapi.co/api/v2/pokemon-species/494/"
        }
    },
    {
        "entry_number": 1,
        "pokemon_species": {
            "name": "snivy",
            "url": "https://pokeapi.co/api/v2/pokemon-species/495/"
        }
    },
...

And then further filtered everything to only grab the name and url split to only use it's dex number from pokemon_species until I had something a little more useable:

# unova.json
{
    "black-2-white-2": {
        "Pokemon": [
            {
                "name": "victini",
                "url": "494"
            },
            {
                "name": "snivy",
                "url": "495"
            },

I then separated each version into its own JSON object, meaning black-2 and white-2 were separated. Next is the fun part, the version exclusives. For the entire span of the mainline games, Pokemon has always released 2 versions together and sometimes a 3rd as a step up from the previous versions. the 2 versions often featured slightly different Pokemon to get people to trade with one another, or outright buy both versions, and the 3rd version would usually add a Pokemon unavailable in the previous versions but also have some Pokemon left out to once again, allow players to trade with others.

My best friend for this section was a website I've used extensively over the years, seribii.net not only does it feature most of what you would need for the games, but it also gives you a nice list of version exclusive pokemon for each and every pokemon game.

So I created a file called exclu.py and added each generational exclusive to an array and iterated the data in the regional files extracting certain Pokemon from the opposite versions

which I ended up doing 20 or so times, adding some tweaks here and there for the additional Pokemon forms and regional variants that have been added over the years. Which finally gave me 8 files with every domestic pokedex from every single mainline Pokemon game, excluding BDSP, and their version-exclusive Pokemon including version-exclusive forms such as Mega-Mewtwo X and Y. I combined the files into one mega file and used that as a filter for the main web app, which ended up working flawlessly and was done with absolutely no additional API calls allowing for both filters to work at once. Here's an example:

The live site can be found here!

The final PR can be found here, and with that, my first Hacktoberfest comes to a close!

What would I do differently?

As I said earlier, I have a really bad habit of overcomplicating functionalities. It would be best if in the future I take a step back and find out the fastest way to implement something and break down the functionality to its most basic roots. Thank you for reading and I hope you have a fantastic Halloween!

Contributing to the Community: Hacktoberfest PR #3

Namatuzio — Tue, 31 Oct 2023 19:05:04 +0000

My 3rd issue this Hacktober was in a fun web app that was looking to recreate the Pokedex. This particular issue was about improving the search functionality, more specifically by increasing the number of Pokemon from 20 to 1017 and the many variants that come with that number.

As someone who loves Pokemon and grew up with the series, I figured I'd give it a shot since it would be something fun.

Process

Based on the requirements realistically all I would need to do is change the request URL for the PokeAPI call. So, when I first forked the app, I explored the many files found within trying to find where my addition would go. Luckily everything was nicely organized within folders and I ended up finding it in a file called FetchApi.js. Simply changing the number from 20 to 1017 would have technically met the requirements for the overall functionality. Though the way things were being handled when a user loaded the site, 1017 requests were simultaneously made to grab the image files of every Pokemon, which of course is terrible practice and ended up breaking the app.

The requirements also noted to reduce the number of API calls being made in order to speed the app up, so clearly the way this information was being handled wasn't appropriate for 1017 Pokemon.

To fix this, I decided to implement 'Lazy Loading' a loading technique used on webpages with a lot of dynamic data that allows users to batch call data when it's needed (i.e. when it's on their screen). After getting it implemented in the PokemonCard.jsx file, everything worked flawlessly! There was one small problem though, some data in the PokeAPI wasn't entirely up to date, most likely because some Pokemon were too new.

Even though Sinistcha has some art floating around, none of it is really official and it's mainly just in-game models that are used. I consulted the owner of the repo and asked how I should handle it. I proposed either leaving it or adding a default image and the owner said to add a default image for Pokemon with no image references. I added the default image and it definitely looked a lot better:

and with that, my job was done... or so I thought. Turns out, I completely forgot to fetch and pull from the main repo, and to make things worse, there were a ton of changes to it. The biggest change was a version of literal Infinite Scrolling which seemed to have completely broken the app. The way this was being handled was by constantly calling the API as you scroll down. The worst part about it was that it would constantly fling you back to the top of the page. Here's an example using my favourite Pokemon as to how it broke the search functionality:

The owner ended up asking me to remove the infinite scrolling stuff, and after getting that resolved, everything was done. The final PR can be found here.

What would I do differently?

Well, this was definitely a big step up from the previous PR but I definitely made mistakes that I have to learn from. Not keeping my fork updated, made my job way harder than it needed to be. It's definitely something that I have to keep in mind when developing an active repo and a lesson I won't soon forget.