DEV Community: Kathryn DiPippo

The Essential Sprint Ceremonies

Kathryn DiPippo — Mon, 07 Jul 2025 21:48:12 +0000

Background
Daily Stand-up
- Daily Stand-up: When
- Daily Stand-up: How
- Daily Stand-up: Why
Triage / Sizing Sessions
- Triage / Sizing Sessions: When
- Triage / Sizing Sessions: How
- Triage / Sizing Sessions: Why
Backlog Grooming
- Backlog Grooming: When
- Backlog Grooming: How
- Backlog Grooming: Why
Sprint Retro
- Sprint Retro: When
- Sprint Retro: How
- Sprint Retro: Why
Sprint Planning
- Sprint Planning: When
- Sprint Planning: How
- Sprint Planning: Why

Background

There are many other guides out there about Sprint Ceremonies. I am taking the time in this article to explain my perspective on the usefulness of each meeting and how to ensure that the time spent to coordinate on them is useful and not a distraction or a waste of time.

For each sprint ceremony, I've broken down into subsections:

When to have the meetings and for how long
How I have learned to effectively run the meeting
Why each meeting is necessary

Daily Stand-up

Daily Stand-up: When

Every day blocked off for 30 minutes but could take as long as 10 minutes even on large development teams.

Daily Stand-up: How

The most effective stand-ups I've been a part of made use of an Excel spreadsheet to allow all team members to write their answers before the start of the actual meeting. In the spreadsheet, dedicate the first column to the list of team member names in the order they will present (I am used to alphabetical order by first name). Title the next 3 columns:

What I did yesterday
What I will do today
Any blockers

Team members will fill out their rows either before or at the very start of the meeting time. Then, in order from top-to-bottom, members will recite their entries out loud. Any further discussion topics that don't belong in the previous rows will be marked at the bottom of the sheet as a "post-scrum".

Once every team member has recited their row and all post-scrums have been discussed, the meeting ends.

Daily Stand-up: Why

I personally enjoy the "what I did yesterday" and "what I will do today" entries as personal goals / checklists to work toward and to keep me focused. But as a team lead, the "blockers" column is the most important, followed by post-scrums. Nothing is more demoralizing than being blocked by a task needed to be performed by someone on the same team and waiting weeks at a time. A blocker that is a quick action item that is visibly written every day proves to be a solid reminder.

Stand-up also provides a dedicated venue as well for brief but constant team discussion. For teams with remote employees, there's a significant mental difference between reaching out for a Zoom call vs. leaning over into a coworker's desk to ask a question.

It's important to ensure that the stand-up meeting stays capped to 30 minutes and any further discussion is properly scheduled and scoped to the correct number of people. This isn't to avoid having long, team conversations but to instead ensure that the meeting itself does not subtly expand to too long.

As new team members acclimate to the company and its processes, it may happen that most stand-ups take place with no blockers or post-scrums to review. This is fine and good! Stand-ups should still be slotted to 30 minutes in the event that there is a larger discussion that needs to happen, even if you'll now experience regular 5-10 minute meetings.

Triage / Sizing Sessions

Triage / Sizing Sessions: When

Triage should be held at a regular cadence depending on how often new tickets come in. For some teams, this may need to be a dedicated block of time after every stand-up; for others, ideally once-a-week. I believe 30 minutes is enough, but adjust longer or shorter as needed based on intake.

Sizing Sessions are separately scheduled ad-hoc meetings when a larger development project is split out into smaller tickets. Because these Epics will typically entail much larger wholistic discussions, I recommend booking a full hour and ending early if deemed appropriate.

Triage / Sizing Sessions: How

Triage tickets are reviewed in order starting from oldest-to-newest. Sizing session tickets are reviewed in the appropriate order to work on them within the given project.

Everyone on the team will be participating in Scrum Poker as they review and size each ticket. I am partial to playscrummy.com as the tool to facilitate this.

For each ticket, each team member needs to mentally answer the following questions:

Is the ticket appropriately categorized (i.e. correct labels and priority)?
Are the outcomes for this ticket clearly understood?
Are the outstanding questions required of this ticket small? For example, the outstanding questions are related to actual code implementation details ("What unit-tests do I write?") and not issues with what conceptually is being asked ("Why are we doing this change when only 1 person has asked for it? Have we checked with other stakeholders on what their thoughts on this change are?")
Would I be able to address this ticket if it were assigned to me? If not, what other information do I need in the ticket to provide further clarity?
Is the scope of this ticket appropriate? Are follow-up tickets needed in order to break out what is being requested into smaller steps?

In practice, whoever is leading the session will reveal the ticket, read the description aloud, and ask for general feedback. If there is feedback, the lead may document these questions as comments onto the ticket. If there is significant feedback, they may then move onto the next ticket without moving the current ticket out of the inbound queue. This means the ticket will be re-reviewed at the next Triage / Sizing Session.

If there is no feedback, the team then uses scrum poker to assign the number of story points to the task. There are a number of guides out there for what story points should correlate to; the important part is that the concept of story points remains consistent to the team. For me, the typical vibe for each number is as follows:

1: Very quick, almost asinine. For example:
- A typo is reported on a documentation page and needs to be changed.
- A configuration update is needed that involves toggling 4 buttons after following a procedure doc.
2: Small change. For example:
- A code change, like a quick bug fix, is needed that requires no test updates
- Boosting code coverage by adding more unit-tests without a functional code change
- Writing documentation
- A code change that would be big but is a copy-and-paste of existing code with smaller changes added onto it
3: Regular change. Majority of tickets fall into this bucket. For example:
- Code changes with both an update to the base code along with adding unit-tests
5: Large change. For example:
- Any change where the path-to-action is uncertain, like investigating a new library or new architectural strategy
- A code change that requires a refactor that touches a significant part of the codebase
8: Too big! Anything given this size typically means that the ticket needs to be re-conceptualized. What uncertainty is causing this to be too big? Are there clear "sections" of outcomes that would be best to split out?
- An exception would be if the work itself is appropriate to group together, i.e. performing a small configuration change against 50+ repositories. Keeping an 8-story-point ticket would be considered more user-friendly to update than 50 1-point tickets and maybe more accurate to the amount of work performed.

After all members have voted, reveal the results:

If all numbers are the same (i.e. "3, 3, 3"), assign that number ("3") and move to the next ticket.
If all numbers are within 1 deviation of each other (i.e. "3, 5, 3"), assign the highest number ("5") and move to the next ticket.
If all numbers are 2+ deviations of each other (i.e. "2, 5, 3"), the members of the team who voted the lowest ("2") and highest ("5") need to be encouraged to discuss their reasoning. This means that there is a fundamental disagreement over the expectations of the ticket. Having this conversation now means that an incorrect assumption was successfully avoided in the future where it would be more difficult to course-correct. By the end of the discussion, members will re-vote to decide the final story point value and move to the next ticket.

Process repeats until all tickets are reviewed or the meeting time has reached its end.

Triage / Sizing Sessions: Why

Every ticket submitted to the team gets to be reviewed as a team. This enables knowledge sharing among team members and avoids having one person bear the burden of understanding and breaking down work. For larger teams or for teams with remote workers, this enables also having a shared understanding of what members of the team are working on.

The key needs for story points are two-fold:

Enable discussion between team members in cases where team members understood the request in a ticket differently
Better understand the speed ("velocity") of a software development team to best estimate project deadlines.

Having each ticket sized at the same time will also surface conversations that may have been skipped. For example, a team reviews a new ticket: verbally, everyone says they understand the contents of the ticket and what it set out to accomplish. However, after sizing the ticket, one of the team members scores it a 2 and another scores it an 8. That's a huge difference! Had the team not sized the ticket, there may have been unspoken confusions over what the actual request of the ticket contains. Keep in mind a delicate balance is needed for how long this session should take. I have seen sizing sessions going on for an extended period of time that result in all team members consistently choosing safe "3"s in the interest of wrapping up sooner without back-and-forth.

From the developers' perspectives, story points should not matter. They should not be used as a comparison metric between team members, and software development team leads should avoid setting up a display for points accomplished as a scoreboard. For other teams like those in IT, a scoreboard may be a fun challenge if the tasks are all very procedural. For software development, it's highly likely that 8 tickets that are each "1" story point will take a significantly shorter amount of time than 1 ticket with "8" story points. This may incentivize wanting to avoid larger, more difficult 5-8 point initiatives in favor of scooping up smaller tickets immediately as they come in, or the potential where manual processes that require smaller tickets to be created are pushed forward instead of seeking out automation.

Meanwhile, team leads should look at the end of each sprint to examine the number of story points accomplished and seek out a trend. This trending line - referred to as "velocity" - will provide a rough estimate for what a comfortable range of story points that can be taken on and completed for each sprint.

Backlog Grooming

Backlog Grooming: When

Once a week for an hour at a point in the week where there's a lull - i.e. afternoon on Fridays - set aside 30 minutes - 1 hour for going through the oldest tickets in your ticket backlog.

Backlog Grooming: How

Bookmark a query or filtered view of tasks that sorts all of them by age created. As a team, review tickets in the same manner as Triage / Sizing Sessions. For each ticket, answer the following questions:

Is the request in the ticket still relevant? Is it outdated or a duplicate of a ticket? If a duplicate, decided as a team which of the two tickets to rely on as the primary ticket (could be this older one or the newer one, depending on how thorough and relevant the content is).
Is the request still needed? If the ticket is to address an issue that popped up once two years ago, this request should be treated as stale and opened as a new ticket if it resurfaces.
Is the ticket's original submitter still asking for this request? Reach back out to them to confirm if their request has changed in the time that the ticket has been sitting in the backlog or if there is a more urgent need that they would rather have addressed.
Re-size the ticket as a group to update the value and confirm that the team as a whole have a consistent understanding of the original request.
Re-prioritize the ticket if appropriate to take on in the near future.

The vibe of this meeting is chill and nostalgic. If the team has been around long enough, this may be a fun trip down memory lane to 3-year old tickets.

Backlog Grooming: Why

It's easy if left unchecked for a software development ticket backlog to build up. There's may nice-to-haves for improving processes but only a limited window for what tasks can actually be prioritized. This meeting enables a balance to be had: all team members should feel comfortable documenting tasks they would like to have accomplished for the betterment of the team and the company, but the team does not feel overwhelmed by having a giant never-ending backlog of tasks that may not be prioritized.

Sprint Retro

Sprint Retro: When

Sprint Retrospective and Sprint Planning go hand-in-hand. Every 2 weeks, schedule a 1-hour meeting with the first half as the sprint retro and the second half as the sprint planning (see Sprint Planning: When).

Sprint Retro: How

Prior to the meeting, set up a team document with the following 4 headings:

🎖️ Kudos - specific thanks to team members who stood out with their efforts over the past sprint
🟢 What went well?
🔴 What went wrong?
💡 Action Items

Set up a 5 minute timer for the group to go through and write answers to the first 3 questions, leaving "Action Items" untouched for now. An alternative as well is for the team lead to go through each question in order and ask for members to speak up, where the manager then transcribes the answers. I am still on the fence about either approach. I would recommend the "Kudos" answers be said out loud; it's more meaningful personally when I was a junior developer to hear a senior developer say that my efforts were appreciated.

After the timer - or if following the manager-lead approach - go through the answers to each column and encourage any follow-up discussions. The answers to questions 2 and 3 should not be specific towards any individual; we are a team, so discussions should ensure ways that the team can better work together to address any shortcomings. If frustrations are pointed towards a different team in the company, what are ways we can improve our own processes to better assist them? (i.e. developing specific procedures to assist with investigation issues on our end before handing off externally for others to troubleshoot).

While going through the discussion for questions 2 and 3, any follow-ups are now written down in the "Action Items" column. After the meeting, the individual Action Items should be delegated to team members to re-made as proper software development tickets. Some retrospectives may not result in Action Items if the result of the discussion was either that there was no glaring process improvement that needs to be made or an issue that was encountered has a low probability of resurfacing.

Sprint Retro: Why

Software is written and maintained by people, so encouraging self-reflection is a must to resolve both software development issues and people issues. Team members should not feel that the issues with the team are stuck in their ways. Everyone should feel comfortable also clearly expressing frustrations in a blameless manner. Individualize the gains and socialize the losses. Clear, open, and honest communication.

Sprint Planning

Sprint Planning: When

Sprint planning takes place right after Sprint retrospective or in the same day. Schedule a 30 minute to 1 hour block of time right after or later in the day, depending on how much preparation is needed before the meeting.

Sprint Planning: How

Look over the past several sprints to see a trend of the number of story points are being closed out. Use this metric ("velocity") as a guide for how many story points to add onto the workload. If there are outstanding tickets not completed from the prior sprint, take them into account when setting up new work to take on. Communicate with stakeholders about potential delays in timing if those outstanding tickets will impact the original estimated timeline for the initiative.

As a team lead, look over the delegated story points just to get a sense for each member's workload and not as a comparison. If someone has a noticeable carryover from sprint-to-sprint not being accomplished, see if some of their tickets can be delegated to others (or in your regular 1-on-1, bring up with them to invite the discussion).

Ideally, for a given team of people, one member can be assigned to an "Operations" ticket queue for one-off requests, short repeated manual procedures, and monitoring application health and logs. The remaining members of the team would be assigned to larger project initiatives with clear estimated deadlines split out.

Sprint Planning: Why

Once a team is a well-oiled machine, Sprint Planning becomes a manner of communicating a timeline for completing a project with stakeholders with proper scheduling well in advance. A new project request comes in from a key stakeholder; how does this request compare to other priorities in the business? After breaking the project down to its individual steps in a sizing session, the cumulative story points compared to the team's velocity will inform us how many sprints to dedicate to accomplishing the work.

The end result is a clear queue of work and estimates. The team is happy because deadlines are not a matter of setting everything to as-soon-as-possible and should be comfortable to accomplish and push for (avoiding crunch, where possible and where the company does not enable crunch-conditions). Stakeholders are happy because they have a clear estimate for when they will be unblocked or can expect work to be delivered without it being lost to a void of uncertainty.

Visualizing local git repository statuses with MermaidJS and a Python CLI

Kathryn DiPippo — Sat, 05 Jul 2025 04:26:05 +0000

Background
How to Use
Example Outputs
Python Script

Background

If you're like me, you have a folder with all of your git repositories held in the same location. In a more organized vision of myself, I switch between repos and work on tasks one-at-a-time. I start work in one git repository, get it finished, push it into a PR, and then I switch to another repo. After I release the PR, I go back into my local folder and switch back to main and pull in latest changes.

Instead though, I have often ended up with a ton of outstanding work in all of my different folders. I will start work only to be sidelined by more important issues and fires. By the time I return, enough time has passed that I've lost track of what the outstanding work needs to be continued. I'll make a separate folder "TEMP" and git clone a fresh version of the project, work in there, accidentally abandon that, keep making separate folders... until finally, my git repository folders are a mess of duplicates and work-in-progresses.

In the past to remedy this, I have manually typed out the folder structure and documented what branch I was looking at and the state of outstanding work and slowly whittle the folders back down to my ideal state. But what if I made a script that automatically assembled a Mermaid diagram instead?

How to Use

The provided Python script (compatible with both MacOS/Linux and Windows operating systems) at the bottom of this article contains a single command powered by the click library.

python script.py --help

Usage: script.py [OPTIONS]

  CLI command to return a mermaid diagram of all git repositories - including
  in subdirectories in a specified local folder along with the branch they're
  currently pointed to and their number of uncommitted changes.

Options:
  --root-dir TEXT  Relative or absolute path to directory to traverse.
                   Defaults to treating current dir where the script is being
                   run as the root.
  --only-errors    If True, hide any git repositories that are pointed to the
                   base branch and have no outstanding changes.
  --output TEXT    If provided, save Mermaid string to file with filename.
  --name TEXT      If provided, filter results to the provided git repository
                   name.
  --help           Show this message and exit.

For example, this command will take the directory located at ../.. from where the script is being called, walk through all of its directories and subdirectories, and generate a Mermaid file mermaid.mmd that shows the complete tree structure and the status of all of the git repositories found.

python script.py --root-dir ../.. --output mermaid.mmd

The additional option --only-errors will hide any git repositories already at a "blank slate" status. This will enable me to know what folders I have yet to address.

python script.py --root-dir ../.. --output mermaid.mmd --only-errors

Also, the additional option --name will filter the git repositories to the name specified. This is useful for cases where the same git repository is cloned in multiple places and to generate diagram with an easy comparison.

Example Outputs

This is an example diagram returned from python script.py --root-dir ../.. --output mermaid.mmd:

flowchart LR
classDef HasChangesPointedToBase fill:#f0ad4e
classDef HasChangesNotPointedToBase fill:#d9534f
classDef NoChangesNotPointedToBase fill:#5bc0de

path["path"] --> path_to
path_to["to"] --> path_to_git-repo-1
path_to["to"] --> path_to_git-repo-2
path_to["to"] --> path_to_TEMP
path_to_TEMP["TEMP"] --> path_to_TEMP_git-repo-1
path_to["to"] --> path_to_TEMP2
path_to_TEMP2["TEMP2"] --> path_to_TEMP2_git-repo-2

path_to_git-repo-1["<b>git-repo-1</b><br>FEATURE-1<br>All changes committed"]:::NoChangesNotPointedToBase
path_to_git-repo-2["<b>git-repo-2</b><br>main<br>9 files changed, 70 insertions(+), 25 deletions(-)"]:::HasChangesPointedToBase
path_to_TEMP_git-repo-1["<b>git-repo-1</b><br>FEATURE-1<br>20 files changed, 150 insertions(+), 125 deletions(-)"]:::HasChangesNotPointedToBase
path_to_TEMP2_git-repo-2["<b>git-repo-2</b><br>main<br>All changes committed"]

This is an example diagram returned from python script.py --root-dir ../.. --output mermaid.mmd --only-errors:

flowchart LR
classDef HasChangesPointedToBase fill:#f0ad4e
classDef HasChangesNotPointedToBase fill:#d9534f
classDef NoChangesNotPointedToBase fill:#5bc0de

path["path"] --> path_to
path_to["to"] --> path_to_git-repo-1
path_to["to"] --> path_to_git-repo-2
path_to["to"] --> path_to_TEMP
path_to_TEMP["TEMP"] --> path_to_TEMP_git-repo-1

path_to_git-repo-1["<b>git-repo-1</b><br>FEATURE-1<br>All changes committed"]:::NoChangesNotPointedToBase
path_to_git-repo-2["<b>git-repo-2</b><br>main<br>9 files changed, 70 insertions(+), 25 deletions(-)"]:::HasChangesPointedToBase
path_to_TEMP_git-repo-1["<b>git-repo-1</b><br>FEATURE-1<br>20 files changed, 150 insertions(+), 125 deletions(-)"]:::HasChangesNotPointedToBase

Python Script

To run this script, the following Python modules need to be installed beforehand (either in a venv or globally):

click==8.1.7
Jinja2==3.1.3

"""CLI command to return a mermaid diagram of all git repositories - including in subdirectories in a specified local
folder along with the branch they're currently pointed to and their number of uncommitted changes."""

import os
import platform
import shutil
import stat
import uuid
from typing import Callable, Optional

import click
import jinja2

MERMAID_FULL_TEMPLATE = """
flowchart LR
classDef HasChangesPointedToBase fill:#f0ad4e
classDef HasChangesNotPointedToBase fill:#d9534f
classDef NoChangesNotPointedToBase fill:#5bc0de
{% for mermaid_route in mermaid_routes %}
{{ mermaid_route }}
{% endfor %}
{% for mermaid_element in mermaid_elements %}
{{ mermaid_element }}
{% endfor %}
"""

MERMAID_ELEMENT_TEMPLATE = """
{{ element_id }}
["
<b>{{ git_repo_name }}</b><br>
{{ git_branch }}<br>
{% if git_outstanding_changes == "" %}
All changes committed
{% else %}
{{ git_outstanding_changes }}
{% endif %}
"]
{% if has_changes and is_pointed_to_base_branch %}
:::HasChangesPointedToBase
{% elif has_changes and not is_pointed_to_base_branch %}
:::HasChangesNotPointedToBase
{% elif not has_changes and not is_pointed_to_base_branch %}
:::NoChangesNotPointedToBase
{% endif %}
"""


def _handle_readonly(function: Callable[[str], object], path: str, _) -> None:
    """On Windows, .idx files in the .git/objects/pack folder get set to readonly and throw PermissionError.
    They need to be converted to write before deletion.
    https://stackoverflow.com/questions/21778356/python-shutil-rmtree-cannot-remove-git-dir-on-win7

    Args:
        function (Callable[..., Any]): Function which raised the exception, depends on the platform and implementation.
        path (str): Path name passed to Function.
        excinfo (BaseException): Exception that was raised. Exceptions raised by onexc will not be caught.
    """
    os.chmod(path, stat.S_IWRITE)
    function(path)


def delete_file_or_folder(file_or_folder_name: str) -> None:
    """Delete file or folder at relative path. Pythonic version of "rm -rf".

    Args:
        file_or_folder_name (str): Relative path to file or folder to recursively delete.

    Raises:
        LookupError: Raised if input fails matching on isfile() or isdir().
    """
    # Silence exception if path does not exit.
    if not os.path.exists(file_or_folder_name):
        return

    if os.path.isfile(file_or_folder_name):
        os.remove(file_or_folder_name)
        return

    if os.path.isdir(file_or_folder_name):
        if platform.system() == "Windows":
            # Try chmod-ing files if error occurs while running on Windows.
            shutil.rmtree(file_or_folder_name, onexc=_handle_readonly)
            return
        shutil.rmtree(file_or_folder_name)
        return

    raise LookupError(f"delete_file_or_folder() input is neither file nor folder: '{file_or_folder_name}'")


class GitDirectory:
    """Class to represent a directory containing a git repository."""

    def __init__(self, git_dir_path: str, base_branch_name: str = "main") -> None:
        """Initialize GitDirectory class.

        Args:
            git_dir_path (str): git directory path, i.e. "/path/to/git-repo".
        """
        self.git_dir_path = git_dir_path
        self.git_dir_path_parts = [entry for entry in git_dir_path.split("/") if entry]
        self.git_repo_name = self.git_dir_path_parts[-1]

        self.git_branch = self.get_git_branch()
        self.base_branch_name = base_branch_name

        self.git_outstanding_changes = self.get_git_outstanding_changes()

    def _get_output_of_command(self, command: str) -> str:
        """Execute provided command and save terminal outputs to self's class attributes.

        Raises:
            RuntimeError: Thrown if command returned an error.

        Args:
            command (str): Terminal command to execute.
        """
        # This method works by running the command with "> filename.txt" argument to save outputs to file,
        # then reads that file for the output and deletes it before exiting.
        temp_output_filename = f"{str(uuid.uuid4())}-command-output-tempfile.txt"
        execute = os.system(f"{command} &> {temp_output_filename}")
        with open(temp_output_filename, "r", encoding="utf-8") as temp_output_file:
            temp_output_content = temp_output_file.read()

        delete_file_or_folder(temp_output_filename)
        if execute == 0:
            return temp_output_content
        raise RuntimeError(temp_output_content)

    def _get_git_command_output(self, git_command: str) -> str:
        """Return output of provided git command in another directory.

        Args:
            git_command (str): Git command, i.e. "git example-command".

        Returns:
            str: Output of provided git command, after it is changed to "git -C /path/to/git-repo example-command".
        """
        git_command = git_command.replace("git ", f"git -C {self.git_dir_path} ")
        return self._get_output_of_command(git_command).strip()

    def get_git_branch(self) -> str:
        """Return current branch in git repository.

        Returns:
            str: Current branch in git repository, i.e. "FEATURE-1".
        """
        return self._get_git_command_output("git branch --show-current")

    def get_git_outstanding_changes(self) -> str:
        """Get git outstanding changes status message.

        Returns:
            str: git outstanding changes status message, i.e. "9 files changed, 70 insertions(+), 25 deletions(-)".
                Will return "" if there are no uncommitted changes.
        """
        git_diff_stat = self._get_git_command_output("git diff --stat")
        return git_diff_stat.split("\n")[-1].strip()

    def has_changes(self) -> bool:
        """Return True if git repository has outstanding, uncommited changes.

        Returns:
            bool: True if git repository has outstanding, uncommited changes.
        """
        return self.git_outstanding_changes != ""

    def is_pointed_to_base_branch(self) -> bool:
        """Return True if git repository is pointed to the base branch.

        Returns:
            bool: True if git repository is pointed to the base branch.
        """
        return self.git_branch == self.base_branch_name

    def get_mermaid_element(self) -> str:
        """Get Mermaid representation of GitRepository object.

        Returns:
            str: Mermaid representation of GitRepository object, i.e.
                'path_to_git-repo["<b>git-repo</b><br>FEATURE-1<br>All changes committed"]:::NoChangesNotPointedToBase'
        """
        environment = jinja2.Environment()
        template = environment.from_string(MERMAID_ELEMENT_TEMPLATE)
        return (
            template.render(
                element_id="_".join(self.git_dir_path_parts).replace("/", "_"),
                git_repo_name=self.git_repo_name,
                git_branch=self.git_branch,
                git_outstanding_changes=self.git_outstanding_changes,
                has_changes=self.has_changes(),
                is_pointed_to_base_branch=self.is_pointed_to_base_branch(),
            )
            .replace("\n", "")
            .strip()
        )

    def get_mermaid_routes(self) -> list[str]:
        """Get all Mermaid element routes, pointing the connections between directories and their subdirectories
        inferred from the path to this git repository.

        Returns:
            list[str]: List of all mermaid element routes inferred from the path to this current git repository, i.e.
                ['path["path"] --> path_to', 'path_to["to"] --> path_to_git-repo']
        """
        mermaid_routes: list[str] = []
        for index in range(1, len(self.git_dir_path_parts)):
            element_one_id = "_".join(self.git_dir_path_parts[:index])
            element_one_name = self.git_dir_path_parts[index - 1]
            element_two_id = "_".join(self.git_dir_path_parts[: index + 1])
            mermaid_routes.append(f'{element_one_id}["{element_one_name}"] --> {element_two_id}')
        return mermaid_routes


def get_all_git_dir_paths(root_dir: str = ".") -> list[str]:
    """Get all git directories (including subdirectores) from root where script is being called.

    Returns:
        list[str]: git directory paths, i.e. ["/path/to/git-repo"].
    """
    git_dir_paths = []
    for root, dirs, files in os.walk(os.path.abspath(root_dir)):  # pylint: disable=unused-variable
        for dir_name in dirs:
            dir_path = os.path.join(root, dir_name)
            if not os.path.isdir(f"{dir_path}/.git"):
                continue
            git_dir_paths.append(dir_path)
    return git_dir_paths


@click.command(
    help="""CLI command to return a mermaid diagram of all git repositories - including in subdirectories in a specified
local folder along with the branch they're currently pointed to and their number of uncommitted changes."""
)
@click.option(
    "--root-dir",
    required=False,
    type=str,
    default=".",
    help="""Relative or absolute path to directory to traverse. Defaults to treating current dir where the script is
being run as the root.""",
)
@click.option(
    "--only-errors",
    "show_only_errors",
    is_flag=True,
    required=False,
    type=bool,
    default=False,
    help="If True, hide any git repositories that are pointed to the base branch and have no outstanding changes.",
)
@click.option(
    "--output",
    "output_filename",
    required=False,
    type=str,
    help="If provided, save Mermaid string to file with filename.",
)
@click.option(
    "--name",
    "git_repo_name",
    required=False,
    type=str,
    help="If provided, filter results to the provided git repository name.",
)
def main(
    root_dir: str = ".",
    show_only_errors: bool = False,
    output_filename: Optional[str] = None,
    git_repo_name: Optional[str] = None,
) -> None:
    """CLI command to return a mermaid diagram of all git repositories - including in subdirectories in a specified
    local folder along with the branch they're currently pointed to and their number of uncommitted changes.

    Args:
        root_dir (str, optional): Relative or absolute path to directory to traverse. Defaults to "." to current dir.
        show_only_errors (bool, optional): If True, hide any git repositories that are pointed to the base branch and
            have no outstanding changes.
        output_filename (Optional[str], optional): If provided, save Mermaid string to file with filename.
        git_repo_name (Optional[str], optional): If provided, filter results to the provided git repository name.
    """
    git_directories: list[GitDirectory] = [
        GitDirectory(git_dir_path) for git_dir_path in get_all_git_dir_paths(root_dir)
    ]

    if git_repo_name is not None:
        git_directories = [
            git_directory for git_directory in git_directories if git_directory.git_repo_name == git_repo_name
        ]

    mermaid_elements: list[str] = []
    mermaid_routes: list[str] = []
    for git_directory in git_directories:
        # If --only-errors, skip git repositories that are pointed to the base branch without outstanding changes.
        if show_only_errors and git_directory.is_pointed_to_base_branch() and not git_directory.has_changes():
            continue

        # If --name provided, skip git repositores that do not have the same git repository name provided.
        if git_repo_name is not None and git_directory.git_repo_name != git_repo_name:
            continue

        mermaid_elements.append(git_directory.get_mermaid_element())
        mermaid_routes += git_directory.get_mermaid_routes()
    mermaid_routes = list(set(mermaid_routes))

    environment = jinja2.Environment()
    template = environment.from_string(MERMAID_FULL_TEMPLATE)
    mermaid_full_str = template.render(
        mermaid_elements=mermaid_elements,
        mermaid_routes=mermaid_routes,
    )

    if output_filename is None:
        print(mermaid_full_str)
        return

    with open(output_filename, "w", encoding="utf-8") as output_file:
        output_file.write(mermaid_full_str)


if __name__ == "__main__":
    main()

How and Why to Use Patches

Kathryn DiPippo — Tue, 24 Jun 2025 22:35:03 +0000

Background
What is a Patch?
Why and When to Patch?
How to Create and Apply a Patch
- Create wrapper install command
- Initialize blank-slate state for third-party dependencies
- Make changes until third-party dependency is functional
- Create and assess initial patchfile result
- Refine patchfile content
- Create post-install patching step
List of Patching Commands
- Command for Creating a Patch
- Command for Applying a Patch

Background

At one of my previous jobs, one of our major headaches was installing and maintaining a single sign-on authentication PHP module called simpleSAMLphp. SSO is a positive on many fronts: end-users don't have to save a password for a new software application and can trust the fine folks at Google or Microsoft for keeping their accounts secure and setting up 2-factor-authentication, and we as developers don't have to dedicate resources towards maintaining and enforcing secure login practices and software architecture.

But the way that we set up simpleSAMLphp was more complicated than it needed to be. The major woe we ran into were how these config files were first maintained. The block of simpleSAMLphp code was copied into our codebase and then directly modified in order to work with the rest of the codebase. This was perfectly fine on the first installation, but then the upgrades would come later. What if we had modified the configuration over time? How do we know what changes need to be applied to the module's newer version after it's put in place? Oftentimes, configuration changes that were made in a previous simpleSAMLphp version were missed on upgrade, and the team would scramble trying to find out what steps had been missed in the midst of a login outage.

It's this kind of scenario where patches come into play. We could have minimized these headaches had we followed the process of:

Documenting the basic installation process of the simpleSAMLphp codebase into our repo
Maintained patchfiles that would apply all the custom changes we need to make to get simpleSAMLphp working with our existing code
Upgrades mean changing the version of simpleSAMLphp installed in the first bullet followed by recreating our patchfile content against the newer version where possible

What is a Patch?

A patch is a dedicated file that dictates how a code change should be applied to an existing file. In pratice, the content of a patchfile looks like a diff (because it's generated using the diff Linux command). Here's an example that modifies the content of cli.js in the doiuse-email NPM package to start with the line #!/usr/bin/env node.

--- node_modules/doiuse-email/bin/cli.js 2024-05-24 15:26:26
+++ cli.js 2024-05-24 15:26:02
@@ -1,3 +1,4 @@
+#!/usr/bin/env node
 import { program } from 'commander';
 import * as fs from 'node:fs';
 import process from 'node:process';

Node projects install third-party dependencies via the npm install command. On my end, I need to either update how package.json is configured (which informs how npm install should run) or I need to make a bundled command such that:

My dependencies are installed
Then my patches are applied.

Why and When to Patch?

Patches are best used in scenarios where we need to keep what changes we're making very clearly defined from what the base codebase of a third-party dependency is. In the Background section, it would have been beneficial for my team to keep our team's configuration changes separate from the defaults provided to us by the simpleSAMLphp codebase being imported.

Patches are also useful if we're making manual modifications to any codebase that is being auto-generated, i.e. if we're using openapi-generator off of an OpenAPI spec to create clients and scaffolded server codebases. We would want to maintain the patch as a way to ensure that subsequent re-generations of the code don't wipe out any manual changes we have to make in-between.

Always think about the future: if in 2 years I need to make an update to some part of our codebase that creates / updates a large swath of code, how likely will I remember this tiny manual change I made in one file?

Additionally, in cases where the third-party dependency being imported is open-source, consider whether the change that needs to be made would be beneficial to others using the codebase. If the change is specific to us, then leaving it as a patch is best. If the change is actually resolving a bug that other people using the third-party dependency are encountering, then consider being a good open-source software samaritan and start a PR with the contents of the patch applied. For efficiency:

Create the patchfile with the bugfix on your team's repository
Create the PR on the open-source repository and work through their validations and reviews
Once the fix is released and the new version of the open-source dependency is tagged, update your team's repository with the newer version and remove the patchfile

Here's a helpful flowchart to navigate whether to create a patch and whether that patch should propagate upstream into its open-source project:

Note: above diagram was made using Mermaid and its flowchart syntax with the following snippet:

flowchart TD
    one[Import <i>Dependency</i> into our codebase]
    two{Does <i>Dependency</i> work with our codebase as-is?}
    three[No Patches needed]
    four[Create Patches of changes needed for <i>Dependency</i>]
    five{Is <i>Dependency</i> open-source & would our changes be beneficial to others?}
    six[Create a PR against <i>Dependency</i>'s repository.]

    one --> two
    two -->|No| four
    two -->|Yes| three
    four --> five
    five -->|Yes| six

How to Create and Apply a Patch

Create wrapper install command

To get started, assume that there's a third-party dependency that is imported and whose contents you can and need to manipulate. For this section, I will walkthrough a scenario where that dependency is an NPM package. But this could apply to other language's package managers (like PHP's packagist) or where a giant .zip of the codebase needs to be unzipped and hooked up (as was the case with simpleSAMLphp). In this scenario, node_modules/ will also be .gitignored and does not need to stay in the codebase for the patch to be applied.

Start by wrapping how all third-party dependencies are installed in one singular, bundled command (i.e. create a Makefile command):

.PHONY: install
install:          ## Install all third-party dependencies.
    rm -rf node_modules
    npm install

Note: your Makefile will need to update the indent in this codeblock from 4-spaces to 1-tab.

Initialize blank-slate state for third-party dependencies

Run make install to start with what the "default" state will be for your dependencies. At this point, your third-party dependencies are installed but not properly working with your codebase. I will refer to the NPM package that needs to be patched as third-party/.

Make changes until third-party dependency is functional

Work inside the node_modules/third-party/ folder in coordination with manual testing and automated unit-testing until the codebase behaves as we need it to. Don't worry about being as reckless as possible or having to keep track of which files are being modified. If the code is too destroyed, the make install command can be re-run to return to the blank-slate state.

In this example, node_modules/third-party/ was finally functional after a number of changes were applied to the following 3 files:

bin/cli.js
src/main.js
src/config.json

Create and assess initial patchfile result

The diff Linux command is used to create the actual content of the patchfiles. In order to figure out what changes were actually performed, we'll need to diff the entire folder we changed against its blank-slate state.

Copy the changed folder to a different location:

mkdir third-party-fixed
cp -r node_modules/third-party .

(Optional) Rename the copied folder to clearly distinguish it in later steps:

mv third-party third-party-fixed

Re-install our dependencies back to blank-slate:

make install

Create and save a diff of both folders:

-r is to run the diff command recursively on both folders (or else result.patch just lists the similar folders 1-level deep in each folder).
-u is to specify that the diff content returned is in the Unified diff format. Without a number passed in with -u, the output diff will also show 3 (default) lines of matching content before-and-after each altered line.

diff -r -u node_modules/third-party third-party-fixed > result.patch

The content of result.patch will show the full updates that were made in the Make changes until third-party dependency is functional section. Here is an example of this full patch with same list of files changed from earlier:

result.patch without refinement:

diff --color -r -u node_modules/third-party/bin/cli.js third-party-fixed/bin/cli.js
--- node_modules/third-party/bin/cli.js 2025-06-24 17:32:22
+++ third-party-fixed/bin/cli.js 2025-06-24 17:36:23
@@ -9,6 +9,7 @@
     .option('--supported-features', 'output the supported email features for the email clients specified')
     .argument('[file]', 'the path to the HTML file to check')
     .action((file, options) => {
+    throw new Error("TEST - stopping here to check how the prior methods work.");
     if (options.supportedFeatures !== undefined && file !== undefined) {
         throw new Error('Only one of `file` or `--supported-features` should be provided.');
     }
diff --color -r -u node_modules/third-party/src/config.json third-party-fixed/src/config.json
--- node_modules/third-party/src/config.json 2025-06-24 17:33:00
+++ third-party-fixed/src/config.json 2025-06-24 17:35:23
@@ -9,7 +9,7 @@
     "_sessionKey": "MyReallySecretPassword1",
     "_port": 443,
     "_aliasPort": 443,
-    "_redirPort": 80,
+    "_redirPort": 121,
     "_redirAliasPort": 80
   },
   "domains": {
@@ -17,7 +17,7 @@
       "_title": "MyServer",
       "_title2": "Servername",
       "_minify": true,
-      "_newAccounts": true,
+      "_newAccounts": false,
       "_userNameIsEmail": true
     }
   },
diff --color -r -u node_modules/third-party/src/main.js third-party-fixed/src/main.js
--- node_modules/third-party/src/main.js 2025-06-24 17:33:40
+++ third-party-fixed/src/main.js 2025-06-24 17:35:02
@@ -91,6 +91,7 @@

 // Convert a string into a blob
 module.exports.data2blob = function (data) {
+    console.log("TEST - AT START OF module.exports.data2blob");
     var bytes = new Array(data.length);
     for (var i = 0; i < data.length; i++) bytes[i] = data.charCodeAt(i);
     var blob = new Blob([new Uint8Array(bytes)]);
@@ -170,9 +171,13 @@
     return doc;
 };
 module.exports.unEscapeLinksFieldName = function (doc) {
+    console.log("TEST - 0 - module.exports.unEscapeLinksFieldName");
     if (doc.links != null) { for (var j in doc.links) { var ue = module.exports.unEscapeFieldName(j); if (ue !== j) { doc.links[ue] = doc.links[j]; delete doc.links[j]; } } }
+    console.log("TEST - 1 - module.exports.unEscapeLinksFieldName");
     if (doc.ssh != null) { for (var j in doc.ssh) { var ue = module.exports.unEscapeFieldName(j); if (ue !== j) { doc.ssh[ue] = doc.ssh[j]; delete doc.ssh[j]; } } }
+    console.log("TEST - 2 - module.exports.unEscapeLinksFieldName");
     if (doc.rdp != null) { for (var j in doc.rdp) { var ue = module.exports.unEscapeFieldName(j); if (ue !== j) { doc.rdp[ue] = doc.rdp[j]; delete doc.rdp[j]; } } }
+    console.log("TEST - 3 - module.exports.unEscapeLinksFieldName");
     return doc;
 };
 //module.exports.escapeAllLinksFieldName = function (docs) { for (var i in docs) { module.exports.escapeLinksFieldName(docs[i]); } return docs; };
@@ -194,13 +199,13 @@
 module.exports.validateUsername = function (username, minlen, maxlen) { return (module.exports.validateString(username, minlen, maxlen) && (username.indexOf(' ') == -1) && (username.indexOf('"') == -1) && (username.indexOf(',') == -1)); };
 module.exports.isAlphaNumeric = function (str) { return (str.match(/^[A-Za-z0-9]+$/) != null); };
 module.exports.validateAlphaNumericArray = function (array, minlen, maxlen) { if (((array != null) && Array.isArray(array)) == false) return false; for (var i in array) { if ((typeof array[i] != 'string') || (module.exports.isAlphaNumeric(array[i]) == false) || ((minlen != null) && (array[i].length < minlen)) || ((maxlen != null) && (array[i].length > maxlen)) ) return false; } return true; };
-module.exports.getEmailDomain = function(email) {
-    if (!module.exports.validateEmail(email, 1, 1024)) {
-        return '';
-    }
-    const i = email.indexOf('@');
-    return email.substring(i + 1).toLowerCase();
-}
+// module.exports.getEmailDomain = function(email) {
+//     if (!module.exports.validateEmail(email, 1, 1024)) {
+//         return '';
+//     }
+//     const i = email.indexOf('@');
+//     return email.substring(i + 1).toLowerCase();
+// }

 module.exports.validateEmailDomain = function(email, allowedDomains) {
     // Check if this request is for an allows email domain

For this example, I can see the 3 files that were changed earlier and what was done to them:

In bin/cli.js, I put a temporary throw new Error() for debugging purposes
In src/main.js, I added a handful of console.log()s for debugging purposes and commented out the getEmailDomain() function.
In src/config.json, I changed the values for two keys in that config _redirPort and _newAccounts.

If instead you know which specific files were affected and only want to get the patchfile content for a single file, rerun the command without the -r and with each argument against a single file:

diff -u node_modules/third-party/src/config.json third-party-fixed/src/config.json > file.patch

Refine patchfile content

It's essential that the content of the patch itself is minimal and essential in order to make it much easier to re-apply to updated versions of the dependency in the future. If a patch in the future needs to be created that overlaps with an existing patch, those two patches should be combined to form one new patch to avoid a complicated layering of changes and potential conflicts.

In the example, after reviewing the giant patchfile created, I've determined that only the changes in src/config.json are needed. The immediate benefits of the patchfile are now clear: the two config changes we need to maintain are small and easy-to-miss. It's easy to see how future upgrades may neglect to update these two values and re-encounter the same issues just worked through:

diff --color -r -u node_modules/third-party/src/config.json third-party-fixed/src/config.json
--- node_modules/third-party/src/config.json 2025-06-24 17:33:00
+++ third-party-fixed/src/config.json 2025-06-24 17:35:23
@@ -9,7 +9,7 @@
     "_sessionKey": "MyReallySecretPassword1",
     "_port": 443,
     "_aliasPort": 443,
-    "_redirPort": 80,
+    "_redirPort": 121,
     "_redirAliasPort": 80
   },
   "domains": {
@@ -17,7 +17,7 @@
       "_title": "MyServer",
       "_title2": "Servername",
       "_minify": true,
-      "_newAccounts": true,
+      "_newAccounts": false,
       "_userNameIsEmail": true
     }
   },

Whether you use one patchfile to apply changes across a number of files or separate patchfiles for each file changed is up-to-you. I personally have only had to write patches for changes against one file. I also would personally lean towards one-patchfile-per-file as a way to better understand the scope of the changes and to more easily work through upgrades in cases where one of the files is removed or dramatically altered with that upgrade.

Create post-install patching step

Your repository should contain a new folder patches/ that contains all of the patchfiles need to be applied after installing third-party dependencies. The organization of this folder entirely depends on how complex of a patching setup is needed.

If there are a lot of third-party dependencies that each need a lot of patches, then consider setting up a more complex folder structure to more easily navigate and understand where changes are being applied. But for our example and for most use cases, storing all patchfiles flat in the patches/ folder is fine.

I recommend the following naming convention by joining the following identifiers:

Ticket ID where the patch needs to be created (i.e. the Ticket ID to install and configure the third-party dependency)
Name of the third-party dependency
Version of that third-party dependency where this patch is applied
File path to where the change is applied with / forward-slashes replaced with _ underscores

For our previous example, result.patch should be moved into patches/ with the name PROJ-123_third-party_1.0.0_src_config.json.patch. Also optionally, the content of this patch can be modified to remove the temporary path where the secondary file was stored. This is entirely for cosmetic purposes:

- diff --color -r -u node_modules/third-party/src/config.json third-party-fixed/src/config.json
+ diff --color -r -u node_modules/third-party/src/config.json config.json
  --- node_modules/third-party/src/config.json 2025-06-24 17:33:00
- +++ third-party-fixed/src/config.json 2025-06-24 17:35:23
+ +++ config.json 2025-06-24 17:35:23
  @@ -9,7 +9,7 @@
       "_sessionKey": "MyReallySecretPassword1",
       "_port": 443,
       "_aliasPort": 443,
  -    "_redirPort": 80,
  +    "_redirPort": 121,
       "_redirAliasPort": 80
     },
     "domains": {
  @@ -17,7 +17,7 @@
         "_title": "MyServer",
         "_title2": "Servername",
         "_minify": true,
  -      "_newAccounts": true,
  +      "_newAccounts": false,
         "_userNameIsEmail": true
       }
     }

I'm happy to have had the opportunity to make a diff on a diff here LOL.

Now, the wrapper install command from earlier needs to be modified to apply the patch:

.PHONY: install
install:          ## Install all third-party dependencies.
    rm -rf node_modules
    npm install
+    patch node_modules/third-party/src/config.json patches/PROJ-123_third-party_1.0.0_src_config.json.patch

Note: your Makefile will need to update the indent in this codeblock from 4-spaces to 1-tab.

Verify that the installation and patching is successfully by performing the following steps:

Run make install to reset the dependencies back to blank-slate state and then programmatically apply the patch.
Manually verify that the files specified in the patchfile contain those changes in your third-party dependency folder (i.e. check that changes that need to be applied to config.json appear inside node_modules/third-party/src/config.json).
Verify that your full codebase implementation now works without any other manual changes needed.
If the third-party dependency is still unsuccessful, repeat steps Make changes until third-party dependency is functional through to this current step.

List of Patching Commands

In short, these are the two key Unix terminal commands:

Command for Creating a Patch

diff command arguments explained:

first argument is the "before" file left un-touched from what is provided in the third-party dependency codebase
second argument is the "after" file with manual changes applied
-r is to run the diff command recursively on both folders (or else result.patch just lists the similar folders 1-level deep in each folder).
-u is to specify that the diff content returned is in the Unified diff format. Without a number passed in with -u, the output diff will also show 3 (default) lines of matching content before-and-after each altered line.

diff two files and save the result in one patchfile:

diff -u node_modules/third-party/src/config.json config-fixed.json > file.patch

diff two folders and save the result in one patchfile:

diff -r -u node_modules/third-party third-party-fixed > result.patch

Command for Applying a Patch

patch command arguments explained:

first argument is the original file left un-touched and located where the third-party dependency will be installed
second argument is the patchfile that specifies how the "first argument" file should be altered

patch example with a single file in node_modules/ with the patch stored in the patches/ folder:

patch node_modules/third-party/src/config.json patches/PROJ-123_third-party_1.0.0_src_config.json.patch

The Essential Software Development Process

Kathryn DiPippo — Tue, 24 Jun 2025 19:13:05 +0000

Background
Required Tooling
- Issue Tracker
- Git Platform
Getting Started
- Setting up the Issue Tracker
- Setting up the Git Repository
Walkthrough of the End-to-End Process
- Issue PROJ-123 is ready to be worked on
- Branch PROJ-123 is created on the git repository
- Assignee pushes to PROJ-123 branch
- Assignee creates Pull Request for PROJ-123 branch
- Team reviews Pull Request for PROJ-123 branch
- Assignee squashes and merges PROJ-123 branch to main branch
- Assignee creates tag for PROJ-123 release
- Feature released in PROJ-123 causes an issue in production
Recommended Enhancements to this Process
- Add as many programmatic checks as possible
- Automate programmatic checks and releases with CI/CD Tooling
Conclusion

Background

The first job I held right after graduating college was chaotic. Leadership was the key factor in all of its issues; the company had experienced a large turnover right before I was hired to fill in the gaps. But we lacked real tech leadership who could mentor and guide us on how to effectively develop in teams. I was very fortunate that my second job did offer that leadership and expertise for effectively solving problems.

Often, I've fantasized, "what would I do differently at my first job knowing what I know now?" And the #1 answer that always comes out on top: I wish we used git.

We had no version control at the time, and the problems we ran into were so obvious. There was no central codebase, so every member of our (albeit small) team was keeping copies of all of the code we had written. I would have dozens of ProductDisplayPageFINAL_FINAL_JUNE10_FINAL.js on my machine in case a teammate of mine's changes would save over mine. Only our CTO had access to the codebase and all of the backups of files pushed into the server. So much time was spent asking him to go in and revert changes. Coworkers would be berated for "not checking" when other people had altered the code, even though there were no formal internal notifications set up. All of our code was being pushed up with entirely manual testing, and it frequently encountered bugs or issues. There was no concept of team-based code reviews. We were all isolated to our own work without knowing what other people were building and pushing.

It was night-and-day just finding out how git and good process could solve all of these problems. I often tell people that if it weren't for having gone through the headaches and hurt of working in places such as my first job that the lessons I learned at my second job would not have stuck. I believe that my career to this point has all served a purpose in instructing me on proper software practices.

This is the essential software development process. I believe that any process that relies on code should start with this backbone and then be elaborated on over time in order to make processes more efficient.

Required Tooling

There are two key tools that are needed for this process.

Issue Tracker

New feature requests, update requests, and bug reports need to be collected, prioritized, and assigned. No matter what platform is selected, ensure that Issues being reported can be linked to using an identifier (ideally a short one).

Examples:

Jira by Atlassian. Jira Issue IDs are of the format PROJECTID-###, i.e. JIRA-123.
Asana. Asana Tasks will need to be configured with a Custom ID field, as ticket IDs via the API are all long UUIDs.
If using only GitHub:
- GitHub offers "Issues" on its repositories. Consider though if using this feature whether issues being reported could be affecting code across other repositories if the code is split out. This may mean setting up a central "issue reporting" repo. The ticket ID would the the Issue # (i.e. "1", but should be used in branch names later as "issue-1").
- GitHub also offers a "Project" management feature

Git Platform

A web platform should be used to make it easy to view code and navigate version history without solely relying on the git tool. For this article, I will be using GitHub as the key facilitator. But all features here should be available across all other platforms as well.

Examples:

GitHub
BitBucket by Atlassian
GitLab

Getting Started

Setting up the Issue Tracker

As mentioned, whatever issue tracker platform is used, all tickets need to have a short unique identifier for each task. This will be easier to use in Git and to verbally relay to others for what is being worked (saying "I'm working on the requested change to the email notifications to make the font size bigger" vs. saying "I'm working on PROJ-123").

All issues should contain the full documentation for what change needs to be performed, and all issues should be reviewed as a group by the development team. I recommend an "Acceptance Criteria" checklist at the bottom of every task that clearly defines what steps need to be accomplished before an issue is marked as done. This checklist should remove any ambiguity when code changes need to be performed.

Setting up the Git Repository

For this article, the git repository will be referred to as git-project-repo. This repo should have a default branch main whose codebase is treated as "stable".

Note: all of the git diagrams in this article were made using Mermaid and its gitGraph syntax. Dev.to does not support Mermaid markdown at this time, so all diagrams were saved as PNGs and are posted here with their accompanying Mermaid snippet.

gitGraph
    commit id:"initial commit"
    commit id:"PROJ-121 | Feature 1" tag:"1.0.0"
    commit id:"PROJ-122 | Feature 2" tag:"1.1.0" type:HIGHLIGHT

This repo has had 2 features created for it. The production implementation should also rely on the "tag"s for the commits and not just relying on what's in main. Production in this case is pointed to tag 1.1.0.

Additionally, the git repository should be set up with a Pull Request Template (if using GitHub, this should be a markdown file stored in .github/PULL_REQUEST_TEMPLATE.md; this will automatically populate all PR base comments with this file). This Template should contain the following sections:

Link to Issue: [PROJ-123](https://issue-tracking.platform.com/PROJ-123)

# Motivation

Why is this change needed?

# Changes Made

- High-level checklist of changes performed
- Could be the list of commits if change is small

# How to Verify

Procedure for either the Issue assignee to self-validate their work or for PR reviewers to review the work here, in addition to reviewing the code changes themselves.

# Release Steps

- [ ] Checklist of steps to perform for releases
- [ ] Could be a link to a full procedure document

Walkthrough of the End-to-End Process

Issue PROJ-123 is ready to be worked on

An Issue with short unique ID PROJ-123, as mentioned in the Setting up the Issue Tracker section, contains the complete request that needs to be actioned. This issue has been reviewed and agreed-upon by the whole developer team and could be theoretically picked up by any member to action. The team has now agreed that PROJ-123 is ready to start now.

Branch PROJ-123 is created on the git repository

The assignee of PROJ-123 creates a new branch on the git-project-repo repository. The branch name is the same as the Issue's ID.

gitGraph
    commit id:"initial commit"
    commit id:"PROJ-121 | Feature 1" tag:"1.0.0"
    commit id:"PROJ-122 | Feature 2" tag:"1.1.0" type:HIGHLIGHT
    branch "PROJ-123"
    checkout "PROJ-123"

Assignee pushes to PROJ-123 branch

The Issue itself should be broken down into the separate steps needed to finish the "Acceptance Criteria". For example, a development task may be broken down into:

Refactor a large function into smaller functions
Update one of the smaller functions to take in a new argument
Update unit-tests to pass
Call that updated function in a different part of the codebase
Add new unit-tests for new functionality

Each of these will be separate commits pushed by the developer into the Issue's branch. I recommend prefixing all commits with the Issue's ID as well, in case there are PRs that end up containing a handful of requests (i.e. if an Issue PROJ-234 is a small typo that needs to be fixed that would be more convenient to tackle at the same time as this request).

gitGraph
    commit id:"initial commit"
    commit id:"PROJ-121 | Feature 1" tag:"1.0.0"
    commit id:"PROJ-122 | Feature 2" tag:"1.1.0" type:HIGHLIGHT
    branch "PROJ-123"
    checkout "PROJ-123"
    commit id:"PROJ-123 | Refactor large func into smaller funcs"
    commit id:"PROJ-123 | Update smaller func with new arg"
    commit id:"PROJ-123 | Update unit-tests to pass"
    commit id:"PROJ-123 | Added call to updated function"
    commit id:"PROJ-123 | Add new unit-tests"
    commit id:"PROJ-234 | Fix typo in docs"

Assignee creates Pull Request for PROJ-123 branch

By now, all steps of the "Acceptance Criteria" are now addressed, and all local validations by the assignee are passing. The assignee is now ready to get the change reviewed by the rest of the team. The assignee now creates a Pull Request to request to merge changes from the PROJ-123 branch into the main branch.

For this example, the Pull Request's top comment will contain the following filled-in template:

Link to Issue: [PROJ-123](https://issue-tracking.platform.com/PROJ-123)
Also addresses: [PROJ-234](https://issue-tracking.platform.com/PROJ-234)

# Motivation

New feature request to change the size of the logo from 40 pixels tall to 50 pixels. This requires refactoring a part of the codebase in order to accept a new parameter. But this will let us make it easier to adjust this size in the future via a config update instead of a code change.

Also fixes a typo in the documentation reported in [PROJ-234](https://issue-tracking.platform.com/PROJ-234).

# Changes Made

- Refactor a large function into smaller functions
- Update one of the smaller functions to take in a new argument
- Update unit-tests to pass
- Call that updated function in a different part of the codebase
- Add new unit-tests for new functionality
- Fixed typo in documentation

# How to Verify

- [x] `make tests` runs all unit-tests locally
- [x] Go to https://internal-test.platform/feature-3 to see the changes in our test environment

# Release Steps

Current tag: `1.1.0`. New tag: `1.2.0`

- [ ] Send email to internal@company.com "Incoming release"
- [ ] Squash and merge PR into "main" branch
- [ ] Create new tag `1.2.0`
- [ ] Manually navigate into https://internal.platform/admin and change the tag to `1.2.0` and press "restart platform"
- [ ] Validate changes in Prod environment
- [ ] If rollback is required, follow the [Rollback Procedure](https://github.com/git-project-repo/README.md#rollback-procedure)

The content of this PR template is important because:

All code changes should be clearly tied to a "why we need this change" and a "how this change was done" for historical references
Reviewers are given a clear set of instructions for reviewing work
Assignees are given a clear set of instructions for releasing work

Any part of this process that is ambiguous will turn into delays (because people are afraid of pushing code or performing a change that will cause problems). Clearly defined procedures (that can be expanded on if there are problems that arise) will remove this worry and hesitation.

The original Issue Tracking Ticket should also be updated with a link to the new Pull Request and indicate that the work is now ready to review.

Team reviews Pull Request for PROJ-123 branch

I personally recommend going simple with this. I like to use the "labels" feature of GitHub and allow users to tag PRs with a "needs review" label that can be filtered against all repositories in a GitHub organization. The benefits of this strategy:

PRs can be started and refined until they're properly ready to review. For example, an assignee might ensure that the "diff" doesn't contain any accidental changes after making a proper PR.
Using the label adequately communicates the status of this work. PRs that have a lot of feedback that may warrant significant rework may be untagged with the "needs review" label before it is re-added again.
All members of the team can see all PRs that are tagged for review and can theoretically receive feedback from anyone.

If your team is large enough, another strategy is to randomly assign all PRs to dedicated reviewers. This also has the added benefit of ensuring that knowledge is being properly shared across the team and prevents a small handful of team members from becoming the sole reviewers. But I have not run into issues with the label-only strategy that couldn't be fixed by asking members of the team to slow down with code reviews to see if that forces others on the team to boost their contributions.

By the end of this step, any PR that is tagged for review should ideally have a minimum of 2 approvals if possible. Just having 1 approval is enough to get another pair of eyes on changes being pushed, but 2 will also allow for adequate knowledge transfer.

Assignee squashes and merges PROJ-123 branch to main branch

In the Pull Request template, the PR for PROJ-123 contains the following checklist:

# Release Steps

Current tag: `1.1.0`. New tag: `1.2.0`

- [ ] Send email to internal@company.com "Incoming release"
- [ ] Squash and merge PR into "main" branch
- [ ] Create new tag `1.2.0`
- [ ] Manually navigate into https://internal.platform/admin and change the tag to `1.2.0` and press "restart platform"
- [ ] Validate changes in Prod environment
- [ ] If rollback is required, follow the [Rollback Procedure](https://github.com/git-project-repo/README.md#rollback-procedure)

This is an example of what steps might be needed for a full push to production. Your team may want to expand on these steps where necessary (like if there are any database backup commands that need to be performed before releases). If, for example, a pre-production environment should be first used to validate the work in a semi-live environment, I recommend expanding that process to still rely on git's versioning:

# Release Steps

Current tag: `1.1.0`. New tag: `1.2.0`

 - [ ] Send email to internal@company.com "Incoming release"
 - [ ] Squash and merge PR into "main" branch
 - [ ] Create new tag `1.2.0`
+- [ ] Manually navigate into https://internal-preprod.platform/admin and change the tag to `1.2.0` and press "restart platform"
+- [ ] Validate changes in Pre-Prod environment
 - [ ] Manually navigate into https://internal.platform/admin and change the tag to `1.2.0` and press "restart platform"
 - [ ] Validate changes in Prod environment
 - [ ] If rollback is required, follow the [Rollback Procedure](https://github.com/git-project-repo/README.md#rollback-procedure)

Now the PR is ready to be squashed and merged into the main branch. From the PR, the commit history will look like this full diagram:

gitGraph
    commit id:"initial commit"
    commit id:"PROJ-121 | Feature 1" tag:"1.0.0"
    commit id:"PROJ-122 | Feature 2" tag:"1.1.0" type:HIGHLIGHT
    branch "PROJ-123"
    checkout "PROJ-123"
    commit id:"PROJ-123 | Refactor large func into smaller funcs"
    commit id:"PROJ-123 | Update smaller func with new arg"
    commit id:"PROJ-123 | Update unit-tests to pass"
    commit id:"PROJ-123 | Added call to updated function"
    commit id:"PROJ-123 | Add new unit-tests"
    commit id:"PROJ-234 | Fix typo in docs"
    checkout "main"
    merge "PROJ-123" id:"PROJ-123 | Feature 3"

But from main (and after the PROJ-123 branch is deleted now that its work is complete), the history will appear more simply as:

gitGraph
    commit id:"initial commit"
    commit id:"PROJ-121 | Feature 1" tag:"1.0.0"
    commit id:"PROJ-122 | Feature 2" tag:"1.1.0" type:HIGHLIGHT
    commit id:"PROJ-123 | Feature 3"

Deleting the branch does not mean the history is gone forever. In GitHub, any commit on main resulting from a PR will contain a link to the original PR in its title. Additionally, the git blame feature will enable us to easily check any line of code to find out any "why" and "how" the change was performed.

Assignee creates tag for PROJ-123 release

Looking at the git graph, note that a new version has not yet been tagged for the new commit. There should be no live environments who are using this code in their implementation. As noted in the example "Release Steps" checklist:

- [x] Squash and merge PR into "main" branch
- [ ] Create new tag `1.2.0`
- [ ] Manually navigate into https://internal.platform/admin and change the tag to `1.2.0` and press "restart platform"
- [ ] Validate changes in Prod environment

Tagging is essential in order to easily revert or release changes together without needing to work through code reversion PRs. With the "pre-production" example noted in the Assignee squashed and mergeds PROJ-123 branch to main branch section, tagging makes it easy to understand what environments are viewing which codebase at-a-glance.

Tags should follow the Semantic Versioning standard because it is easy-to-follow and well-documented.

The assignee now tags the new release with "1.2.0" because it is a new feature that is not backwards-incompatible with the existing codebase.

gitGraph
    commit id:"initial commit"
    commit id:"PROJ-121 | Feature 1" tag:"1.0.0"
    commit id:"PROJ-122 | Feature 2" tag:"1.1.0" type:HIGHLIGHT
    commit id:"PROJ-123 | Feature 3" tag:"1.2.0"

The assignee now navigates into the production environment and switches the tag for its code reference from "1.1.0" to "1.2.0".

gitGraph
    commit id:"initial commit"
    commit id:"PROJ-121 | Feature 1" tag:"1.0.0"
    commit id:"PROJ-122 | Feature 2" tag:"1.1.0"
    commit id:"PROJ-123 | Feature 3" tag:"1.2.0" type:HIGHLIGHT

By now, all the steps for releasing PROJ-123 are complete. The original Issue Tracking Ticket should also be updated with a comment on the release's completed status and any outcomes encountered.

Feature released in PROJ-123 causes an issue in production

Oh no! The release that was pushed ended up surfacing an incident in production! This issue may be something that is noticed immediately or results in a critical bug issue with ID PROJ-124.

Incidents like these should be planned as a matter of when and not if; code is written and maintained by humans. Having procedures in place in advance is crucial and necessary and will instill trust in the team moving forward.

There are two strategies that can be applied here: fail-forward or rollback.

Fail-forward means that the root cause of the issue was quickly located in the codebase. Maybe there's a small typo in a config name that wasn't caught with linting. Fail-forward means whoever is addressing this fire creates a PR with this changes and gets it reviewed and approved following all the steps in the Walkthrough of the End-to-End Process again.

Rollback means either that the root cause of the issue can't be located quickly enough to go through with failing-forward or that the fire being caused is too great to allow for time to investigate the root cause right away. Luckily, rollback is easy with our current configuration by switching the tag back to the last "stable" version of the codebase. There may be other steps as well for rolling back, like if an older version of a backend database needs to be restored.

gitGraph
    commit id:"initial commit"
    commit id:"PROJ-121 | Feature 1" tag:"1.0.0"
    commit id:"PROJ-122 | Feature 2" tag:"1.1.0" type:HIGHLIGHT
    commit id:"PROJ-123 | Feature 3" tag:"1.2.0"

Whatever issue is encountered here, we are luckily able to effectively figure out these problems because:

Version control makes it fast for us to switch between "safe" and "new" releases.
All releases will be small, modular changes with clearly outlined diffs. Releases should not be a signficant amount of alterations that make it difficult to pin-point the culprit.
Every part of this process has a documented procedure that can be altered or expanded to cover any gap that was missing where this original issue was introduced (like if an extra unit-test or check would have caught this culprit when the PR was in review).

Recommended Enhancements to this Process

The Walkthrough of the End-to-End Process now covers all of the basics for using git as part of any code change process. Here are all the ways that this backbone can be streamlined.

Add as many programmatic checks as possible

I firmly believe that any code or machine-readable specifications should be stored in version control and reviewed by all members of a team. This includes Markdown documentation as well as XML-as-code files (such as any no-code software services that saves its "code" as XML).

For any work being version controlled, I recommend looking into tooling to automate:

Formatting: Automatically re-structure content to a single standard (i.e. forcing code from being tab-indented to space-indented).
Linting: Checking if content adheres to a set of rules that require manual fixes (i.e. checking that all functions defined in code contain a top-level comment explaining what the function does and what its arguments mean).
Unit-testing: Run a full suite of automated checks against a code implementation. This ensures both backwards-compatibility as well as that new features match expected behavior.

More programmatic checks that are set up against repositories should also result in faster and more thoughtful code reviews. Human reviews should not be wasted on trivial checks like "this file is missing a newline at the end of it" and should instead be thoughtful and along the lines of "maybe it would be more maintainable for us to do things this way instead of that way".

Every time there is an issue like what takes place in Feature released in PROJ-123 causes an issue in production, consider whether a new programmatic check or an expansion to an existing programmatic check would have caught the issue during code reviews.

Automate programmatic checks and releases with CI/CD Tooling

Now that more programmatic checks are set up, it's time to fully automate them into this process using CI/CD tooling.

CI/CD tooling can be used for both the "code review" phase (to automate all the "How to Verify" checkboxes) as well as the "release" phase (to automate all the "Release Steps" checkboxes).

Examples:

I am a personal fan of navigating through DevOps with the following mindset:

Write a manual procedure for accomplishing a task and follow it enough times until it's set in stone. The procedure can now function as pseudocode for any automation implementation.
Convert the manual procedure to a streamlined programmatic command
Invoke the programmatic command in a CI/CD workflow

This mindset ensures that time isn't wasted troubleshooting the programmatic or CI/CD configuration until the team fully knows whether its useful to incorporate it into their processes (or to know if the process is too time consuming to continue doing manually).

Conclusion

The process outlined here is not a reinvented wheel. For further reading, I recommend reading any guide about Trunk-Based Development, the actual terminology for this process (the "trunk" is the main branch on a git repository).

Software development by now is a solved science, and it's just a matter of understanding and utilizing these best practices. I'm thankful for having had the opportunity to experience the pain of a software development team that greatly struggled without having a system like this in place.

By following this process, I can personally guarantee that the following issues will be resolved:

No more asking why a line of code was added or when it was added
No more worry about having your work wiped out by someone else's code change. Team members can work against the same file in their own branches without worrying about stepping on someone else's toes (this will require navigating a merge conflict, but a merge conflict is way better than losing your work entirely).
All members of the team are aware of all changes being worked on across all codebases and have the opportunity to ask questions and keep open communication during code reviews. No more knowledge silos during software development.
All review, release, and rollback processes are documented and can be followed by anyone on the team without having a single-person-of-failure.
No more worry about performing rollbacks. All changes being pushed are easy to investigate for root causes.
Any team can become a well-oiled machine.

Strategies to reduce "visual" code complexity

Kathryn DiPippo — Sun, 06 Apr 2025 14:55:20 +0000

Code readability is a big factor to that maintenance process. Code that is written well is easy and fast to read when problems arise. Same with infrastructure, it's never about just the initial build. Those structures need to be built with maintenance in mind: how to extend the longevity of a project to make the hours spent building it worthwhile. We need to care about the full lifecycle of any feature at every step.

Understanding how to reduce "visual" complexity is key, not just code complexity (although both, in my opinion, go hand-in-hand: code that has less cyclomatic complexity is often easier to read).

This article contains some general philosophies I've adopted with my approach to programming, specifically with how to better arrange logic statements. I am not the pioneer of these concepts and will link to other blog posts that I feel are more articulate about this than I am.

All examples are written with Python, but this applies to all programming languages with logic operators.

Sections

Go completely "else"-less
"Flip" logic statements to arrange by size of inner content
Implement "getters" where possible over "setters"
Try/Catch the smallest statement possible

Go completely "else"-less

Avoiding using else statements entirely has been the most dramatic but effect change I've enacted to write better code. else is convenient but masks the actual condition being hit before code is run.

Suppose I have the following code:

if condition_1:
    print("🟦")
else:
    if condition_2:
        print("🟨")
    else:
        print("🟥")

What is the true logic that results in "🟥" getting printed? If that line was actually an exception thrown, how can I quickly understand this code in order to address the issue at hand? The example here is a smaller implementation of what might be more complicated logic routing and inner blocks of code.

Instead, consider firstly explicitly writing what each else means, as in the negative of the previous if statement:

if condition_1:
    print("🟦")
if not condition_1:
    if condition_2:
        print("🟨")
    if not condition_2:
        print("🟥")

Being able to articulatly write out each else may make it easier to find where guard clauses could be used, or where it would be more readable to combine the sub-conditions together. I am personally a fan of migrating this type of block into a function and returning with each if statement hit:

def print_color() -> None:
    if condition_1:
        print("🟦")
        return

    if condition_2:
        print("🟨")
        return

    print("🟥")

The only exception to else would be for very small snippets of code where it is convenient to do so, such as Python's version of a ternary operator:

new_variable = "Value1" if condition_1 else "Value2"

Another example is if I need to shove a mapped value into a method where it makes sense for me to NOT keep the mapped value for other purposes:

run_weird_function(
  input_arg = "Value1" if condition_1 else "Value2"
)
# Strs "Value1" and "Value2" are not referenced anywhere else in the code.

But in general, my recommendation is to move this to a function and ensure that the function name is straight-forward when read later throughout the codebase:

def get_variable() -> str:
    if condition_1:
        return "Value1"
    return "Value2"

Recommended articles for further reading:

"Flip" logic statements to arrange by size of inner content

This goes hand-in-hand with the removal of ambiguous else statements: the shortest amount of code should get run first to allow for the larger blocks of code to lie "flat" / run with less indentation. Guard clauses are a good example, where the "content" is typically a single line of code to raise an exception.

Take this code snippet for example:

for entry in full_list:
    if condition_1:
        print("🟦")
        print("🟦")
        print("🟦")
        print("🟦")
        print("🟦")
        print("🟦")
        print("🟦")
        print("🟦")
        print("🟦")
    else if condition_2:
        print("🟨")
        print("🟨")
        print("🟨")
    else:
        raise RuntimeError("🟥")

In order for me to understand the else here, I need to physically scroll far up the code and mentally negate the other two conditions. I use the indent-rainbow VSCode extension to help with this type of troubleshooting, but this feature requires me to load the code locally. GitHub's code viewer does not have a feature at this time to easily show what if each else is tied to.

The RuntimeError() here should be migrated to the top as a proper guard clause that clearly states why it is getting run. I do this by "flipping" the logic: negate the condition and move it first.

Following it, the 3 "🟨" statements should be checked and run first before the 9 "🟦" statements. I am also a fan of adding a comment for any "implicit" logic statements being checked as a result of this re-arranging:

for entry in full_list:
    if not condition_1 and not_condition_2:
        raise RuntimeError("🟥")

    if condition_2:
        print("🟨")
        print("🟨")
        print("🟨")
        continue

    # if condition_1:
    print("🟦")
    print("🟦")
    print("🟦")
    print("🟦")
    print("🟦")
    print("🟦")
    print("🟦")
    print("🟦")
    print("🟦")

continue is my friend here for statements like this. But ideally, I would migrate any logic that needs to "exit early" into a separate function with proper return statements.

Implement "getters" where possible over "setters"

In my experience, it is way more difficult to both unittest and debug a function if the output of the command is to modify an existing structure in-place. Instead, write individual getters for separate actions performed and then stitch them together into clearly-defined setters

For example, say I have an initial dict and want to instead return a different dict:

before_dict = {  # Called "before_dict.json" in later example.
    "00001": {
        "date": "2025-01-01",
        "user": "jdoe",
        "title": "Lorem"
    },
    "00002": {
        "date": "2025-01-02",
        "user": "jsmith",
        "title": "Ipsum"
    }
}

# ... code goes here ...

after_dict = {  # Called "after_dict.json" in later example.
    "00001": {
        "entry": "2025-01-01 | jdoe | Lorem"
    },
    "00002": {
        "entry": "2025-01-02 | jsmith | Ipsum"
    }
}

One strategy might be to write a method that modifies in-place the input dict with the updated values:

def modify_input_dict(input_dict: dict) -> None:
    for entry in input_dict:
        input_dict[entry]["entry"] = " | ".join(
            [
                input_dict[entry]["date"],
                input_dict[entry]["user"],
                input_dict[entry]["title"]
            ]
        )
        del input_dict[entry]["date"]
        del input_dict[entry]["user"]
        del input_dict[entry]["title"]

While this code will perform the task at hand, the lack of rigidity of the structure makes it more confusing to understand what the data looks like during its execution. We might need to do extra checks in other parts of the code (i.e. to check for the existing of keys like date) due to the lack of enforcement in this first upstream method. Another idea would be to not delete the keys in this example and to instead attach the entry field to the dict; but over time, that might cause this structure to become very large and filled with redundant fields.

Unit-testing with this approach also becomes a necessity: the stored mocks for the input and output would be useful as "documentation" for the modification being performed.

Instead, I recommend always writing methods that return and then using those as "setters" if absolutely necessary:

def get_entry(date: str, user: str, title: str) -> str:
    return f"{date} | {user} | {title}"

def get_modified_input_dict(input_dict: dict) -> dict:
    modified_dict = {}
    for entry in input_dict:
        modified_dict[entry] = {
            "entry": get_entry(
                input_dict[entry]["date"],
                input_dict[entry]["user"],
                input_dict[entry]["title"]
            )
        }
    return modified_dict

The method cleanly defines the "before" and "after" in its own implementation. I understand the structure of output because I can see the whole thing here without needing to navigate to the unit-test fixtures. Writing unit-tests against this method is also easier: I can test the "actual" output against an "expected" output with a fed in "input" without needing to keep track of one or more variables for modifications:

def test_before() -> None:
    """Tests for the implementation where data is modified in-place."""
    actual_output = get_mock("before_dict.json")  # We confusingly name this "output" despite it being the input mock.
    modify_input_dict(actual_output)
    expected_output = get_mock("after_dict.json")
    assert actual_output == expected_output

def test_after() -> None:
    """Tests for the implementation where getters are used instead."""
    input_dict = get_mock("before_dict.json")
    actual_output = get_modified_input_dict(input_dict)
    expected_output = get_mock("after_dict.json")
    assert actual_output == expected_output

Try/Catch the smallest statement possible

Suppose I have the following function:

def perform_task() -> None:
    try:
        value1 = get_variable()
        value2 = get_variable()
        func_might_throw_runtime_error(value1)
        value3 = func_might_throw_assertion_error(value1)
        func_might_throw_runtime_error(value2)
        value3 = transform_values(value1, value2, value3)
        func_might_throw_runtime_error(value3)
        value3.sort()
        return value3
    except ...

If the logs return a RuntimeError, how do I know which statement here was the root cause? What if the message returned from func_might_throw_runtime_error() is the same regardless of the input being sent in? Maybe we don't have control over this method, so we lack the ability to update the message with the specific erroring value.

At-a-glance, there is a lot going on to adequately drill into the problem without cross-referencing with the stack trace. We will eventually find the problem, but it will take extra, unnecessary lost time to get there.

Instead, each try/except should be set to wrap exactly the amount of code where the error can be encountered:

def perform_task() -> None:
    value1 = get_variable()
    value2 = get_variable()

    try:
        func_might_throw_runtime_error(value1)
    except RuntimeError as runtime_error:
        raise RuntimeError(f"Error: value1 = {value1}") from runtime_error

    try:
        value3 = func_might_throw_assertion_error(value1)
    except AssertionError as assertion_error:
        raise AssertionError(f"Error: value1 = {value1}") from assertion_error

    try:
        func_might_throw_runtime_error(value2)
    except RuntimeError as runtime_error:
        raise RuntimeError(f"Error: value2 = {value2}") from runtime_error


    value3 = transform_values(value1, value2, value3)

    try:
        func_might_throw_runtime_error(value3)
    except RuntimeError as runtime_error:
        raise RuntimeError(f"Error: value3 = {value3}") from runtime_error

    value3.sort()
    return value3

The above snippet serves as an example of what might be seen in practice. The resulting method is significantly larger but is much more useful when needing to drill into issues. We can clearly map each inner function called to the outer error message in our logs without the stack trace.

We may also be making incorrect assumptions around the behaviors of the other methods present here too. What if transform_values() also returns a RuntimeError with certain inputs? We can save ourselves so much heartache and debugging hours by updating our code to try/except wrap each method; this will surface these assumptions when they're encountered properly and allow us to go in and wrap those assumptions with better handling.

My go-to Python snippets

Kathryn DiPippo — Wed, 24 Apr 2024 13:19:49 +0000

My list of go-to Python snippets that I frequently look up, all in one place! Most of the time, I'm just searching for how to parse files into Pythonic objects.

Read File Into Words
Read JSON File into Dict
Read CSV File into Dict List
- Read CSV File into Dict List without "\ufeff"
Read YAML File into Dict

Read File Into Words

def read_file_into_words(filename: str) -> list[str]:
    """Parse a text file into a 2D list of words, i.e.
        "This is an example file with
        multiple lines of text" becomes
        ["This is an example file with", "multiple lines of text"].

    Args:
        filename (str): Path to text file.

    Returns:
        list[list[str]]: 2D list of string words.
    """
    with open(filename, "r", encoding="utf-8") as input_file:
        content = input_file.readlines()
    content = [word.strip() for word in content]
    return content

Read JSON File into Dict

import json

def read_json_file_into_dict(filename: str) -> dict:
    """Parse a JSON file into a dictionary.

    Args:
        filename (str): Path to JSON file.

    Returns:
        dict: Dictionary representation of JSON file contents.
    """
    with open(filename, "r", encoding="utf-8") as json_file:
        content = json.load(json_file)
    return content

Read CSV File into Dict List

import csv

def read_csv_file_into_dict_list(filename: str) -> list[dict]:
    """Parse a CSV file into a list of dictionaries.

    Args:
        filename (str): Path to CSV file.

    Returns:
        dict: List of dictionary representations of CSV file contents.
    """
    content = []
    with open(filename, newline="", encoding="utf-8") as csv_file:
        reader = csv.DictReader(csv_file)
        for row in reader:
            content.append(dict(row))
    return content

Read CSV File into Dict List without "\ufeff"

Modification of the previous snippet with an additional method that removes the \ufeff special character if present in the column name.

import csv

def rename_dict_key(input_dict: dict, old_key: str, new_key: str) -> dict:
    """Rename key in dictionary while preserving original key order.

    Args:
        input_dict (dict): Dictionary to apply key rename.
        old_key (str): Current name of key.
        new_key (str): New name of key.

    Returns:
        dict: Dictionary with applied key rename.
    """
    return {key if key != old_key else new_key: value for key, value in input_dict.items()}

def read_csv_file_into_dict_list(filename: str) -> list[dict]:
    """Parse a CSV file into a list of dictionaries.

    Args:
        filename (str): Path to CSV file.

    Returns:
        dict: List of dictionary representations of CSV file contents.
    """
    text_content = read_file_into_words(filename)
    print(text_content[0])

    content = []
    with open(filename, newline="", encoding="utf-8") as csv_file:
        reader = csv.DictReader(csv_file)
        for row in reader:
            row_dict = dict(row)
            row_keys_to_rename = [current_key for current_key in row_dict.keys() if "\ufeff" in current_key]

            for row_key_to_rename in row_keys_to_rename:
                row_dict = rename_dict_key(row_dict, row_key_to_rename, row_key_to_rename.replace("\ufeff", ""))

            content.append(row_dict)
    return content

Read YAML File into Dict

def read_yaml_into_dict(filename: str) -> dict:
    """Return content of YAML file as dictionary.

    Args:
        filename (str): Path to YAML file to read and return.

    Returns:
        dict: Content of YAML file as dictionary.
    """
    with open(filename, "r", encoding="utf-8") as yaml_file:
        yaml_contents = yaml_file.read()
    return yaml.safe_load(yaml_contents)

How to configure Makefile to dynamically generate list of commands

Kathryn DiPippo — Mon, 04 Mar 2024 17:55:38 +0000

In past projects I've worked on, the first command inside the Makefile was always a manually maintained list of commands available in the file:

.PHONY: list command-one command-two

list:
    @echo "command-one    This command performs A, B, and C"
    @echo "command-two    This command performs X, Y, and Z"

command-one:
    @echo "Example command"

command-two:
    @echo "Example command"

But thanks to the Makefile set up in this Python starter project, there's an even handier way to have make list shown above be generated without updating two places.

How it works

At the top of the Makefile

.PHONY: help
help:             ## Show the help.
    @echo "Usage: make <target>"
    @echo ""
    @echo "Targets:"
    @fgrep "##" Makefile | fgrep -v fgrep

The fgrep here is the key, where the Makefile itself is parsed for any comments prefixed with ## and printing the entire lines. Any comments with just one # will be ignored. With the help command itself, the second line is what is fgrep-ed and printed.

Full example

.PHONY: help
help:             ## Show the help.
    @echo "Usage: make <target>"
    @echo ""
    @echo "Targets:"
    @fgrep "##" Makefile | fgrep -v fgrep

.PHONY: command-one
command-one:      ## This command performs A, B, and C
    @echo "Example command"
    # Inner comment to describe script.
    @echo "run shell script"

.PHONY: command-two
command-two:      ## This command performs X, Y, and Z
    @echo "Example command"

.PHONY: command-hidden
command-hidden:   # This command performs initialization
    @echo "Example hidden command"

The results of running make or make help:

Usage: make <target>

Targets:
help:             ## Show the help.
command-one:      ## This command performs A, B, and C
command-two:      ## This command performs X, Y, and Z

Note: for commands with dependencies to others, I found it useful to instead copy the command for the comment line and then have a second line for the dependencies, i.e.:

.PHONY: command-two
command-two:      ## This command performs X, Y, and Z
command-two: clean build
    @echo "Example requires 'make clean' & 'make build'"

Thought this was a really interesting configuration and wanted to share. Hope you find it useful too!

You don't need a personal website (and why I no longer have one)

Kathryn DiPippo — Sat, 16 Oct 2021 07:16:05 +0000

I often tell the story of how a simple website made me fall in love with programming. I was given a brief training assignment to build a website from scratch using HTML and CSS, with no prior knowledge beforehand. Something about the immediate back-and-forth between building the code and testing the display made something click. Soon after, I would spend hours after work building neat dashboards and landing pages with server-less implementations. Google Sheets used as a replacement for databases, hundreds of lines of jQuery added for interactivity. All of this was incorporated into my personal site, which served as both a list of my projects and demonstration of my abilities throughout the years.

Several years and hundreds of hours later, I am retiring my personal site. This decision was not easy to come to, but I know that my goals and abilities at this moment make it disadvantageous to keep it up.

Preface: Do you need a personal site?

Before getting started, I would still advocate for creating your own personal site if there's a specific need that it fills. If you are a freelance website builder, a personal site is necessary as a way of showcasing your skills. Others with design-oriented positions may find that the complete creative control a personal site provides best projects your abilities than using other platforms like Dribbble. Additionally, a personal site can be useful as a central resource if you have a sizeable audience, even if the only content for display is a list of your social media profiles.

I do not fall into the above categories. The goals of my own personal site are different from the goals of the examples listed above. Here are my main reasons for going forward with the deprecation and what may be insightful if you are skeptical about your own personal website:

1) It's a large time sink

There's just something about my personal website that makes me devote way more time than necessary to updating it. Routine visits to update the content to the latest updates inevitably become updates to the layout as well. The colors never feel correct, the layout is too short or too large or too cluttered, there's never enough transitions, there's too many transitions, etc. Every attempt to revamp the layout spins to a large, excessive TODO.

I certainly have a greater appreciation towards UI designers. I am my own nightmare client, constantly asking for small, inconsequential changes on the website and never feeling satisifed.

2) It's a larger time sink than it deserves to be

I recommend this article for additional info, as well as some interesting insights in the comments:

Don't waste your time on a (React) portfolio website - 60+ hiring managers and a survey

Johannes Kettmann for Profy.dev ・ Aug 6 '21

#react #career #webdev #javascript

The core summary boils down to a common sentiment around (software developer) hiring managers: a personal website is appealing to hiring managers but do not increase a candidate's chances. The primary emphasis and discussion in job interviews mostly focus on GitHub projects and contributions.

In order for a personal website to be effective, it needs to be fully functional (i.e. works with every phone and browser) and updated. You either spend an excessive amount of time to satisfy these requirements and gain nothing or risk being judged poorly if the website breaks while the hiring manager looks at it or it contains outdated information.

3) Other platforms fulfill the need for a personal website better

Breaking it down, the problems that a personal website solve can boil down to the following:

A concise location with links to other platforms like LinkedIn, Twitter, etc.
A portfolio with examples of project work with information

As a programmer with a number of open source repositories, my GitHub profile is the best showcase of my skills as a developer. I could attach a link to these repos on my resume when reaching out to companies, but even that feels too low-level. Thankfully, GitHub released an update in mid 2020 that added a customizable bio section visible on your profile page. I can highlight specific repositories and recent blog posts (if I was cool, I would automatically update a list of recent blog posts with a GitHub Actions workflow).

Other platforms cover any other holes that a personal site would cover, namely linking to Dev.to as a blogging platform and linking to LinkedIn for my professional credentials. These platforms are way easier to maintain and keep updated, without the mental push to want to update the UI in tandem.

What should replace your personal website instead?

Since GitHub will be the main focus of discussion in potential discussions with companies, it should also be the main focus when searching for a replacement to a personal site.

The steps I am taking in the deprecation process:

GitHub will act as my work portfolio:
- Update GitHub bio with clear links to my LinkedIn (for professional experience) and Dev.to (for blogging)
- Update GitHub projects with well defined READMEs
- Highlight a handful of my highlighted GitHub repositories. Write articles about their motivation and development.
LinkedIn will act as a resume (although my actual resume is now hosted on resume.io)
- Update LinkedIn to link back to GitHub and Dev.to profiles.
- Update profile to mimic Resume credentials list
Dev.to will act as a blog
- Update Dev.to profile to link back to GitHub and LinkedIn
- Write articles to fill in the remaining gaps a website would provide that a repo README can't fulfill, i.e. articles about obstacles encountered in projects and how you overcame them, reasoning behind selecting a specific toolchain, etc.

All of this will amount to 3 social media platforms to represent myself professionally. The key and motivation was to create a network of clearly defined tools that are first-and-foremost easy to update.

The true replacement for a personal site is something that will be updated on a regular basis above all else.

4 beginner-friendly Github Actions snippets

Kathryn DiPippo — Wed, 09 Jun 2021 14:34:15 +0000

GitHub Actions are extremely useful for those of you interested in setting up a CICD pipeline. Workflows are not only easy to implement but even easier to hook into your existing projects. These are a list of snippets that I found useful while trying to understand how to add these to my repos and how to develop my own.

I validated these snippets by setting up a separate repo to test opening PRs. I also recommend using nektos/act as a CLI to validate your workflows if you want a more seamless dev experience.

nektos / act

Run your GitHub Actions locally 🚀

Overview

"Think globally, act locally"

Run your GitHub Actions locally! Why would you want to do this? Two reasons:

Fast Feedback - Rather than having to commit/push every time you want to test out the changes you are making to your .github/workflows/ files (or for any changes to embedded GitHub actions), you can use act to run the actions locally. The environment variables and filesystem are all configured to match what GitHub provides.
Local Task Runner - I love make. However, I also hate repeating myself. With act, you can use the GitHub Actions defined in your .github/workflows/ to replace your Makefile!

How Does It Work?

When you run act it reads in your GitHub Actions from .github/workflows/ and determines the set of actions that need to be run. It uses the Docker API to either pull or build the necessary images, as defined in your workflow…

View on GitHub

List of Snippets

Post a comment automatically when a PR is opened
Run a PHP snippet
Open a Jira ticket when Dependabot opens a PR (for Hosted Jira instances)
Update empty PR body with a picture of Barney

Run a PHP snippet

For those of you (like myself) who work on teams developing primarily in PHP, it may be useful to also write your adhoc scripts in PHP for less context switching. TEST_ENV is placed here as an example environmental variable should anything about the PR's metadata need to be passed into the script.

Add the below file to your repo as .github/workflows/php.yml, or whatever other workflow name you find appropriate.



name: Testing php scripts

on:
  pull_request:
    types: [opened]

jobs:
  version_comment:
    name: Comment on a new PR with reminder
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - name: Setup PHP
      uses: shivammathur/setup-php@v2
      with:
        php-version: '7.3'
    - name: Check PHP version
      run: php --version
    - name: Run test.php
      env:
        TEST_ENV: "Hello from ENV"
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
      run: php scripts/test.php

Add the below file to your repo as scripts/test.php.



<?php

echo "Hello! This is scripts/test.php\n";

$github_token = getenv('GITHUB_TOKEN');
$test_env = getenv('TEST_ENV');

echo "$github_token = " . $github_token . "\n";
echo "$test_env = " . $test_env . "\n";

Open a Jira ticket when Dependabot opens a PR (for Hosted Jira instances)

The work for these files is that GitHub actions runs a Python script, and this script is the one responsible for firing off the Jira ticket's creation.

This script will work for Jira instances that are both Cloud-based and Hosted. However, for those of you using Cloud-based instances, I encourage you to look into using the Jira Create issue GitHub Action already provided by Atlassian. This will ensure you are using the latest and greatest. This feature is not available to those with Hosted instances.

Before starting, you will need to have a Jira account with a username and password that can be used on behalf of API calls. Naviate to https://github.com/YOUR_USERNAME/YOUR_REPO/settings/secrets/actions and select "New repository secret". Save your Jira account's username as JIRA_USER and your Jira account's password as JIRA_PASS.

Add the below file to your repo as .github/workflows/dependabot_to_jira.yml, or whatever other workflow name you find appropriate.



name: Dependabot Jira Sync

on:
  pull_request:
    types: [opened]

jobs:
  run_jira_ticket:
    name: Create Jira Ticket for Dependabot PR
    if: contains(github.event.pull_request.labels.*.name, 'Dependencies')
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - name: Set up Python
      uses: actions/setup-python@v1
      with:
        python-version: '3.x'
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements.txt
    - name: Create Jira Ticket and Update PR Title
      env:
        GITHUB_JSON: ${{ toJson(github) }}
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        JIRA_USER: ${{ secrets.JIRA_USER }}
        JIRA_PASS: ${{ secrets.JIRA_PASS }}
      run: python3 scripts/jira_ticket.py

Add the below file to your repo as scripts/jira_ticket.py.
Make sure that you change jiraUrl to the url of your Jira hosted instance.



import os
import json
import requests

# Get ENV variables.
def getENV():
  githubJson = os.environ['GITHUB_JSON']
  githubToken = os.environ['GITHUB_TOKEN']
  github = json.loads(githubJson)
  jiraUser = os.environ['JIRA_USER']
  jiraPass = os.environ['JIRA_PASS']
  return github, githubToken, jiraUser, jiraPass

def createJiraTicket(jiraUser, jiraPass, githubPRTitle, githubPRURL):
  jiraUrl = f"https://backlog.YOURINSTANCE.com/rest/api/2/issue"
  jiraHeaders = {"Content-Type": "application/json"}
  jiraBody = {
    "fields": {
      "project": {"key": "EEE"},
      "summary": f"{githubPRTitle}",
      "description": f"{githubPRURL} | {jiraUser}",
      "issuetype": {"name": "Ticket"}
    }
  }
  response = requests.post(jiraUrl, auth=(jiraUser, jiraPass), headers=jiraHeaders, json=jiraBody)
  print(response.status_code)
  if response.status_code == 201:
    jiraResponse = json.loads(response.text)
    return jiraResponse['key']

# Make API call to GitHub to change PR title.
def updateGithubPRTitle(githubToken, jiraKey, githubPRTitle, githubRepo, githubPRNum):
  githubHeaders = {"Authorization": f"token {githubToken}"}
  githubBody = {"title": f"{jiraKey} | {githubPRTitle}"}
  githubUrl = f"https://api.github.com/repos/{githubRepo}/pulls/{githubPRNum}"
  response = requests.patch(githubUrl, json=githubBody, headers=githubHeaders)
  print(response.status_code)

if __name__ == "__main__":
  github, githubToken, jiraUser, jiraPass = getENV()

  githubRepo = github['repository']
  githubPRURL = github['event']['pull_request']['_links']['html']['href']
  githubPRTitle = github['event']['pull_request']['title']
  githubPRNum = github['event']['pull_request']['number']

  jiraKey = createJiraTicket(jiraUser, jiraPass, githubPRTitle, githubPRURL)
  print(jiraKey)
  updateGithubPRTitle(githubToken, jiraKey, githubPRTitle, githubRepo, githubPRNum)

Update empty PR body with a picture of Barney

One of my coworkers used to do this whenever a PR was opened without a body for a significant amount of time. This is a fun GitHub action the team implemented together to learn more about the GitHub action development process.

This snippet is different from the others in that we constructed our own connector and stored it in the repo with the workflow's file. Alternatively, if you wish for the work to be more widely available, you would keep the repo dedicated to this singular action and publish it to the Marketplace.

Add the below file to your repo as .github/workflows/barney.yml, or whatever other workflow name you find appropriate.



name: Body by Barney

on:
  pull_request:
    types: [opened]

jobs:
  barney_job:
    runs-on: ubuntu-latest
    name: Barney is here
    env:
      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
    steps:
      # To use this repository's private action, you must check out the repository
      - name: Checkout
        uses: actions/checkout@v2
      - name: I love you
        uses: ./ # Uses an action in the root directory
        id: barney

barney.yml points to the root of the repository for where the action is located, specifically in this case an action.yml file. Add the below file to your repo as action.yml.



name: 'Barney'
description: 'Barney'
runs:
  using: 'docker'
  image: 'Dockerfile'
branding:
  icon: heart
  color: red

action.yml then points to a Dockerfile, also located to the root of your repo. Add the below file to your repo as Dockerfile.



FROM node:slim

# A bunch of `LABEL` fields for GitHub to index
LABEL "com.github.actions.name"="Body by Barney"
LABEL "com.github.actions.description"="Auto-populate empty PR descriptions."
LABEL "com.github.actions.icon"="edit"
LABEL "com.github.actions.color"="purple"
LABEL "repository"="http://github.com/YOUR_USERNAME/YOUR_REPO"
LABEL "homepage"="http://github.com/YOUR_USERNAME/YOUR_REPO"
LABEL "maintainer"="YOUR NAME (https://github.com/YOUR_USERNAME)"

# install
COPY package*.json ./
RUN npm ci --only=production

# start
COPY . .
ENTRYPOINT ["node", "/index.js"]

Dockerfile then points to /index.js for where the remainder of this action fires off, another file in the root. Add the below file to your repo as index.js. The action here will comment in logs whether Barney is applied or needed on newly opened PRs.



const { Toolkit } = require('actions-toolkit')
const getPR = require('./lib/get-pr')
const updatePR = require('./lib/update-pr')

Toolkit.run(async tools => {
  const pr_result = await getPR(tools)
  tools.log.debug("PR_BODY=" + pr_result.data.body);
  if (!pr_result.data.body) {
    await updatePR(tools);
    tools.log.success('Barney has been applied.')
  } else {
    tools.log.success('No Barney needed.')
  }
}, {
  event: ['pull_request.opened'],
  secrets: ['GITHUB_TOKEN']
})

index.js now looks to a folder lib/ where the rest of the files are located. Instead of storing the full implementation in one file, we still make use of separating the components of this action into parts - in the event the code needs to be revisited and refactored.

lib/ contains two files: get-pr.js for getting information about the PR and update-pr.js for sending an API call with the image of Barney in the description.

Add the below file to your repo as get-pr.js.



/**
 * Return the PR.
 * @param {import('actions-toolkit').Toolkit} tools
 */
module.exports = async function getPR (tools) {
  return tools.github.pulls.get({
    ...tools.context.repo,
    pull_number: tools.context.payload.pull_request.number,
  })
}

Add the below file to your repo as update-pr.js.



/**
 * Update the PR body with Barney.
 * @param {import('actions-toolkit').Toolkit} tools
 */
module.exports = async function updatePR (tools) {
  const barney_img = 'https://user-images.githubusercontent.com/1390106/66919899-3a406900-eff0-11e9-83ba-4e6c4c3dbfaf.jpg';
  return tools.github.pulls.update({
    ...tools.context.repo,
    pull_number: tools.context.payload.pull_request.number,
    body: '![image](' + barney_img + ')'
  })
}

And there you have it! You can expand update-pr.js with whatever content you find appropriate. This snippet will specifically update an empty PR body with the below image:

Using Slack for phone two-factor-authentication with help from Twilio

Kathryn DiPippo — Fri, 07 May 2021 18:14:42 +0000

For several months at my position, I worked with an online platform that solely utilized text messages for two-factor authentication (otherwise abbreviated 2FA). Security is vital, but security can also be increasingly annoying when the website forces logouts every 15 minutes or you frequently change location around the office and prefer to leave your phone locked in your desk.

The intent of this workaround is not to reduce security but to make it more personally convenient and discourage disabling the security entirely. I may not have my phone on me, but I'm definitely with my work laptop with Slack open.

The High-Level Pipeline

This procedure assumes two things:

The platform you want to use this procedure against allows for simple phone number TFA. You would receive a text message with a number or random character string code that you would type into the verification entry.
The platform allows Twilio-type numbers. This is not the case with every platform that uses phone number 2FA! Your mileage may vary.

The high-level steps that will be broken down in this walkthrough:

Obtain a Twilio phone number
Create webhook listener in Slack
Create phone webhook in Twilio
Update your 2FA phone number on the web platform with the Twilio phone number

We will be updating the 2FA phone number with the Twilio number last - in the event the platform requires a confirmation text message immediately upon changing. However, it may be useful to perform this step earlier in case there is a restriction against programmable Twilio-based phone numbers are mentioned in the earlier bullet points.

Obtain a Twilio Phone Number

Twilio is a cloud communications platform as a service (CPaaS) that allows software developers to programmatically make and receive phone calls, send and receive text messages, and perform other communication functions using its web service APIs. We will be using it as the pipeline connecting your platform to Slack.

When you create a Twilio account, you'll be given a single Twilio phone number with a limited amount of trial usage. However, despite how frequently I was logging into the platform I set this up for, it took over a year for the base free trial limits to be utilized - and after that, a single $15 payment has lasted far beyond that.

Additional phone numbers on this account may require additional purchases.

By the end of this step, you will now have a Twilio phone number complete with an area code. Copy this for later.

Create webhook listener in Slack

Navigate back to your Slack instance. Please check out Slack's walkthrough for setting up an incoming webhook before continuing. The page will guide you with how to make a Slack App and the permissions necessary. I recommend setting the webhook's destination to your own private DMs as opposed to a channel.

By the end of this step, you will now also have a Slack webhook URL formatted similarly to https://hooks.slack.com/services/???/???/???. Copy this for later.

Create phone webhook in Twilio Studio

Twilio Studio is a visual editor for building communications workflows. We will be using this as the brain to take messages sent to our Twilio phone number to a webhook. This step is performed before you change your phone number in the event the platform will send a confirmation text after applying the switch.

While logged into the Twilio platform, navigate to https://www.twilio.com/console/studio/dashboard. You can also navigate to Studio by selecting the icon with 3 dots on the left menu and select "Studio" that appears under the "Runtime" subheading.
Under "Recent Flows" select the button to "Create new flow".
For "Flow Name", I named my own flow "SMS Notification", but feel free to name it something with greater identification for its purpose. Select "Next".
Scroll down in the available templates to the option "Import from JSON". Select this box and then select "Next".
Copy the below JSON snippet into the text box that appears on this screen. Before continuing, change the value on line 45 for the webhook url from the placeholder I've put to the URL you generated as part of the second step.



{
  "description": "A New Flow",
  "states": [
    {
      "name": "Trigger",
      "type": "trigger",
      "transitions": [
        {
          "next": "post_to_slack",
          "event": "incomingMessage"
        },
        {
          "event": "incomingCall"
        },
        {
          "event": "incomingRequest"
        }
      ],
      "properties": {
        "offset": {
          "x": 0,
          "y": 0
        }
      }
    },
    {
      "name": "post_to_slack",
      "type": "make-http-request",
      "transitions": [
        {
          "event": "success"
        },
        {
          "event": "failed"
        }
      ],
      "properties": {
        "offset": {
          "x": 50,
          "y": 150
        },
        "method": "POST",
        "content_type": "application/json;charset=utf-8",
        "body": "{\"text\": \"{{trigger.message.Body}}\"}",
        "url": "https://hooks.slack.com/services/???/???/???"
      }
    }
  ],
  "initial_state": "Trigger",
  "flags": {
    "allow_concurrent_calls": true
  }
}

Clicking next will create the below end result:

This studio flow is its own webhook to listen for SMS triggers and forward the message to a Slack webhook. To connect this implementation to your phone number:

Go back to the left menu and select "Phone Numbers" under the "Super Network" subheading.
Click on the phone number you are going to use for this Platform -> Twilio -> Slack pipeline.
Scroll to the bottom of the "Configure" page to the "Messaging " section.
- "Configure With" should be set to "Webhooks, TwiML Bins, Functions, Studio, or Proxy"
- Change the "A Message Comes In" dropdown to "Studio Flow"
- Change the dropdown directly to the right to the name of the studio flow you created just earlier.
- Select "Save" at the bottom of the page.

You have now finished setting up Twilio for this phone number to trigger a Twilio-facing webhook to fire off a Slack webhook.

Update your 2FA phone number on the web platform with the Twilio phone number

Last but not least, update the platform with your new phone number. You should now be seeing a Slack message pop in every time you need to perform 2FA and enter an SMS code.

Hope this tutorial was helpful! Feel free to comment with any questions or updates to this procedure.

Submit Your Hacktoberfest Repos to up-for-grabs!

Kathryn DiPippo — Sun, 27 Sep 2020 00:25:39 +0000

As Hacktoberfest 2020 descends in less than a week (!!!), I wanted to give a shoutout to up-for-grabs.net.

up-for-grabs / up-for-grabs.net

This is a list of projects which have curated tasks specifically for new contributors. These issues are a great way to get started with a project, or to help share the load of working on open source projects. Jump in!

up-for-grabs.net

This repository contains the content for the Up-For-Grabs website, a list of projects with curated tasks for new contributors.

List your project

If you know of or own a project that should be listed on Up for Grabs follow the instructions to open a pull request.

Contributing

If you would like to get involved with the project, please read the CONTRIBUTING.md file as it contains:

Instructions about getting your environment setup
Guidance for testing locally
Commands to run to verify changes

How does the site work?

If you want to learn more about the various parts of the Up-for-Grabs project, the How it works document is a good overview.

Contributors ✨

Thanks to these wonderful people who have improved the code and documentation to help this project grow. (emoji key):

Filipe Magalhaes Gomes
🖋

Manish Kumar Khedawat
💻

SpaceEEC
💻

João Manoel Lins
🖋

Crystal

…

View on GitHub

I find it a super useful resource when looking for OSS to contribute to with my current skillset. I'd love for others who are prepping their projects for Hacktoberfest to make use of it - or those who are also exploring what's out there.

If you are prepping your repos for Hacktoberfest, you can use the Hacktoberfest label as the "beginner-friendly" tag when submitting. OpenFaaS, an open source implementation of Function as a Service, and Matomo, "the leading open alternative to Google Analytic", both use their "Hacktoberfest" issue labels in this manner, for example.

Hope you find this resource as helpful as I do. I'm more than happy to assist with any PRs if you want your repo added to the listing. Good luck during Hacktoberfest!

Rendering OpenAPI specs in Angular

Kathryn DiPippo — Sat, 19 Sep 2020 04:17:33 +0000

APIs and API documentation go hand-in-hand. With the help of of the swagger-ui-dist npm package, it's super easy to take an OAS YAML or JSON file and display it as a separate page for others to browse. This walkthrough will create an Angular component dedicated to showing the API documentation.

Assemble your OAS spec

Both JSON and YAML can be used for this process. The main key is that the file needs to be accessible via a live URL. You can do this by taking your spec and pushing it to a public repo and utlilizing GitHub's "Raw" file feature.

I will be using the files https://raw.githubusercontent.com/OAI/OpenAPI-Specification/master/examples/v3.0/petstore.yaml found in the examples/ folder for OAS v3.

OAI / OpenAPI-Specification

The OpenAPI Specification Repository

Add swagger-ui-dist to package.json and angular.json files

In the root of your project, run:



$ npm install swagger-ui-dist

This will add the swagger-ui-dist CSS and JS files needed to render the API documentation layout.

You'll next need to include said files into the "styles" and "scripts" section of your Angular project for them to be pulled in. See the lines below marked with +s for what to add to the project build:



"architect": {
  "build": {
    "builder": "@angular-devkit/build-angular:browser",
    "options": {
      "outputPath": "dist/example-angular-project",
      "index": "src/index.html",
      "main": "src/main.ts",
      "polyfills": "src/polyfills.ts",
      "tsConfig": "tsconfig.app.json",
      "aot": true,
      "assets": [
        "src/favicon.ico",
        "src/assets"
      ],
      "styles": [
+       "node_modules/swagger-ui-dist/swagger-ui.css",
        "src/styles.css"
      ],
      "scripts": [
+       "node_modules/swagger-ui-dist/swagger-ui-bundle.js",
+       "node_modules/swagger-ui-dist/swagger-ui-standalone-preset.js"
      ]
    },

Create a new Angular component

Continue as you would with adding an Angular component.



$ ng g c api

Update app-routing.module.ts to route https://localhost:4200/api to this component.



import { NgModule } from '@angular/core';
import { Routes, RouterModule } from '@angular/router';
import { HomeComponent } from './home/home.component';
import { ApiComponent } from './api/api.component';

const routes: Routes = [
  { path: '', component: HomeComponent },
  { path: 'api', component: ApiComponent },
];

@NgModule({
  imports: [RouterModule.forRoot(routes)],
  exports: [RouterModule]
})
export class AppRoutingModule { }

Spin up the project using ng serve to confirm that https://localhost:4200/api shows you the generated <p>api works!</p> message.

Updating ApiComponent to utilize swagger-ui-dist

Change the api.component.html file to include the below div. The swagger-ui ID will be what the OpenAPI spec display gets rendered inside.



<div id="swagger-ui"></div>

Meanwhile, api.component.ts should be updated with the following:



import { Component, OnInit } from '@angular/core';
import { SwaggerUIBundle, SwaggerUIStandalonePreset } from 'swagger-ui-dist';

@Component({
  selector: 'app-api',
  templateUrl: './api.component.html',
  styleUrls: ['./api.component.css']
})
export class ApiComponent implements OnInit {

  constructor() { }

  ngOnInit(): void {
    const ui = SwaggerUIBundle({
      dom_id: '#swagger-ui',
      layout: 'BaseLayout',
      presets: [
        SwaggerUIBundle.presets.apis,
        SwaggerUIBundle.SwaggerUIStandalonePreset
      ],
      url: 'https://raw.githubusercontent.com/OAI/OpenAPI-Specification/master/examples/v2.0/yaml/petstore.yaml',
      operationsSorter: 'alpha'
    });
  }

}

Going through the notable changes, we first import SwaggerUIBundle and SwaggerUIStandalonePreset from the swagger-ui-dist node package added earlier. This will allow us to initial the SwaggerUIBundle() call that gets made when the component initializes.

We can see that swagger-ui ID mentioned prior. This can be changed if needed (i.e. if you want multiple specs to show on the same page and need to distinguish between the two). For more information about these settings and ways to customize them, the SwaggerUI documentation can be found at https://swagger.io/docs/open-source-tools/swagger-ui/usage/configuration/.

Most notably, the url is just set to the raw.githubusercontent.com URL for the OpenAPI spec's YAML file.

Running ng serve on the same page, and you should now see your API documentation nicely rendered!

Conclusion

With just a few steps, we are now able to house a dynamic view of the API documentation in our Angular project. I recommend using this method to host your documentation with GitHub pages while also providing users to download the spec raw.

DEV Community: Kathryn DiPippo

The Essential Sprint Ceremonies

Table of Contents

Background

Daily Stand-up

Daily Stand-up: When

Daily Stand-up: How

Daily Stand-up: Why

Triage / Sizing Sessions

Triage / Sizing Sessions: When

Triage / Sizing Sessions: How

Triage / Sizing Sessions: Why

Backlog Grooming

Backlog Grooming: When

Backlog Grooming: How

Backlog Grooming: Why

Sprint Retro

Sprint Retro: When

Sprint Retro: How

Sprint Retro: Why

Sprint Planning

Sprint Planning: When

Sprint Planning: How

Sprint Planning: Why

Visualizing local git repository statuses with MermaidJS and a Python CLI

Table of Contents

Background

How to Use

Example Outputs

Python Script

How and Why to Use Patches

Table of Contents

Background

What is a Patch?

Why and When to Patch?

How to Create and Apply a Patch

Create wrapper install command

Initialize blank-slate state for third-party dependencies

Make changes until third-party dependency is functional

Create and assess initial patchfile result

Refine patchfile content

Create post-install patching step

List of Patching Commands

Command for Creating a Patch

Command for Applying a Patch

The Essential Software Development Process

Table of Contents

Background

Required Tooling

Issue Tracker

Git Platform

Getting Started

Setting up the Issue Tracker

Setting up the Git Repository

Walkthrough of the End-to-End Process

Issue PROJ-123 is ready to be worked on

Branch PROJ-123 is created on the git repository

Assignee pushes to PROJ-123 branch

Assignee creates Pull Request for PROJ-123 branch

Team reviews Pull Request for PROJ-123 branch

Assignee squashes and merges PROJ-123 branch to main branch

Assignee creates tag for PROJ-123 release

Feature released in PROJ-123 causes an issue in production

Recommended Enhancements to this Process

Add as many programmatic checks as possible

Automate programmatic checks and releases with CI/CD Tooling

Conclusion

Strategies to reduce "visual" code complexity

Sections

Go completely "else"-less

"Flip" logic statements to arrange by size of inner content

Implement "getters" where possible over "setters"

Try/Catch the smallest statement possible

My go-to Python snippets

Table of Contents

Read File Into Words

Read JSON File into Dict

Read CSV File into Dict List

Read CSV File into Dict List without "\ufeff"

Read YAML File into Dict