Table of Contents
Background
If you're like me, you have a folder with all of your git repositories held in the same location. In a more organized vision of myself, I switch between repos and work on tasks one-at-a-time. I start work in one git repository, get it finished, push it into a PR, and then I switch to another repo. After I release the PR, I go back into my local folder and switch back to main and pull in latest changes.
Instead though, I have often ended up with a ton of outstanding work in all of my different folders. I will start work only to be sidelined by more important issues and fires. By the time I return, enough time has passed that I've lost track of what the outstanding work needs to be continued. I'll make a separate folder "TEMP" and git clone a fresh version of the project, work in there, accidentally abandon that, keep making separate folders... until finally, my git repository folders are a mess of duplicates and work-in-progresses.
In the past to remedy this, I have manually typed out the folder structure and documented what branch I was looking at and the state of outstanding work and slowly whittle the folders back down to my ideal state. But what if I made a script that automatically assembled a Mermaid diagram instead?
How to Use
The provided Python script (compatible with both MacOS/Linux and Windows operating systems) at the bottom of this article contains a single command powered by the click library.
python script.py --help
Usage: script.py [OPTIONS]
CLI command to return a mermaid diagram of all git repositories - including
in subdirectories in a specified local folder along with the branch they're
currently pointed to and their number of uncommitted changes.
Options:
--root-dir TEXT Relative or absolute path to directory to traverse.
Defaults to treating current dir where the script is being
run as the root.
--only-errors If True, hide any git repositories that are pointed to the
base branch and have no outstanding changes.
--output TEXT If provided, save Mermaid string to file with filename.
--name TEXT If provided, filter results to the provided git repository
name.
--help Show this message and exit.
For example, this command will take the directory located at ../.. from where the script is being called, walk through all of its directories and subdirectories, and generate a Mermaid file mermaid.mmd that shows the complete tree structure and the status of all of the git repositories found.
python script.py --root-dir ../.. --output mermaid.mmd
The additional option --only-errors will hide any git repositories already at a "blank slate" status. This will enable me to know what folders I have yet to address.
python script.py --root-dir ../.. --output mermaid.mmd --only-errors
Also, the additional option --name will filter the git repositories to the name specified. This is useful for cases where the same git repository is cloned in multiple places and to generate diagram with an easy comparison.
Example Outputs
This is an example diagram returned from python script.py --root-dir ../.. --output mermaid.mmd:
flowchart LR
classDef HasChangesPointedToBase fill:#f0ad4e
classDef HasChangesNotPointedToBase fill:#d9534f
classDef NoChangesNotPointedToBase fill:#5bc0de
path["path"] --> path_to
path_to["to"] --> path_to_git-repo-1
path_to["to"] --> path_to_git-repo-2
path_to["to"] --> path_to_TEMP
path_to_TEMP["TEMP"] --> path_to_TEMP_git-repo-1
path_to["to"] --> path_to_TEMP2
path_to_TEMP2["TEMP2"] --> path_to_TEMP2_git-repo-2
path_to_git-repo-1["<b>git-repo-1</b><br>FEATURE-1<br>All changes committed"]:::NoChangesNotPointedToBase
path_to_git-repo-2["<b>git-repo-2</b><br>main<br>9 files changed, 70 insertions(+), 25 deletions(-)"]:::HasChangesPointedToBase
path_to_TEMP_git-repo-1["<b>git-repo-1</b><br>FEATURE-1<br>20 files changed, 150 insertions(+), 125 deletions(-)"]:::HasChangesNotPointedToBase
path_to_TEMP2_git-repo-2["<b>git-repo-2</b><br>main<br>All changes committed"]
This is an example diagram returned from python script.py --root-dir ../.. --output mermaid.mmd --only-errors:
flowchart LR
classDef HasChangesPointedToBase fill:#f0ad4e
classDef HasChangesNotPointedToBase fill:#d9534f
classDef NoChangesNotPointedToBase fill:#5bc0de
path["path"] --> path_to
path_to["to"] --> path_to_git-repo-1
path_to["to"] --> path_to_git-repo-2
path_to["to"] --> path_to_TEMP
path_to_TEMP["TEMP"] --> path_to_TEMP_git-repo-1
path_to_git-repo-1["<b>git-repo-1</b><br>FEATURE-1<br>All changes committed"]:::NoChangesNotPointedToBase
path_to_git-repo-2["<b>git-repo-2</b><br>main<br>9 files changed, 70 insertions(+), 25 deletions(-)"]:::HasChangesPointedToBase
path_to_TEMP_git-repo-1["<b>git-repo-1</b><br>FEATURE-1<br>20 files changed, 150 insertions(+), 125 deletions(-)"]:::HasChangesNotPointedToBase
Python Script
To run this script, the following Python modules need to be installed beforehand (either in a venv or globally):
click==8.1.7
Jinja2==3.1.3
"""CLI command to return a mermaid diagram of all git repositories - including in subdirectories in a specified local
folder along with the branch they're currently pointed to and their number of uncommitted changes."""
import os
import platform
import shutil
import stat
import uuid
from typing import Callable, Optional
import click
import jinja2
MERMAID_FULL_TEMPLATE = """
flowchart LR
classDef HasChangesPointedToBase fill:#f0ad4e
classDef HasChangesNotPointedToBase fill:#d9534f
classDef NoChangesNotPointedToBase fill:#5bc0de
{% for mermaid_route in mermaid_routes %}
{{ mermaid_route }}
{% endfor %}
{% for mermaid_element in mermaid_elements %}
{{ mermaid_element }}
{% endfor %}
"""
MERMAID_ELEMENT_TEMPLATE = """
{{ element_id }}
["
<b>{{ git_repo_name }}</b><br>
{{ git_branch }}<br>
{% if git_outstanding_changes == "" %}
All changes committed
{% else %}
{{ git_outstanding_changes }}
{% endif %}
"]
{% if has_changes and is_pointed_to_base_branch %}
:::HasChangesPointedToBase
{% elif has_changes and not is_pointed_to_base_branch %}
:::HasChangesNotPointedToBase
{% elif not has_changes and not is_pointed_to_base_branch %}
:::NoChangesNotPointedToBase
{% endif %}
"""
def _handle_readonly(function: Callable[[str], object], path: str, _) -> None:
"""On Windows, .idx files in the .git/objects/pack folder get set to readonly and throw PermissionError.
They need to be converted to write before deletion.
https://stackoverflow.com/questions/21778356/python-shutil-rmtree-cannot-remove-git-dir-on-win7
Args:
function (Callable[..., Any]): Function which raised the exception, depends on the platform and implementation.
path (str): Path name passed to Function.
excinfo (BaseException): Exception that was raised. Exceptions raised by onexc will not be caught.
"""
os.chmod(path, stat.S_IWRITE)
function(path)
def delete_file_or_folder(file_or_folder_name: str) -> None:
"""Delete file or folder at relative path. Pythonic version of "rm -rf".
Args:
file_or_folder_name (str): Relative path to file or folder to recursively delete.
Raises:
LookupError: Raised if input fails matching on isfile() or isdir().
"""
# Silence exception if path does not exit.
if not os.path.exists(file_or_folder_name):
return
if os.path.isfile(file_or_folder_name):
os.remove(file_or_folder_name)
return
if os.path.isdir(file_or_folder_name):
if platform.system() == "Windows":
# Try chmod-ing files if error occurs while running on Windows.
shutil.rmtree(file_or_folder_name, onexc=_handle_readonly)
return
shutil.rmtree(file_or_folder_name)
return
raise LookupError(f"delete_file_or_folder() input is neither file nor folder: '{file_or_folder_name}'")
class GitDirectory:
"""Class to represent a directory containing a git repository."""
def __init__(self, git_dir_path: str, base_branch_name: str = "main") -> None:
"""Initialize GitDirectory class.
Args:
git_dir_path (str): git directory path, i.e. "/path/to/git-repo".
"""
self.git_dir_path = git_dir_path
self.git_dir_path_parts = [entry for entry in git_dir_path.split("/") if entry]
self.git_repo_name = self.git_dir_path_parts[-1]
self.git_branch = self.get_git_branch()
self.base_branch_name = base_branch_name
self.git_outstanding_changes = self.get_git_outstanding_changes()
def _get_output_of_command(self, command: str) -> str:
"""Execute provided command and save terminal outputs to self's class attributes.
Raises:
RuntimeError: Thrown if command returned an error.
Args:
command (str): Terminal command to execute.
"""
# This method works by running the command with "> filename.txt" argument to save outputs to file,
# then reads that file for the output and deletes it before exiting.
temp_output_filename = f"{str(uuid.uuid4())}-command-output-tempfile.txt"
execute = os.system(f"{command} &> {temp_output_filename}")
with open(temp_output_filename, "r", encoding="utf-8") as temp_output_file:
temp_output_content = temp_output_file.read()
delete_file_or_folder(temp_output_filename)
if execute == 0:
return temp_output_content
raise RuntimeError(temp_output_content)
def _get_git_command_output(self, git_command: str) -> str:
"""Return output of provided git command in another directory.
Args:
git_command (str): Git command, i.e. "git example-command".
Returns:
str: Output of provided git command, after it is changed to "git -C /path/to/git-repo example-command".
"""
git_command = git_command.replace("git ", f"git -C {self.git_dir_path} ")
return self._get_output_of_command(git_command).strip()
def get_git_branch(self) -> str:
"""Return current branch in git repository.
Returns:
str: Current branch in git repository, i.e. "FEATURE-1".
"""
return self._get_git_command_output("git branch --show-current")
def get_git_outstanding_changes(self) -> str:
"""Get git outstanding changes status message.
Returns:
str: git outstanding changes status message, i.e. "9 files changed, 70 insertions(+), 25 deletions(-)".
Will return "" if there are no uncommitted changes.
"""
git_diff_stat = self._get_git_command_output("git diff --stat")
return git_diff_stat.split("\n")[-1].strip()
def has_changes(self) -> bool:
"""Return True if git repository has outstanding, uncommited changes.
Returns:
bool: True if git repository has outstanding, uncommited changes.
"""
return self.git_outstanding_changes != ""
def is_pointed_to_base_branch(self) -> bool:
"""Return True if git repository is pointed to the base branch.
Returns:
bool: True if git repository is pointed to the base branch.
"""
return self.git_branch == self.base_branch_name
def get_mermaid_element(self) -> str:
"""Get Mermaid representation of GitRepository object.
Returns:
str: Mermaid representation of GitRepository object, i.e.
'path_to_git-repo["<b>git-repo</b><br>FEATURE-1<br>All changes committed"]:::NoChangesNotPointedToBase'
"""
environment = jinja2.Environment()
template = environment.from_string(MERMAID_ELEMENT_TEMPLATE)
return (
template.render(
element_id="_".join(self.git_dir_path_parts).replace("/", "_"),
git_repo_name=self.git_repo_name,
git_branch=self.git_branch,
git_outstanding_changes=self.git_outstanding_changes,
has_changes=self.has_changes(),
is_pointed_to_base_branch=self.is_pointed_to_base_branch(),
)
.replace("\n", "")
.strip()
)
def get_mermaid_routes(self) -> list[str]:
"""Get all Mermaid element routes, pointing the connections between directories and their subdirectories
inferred from the path to this git repository.
Returns:
list[str]: List of all mermaid element routes inferred from the path to this current git repository, i.e.
['path["path"] --> path_to', 'path_to["to"] --> path_to_git-repo']
"""
mermaid_routes: list[str] = []
for index in range(1, len(self.git_dir_path_parts)):
element_one_id = "_".join(self.git_dir_path_parts[:index])
element_one_name = self.git_dir_path_parts[index - 1]
element_two_id = "_".join(self.git_dir_path_parts[: index + 1])
mermaid_routes.append(f'{element_one_id}["{element_one_name}"] --> {element_two_id}')
return mermaid_routes
def get_all_git_dir_paths(root_dir: str = ".") -> list[str]:
"""Get all git directories (including subdirectores) from root where script is being called.
Returns:
list[str]: git directory paths, i.e. ["/path/to/git-repo"].
"""
git_dir_paths = []
for root, dirs, files in os.walk(os.path.abspath(root_dir)): # pylint: disable=unused-variable
for dir_name in dirs:
dir_path = os.path.join(root, dir_name)
if not os.path.isdir(f"{dir_path}/.git"):
continue
git_dir_paths.append(dir_path)
return git_dir_paths
@click.command(
help="""CLI command to return a mermaid diagram of all git repositories - including in subdirectories in a specified
local folder along with the branch they're currently pointed to and their number of uncommitted changes."""
)
@click.option(
"--root-dir",
required=False,
type=str,
default=".",
help="""Relative or absolute path to directory to traverse. Defaults to treating current dir where the script is
being run as the root.""",
)
@click.option(
"--only-errors",
"show_only_errors",
is_flag=True,
required=False,
type=bool,
default=False,
help="If True, hide any git repositories that are pointed to the base branch and have no outstanding changes.",
)
@click.option(
"--output",
"output_filename",
required=False,
type=str,
help="If provided, save Mermaid string to file with filename.",
)
@click.option(
"--name",
"git_repo_name",
required=False,
type=str,
help="If provided, filter results to the provided git repository name.",
)
def main(
root_dir: str = ".",
show_only_errors: bool = False,
output_filename: Optional[str] = None,
git_repo_name: Optional[str] = None,
) -> None:
"""CLI command to return a mermaid diagram of all git repositories - including in subdirectories in a specified
local folder along with the branch they're currently pointed to and their number of uncommitted changes.
Args:
root_dir (str, optional): Relative or absolute path to directory to traverse. Defaults to "." to current dir.
show_only_errors (bool, optional): If True, hide any git repositories that are pointed to the base branch and
have no outstanding changes.
output_filename (Optional[str], optional): If provided, save Mermaid string to file with filename.
git_repo_name (Optional[str], optional): If provided, filter results to the provided git repository name.
"""
git_directories: list[GitDirectory] = [
GitDirectory(git_dir_path) for git_dir_path in get_all_git_dir_paths(root_dir)
]
if git_repo_name is not None:
git_directories = [
git_directory for git_directory in git_directories if git_directory.git_repo_name == git_repo_name
]
mermaid_elements: list[str] = []
mermaid_routes: list[str] = []
for git_directory in git_directories:
# If --only-errors, skip git repositories that are pointed to the base branch without outstanding changes.
if show_only_errors and git_directory.is_pointed_to_base_branch() and not git_directory.has_changes():
continue
# If --name provided, skip git repositores that do not have the same git repository name provided.
if git_repo_name is not None and git_directory.git_repo_name != git_repo_name:
continue
mermaid_elements.append(git_directory.get_mermaid_element())
mermaid_routes += git_directory.get_mermaid_routes()
mermaid_routes = list(set(mermaid_routes))
environment = jinja2.Environment()
template = environment.from_string(MERMAID_FULL_TEMPLATE)
mermaid_full_str = template.render(
mermaid_elements=mermaid_elements,
mermaid_routes=mermaid_routes,
)
if output_filename is None:
print(mermaid_full_str)
return
with open(output_filename, "w", encoding="utf-8") as output_file:
output_file.write(mermaid_full_str)
if __name__ == "__main__":
main()
Top comments (0)