DEV Community: Henrique Sagara

Life After Graduation: A New Chapter

Henrique Sagara — Sat, 29 Mar 2025 16:11:17 +0000

It’s been a while since I’ve written anything here. This blog has mostly been dedicated to sharing my open-source contributions. However, due to a few things that have been happening in my life, I’ve stepped away from contributing for now.

Post-Grad Life

Since graduating in December 2024, I’ve been actively applying for software developer roles. Unfortunately, the job market has been quite tough for recent grads.

I was lucky enough to land a remote developer job—but I was laid off during my very first week. The reason? Management expected me to have more experience than I actually did. It was a tough moment, but also a valuable lesson in how unpredictable this industry can be.

After the Layoff

Since then, I’ve been focusing on a side project I started with a friend during our final year of college. The project got accepted into our college’s startup incubator, which was incredibly exciting and gave us a real sense of momentum.

I’ve also continued job hunting, but it’s been a bit discouraging. I’ve been applying through the usual platforms—LinkedIn, Indeed, Glassdoor—and it’s frustrating to receive the same “Unfortunately…” email over and over. Sometimes recruiters reach out to me on LinkedIn, but after I express interest, they just disappear without a word. It’s draining, but I know I’m not alone in this experience.

What’s Next

I’ve decided to go all-in on the project with my friend. It’s something we’re building together from the ground up, and I’m genuinely excited about it. I’ll be sharing updates on our journey here—not technical details, but the ups and downs of building something of our own.

My hope is that by sharing this experience, it might resonate with someone else out there trying to figure things out after graduation. Maybe even inspire someone to start their own thing too.

I’m also open to feedback, constructive criticism, or suggestions—feel free to leave a comment below!

Building a Digital Signage Content Management System: A Challenge and a Journey

Henrique Sagara — Tue, 17 Dec 2024 20:44:06 +0000

Building a Digital Signage Content Management System: A Challenge and a Journey

Recently, I completed an exciting code challenge for a media company specializing in digital kiosks. The task? To create a Digital Signage Content Management System that connects a web application with a desktop display app using WebSockets for communication.

The Project Overview

The challenge required building a system with the following core features:

Web Dashboard Features

A simple content creation interface using Fabric.js to add text, images, and basic layouts.
Content preview functionality.
Optional content scheduling capabilities.
A content list with status indicators (e.g., draft, posted).
Basic content organization using folders or categories.
Device status monitoring for the connected display app.

Electron Display Application

Full-screen content display with support for text and images.
Manual sync button with a status indicator.
An auto-sync toggle option for seamless updates.
Offline content playback for reliability.
System tray controls for managing the app.
A connection status indicator for the WebSocket server.
Basic error notifications.

Offline Capabilities

Local content storage for offline playback.
Content caching to ensure smooth operation without an internet connection.
Automatic content synchronization when the connection is restored.
Offline status indicators for visibility.

Tackling the Implementation

The first major challenge I faced was designing the implementation. I restarted the project multiple times, initially focusing on the Electron app. However, after much research and experimentation, I realized that starting with the web app and then connecting it to the Electron display app was a better choice. This allowed me to establish the core functionality of content management before tackling the communication layer with WebSockets.

For this project I chose Next.js due to its better integration with Vercel, making it incredibly easy to deploy and scale the app.

Learning Fabric.js and Electron from Scratch

One of the biggest hurdles was learning Fabric.js and Electron, two tools I had never used before:

Fabric.js: This powerful canvas library enabled the creation of interactive content like text, images, and shapes. However, its documentation was not very beginner-friendly, so I spent significant time watching tutorials and experimenting with examples.
Electron: I appreciated Electron's well-documented "Quick Start" guide, which allowed me to quickly build the desktop application. It helped me understand how to wrap a web app into a desktop environment and implement offline features.

The WebSocket Challenge

The WebSocket feature was particularly tricky. I successfully built a WebSocket server that enabled the Electron app to replicate the web app's content. However, real-time updates between the two apps proved to be a challenge. While I couldn't get it working perfectly for this version, I plan to revisit and improve it to achieve full real-time synchronization.

Lessons Learned

This project was both challenging and rewarding. I gained valuable experience in:

Building web dashboards with complex canvas interactions using Fabric.js.
Creating a cross-platform desktop app with Electron.
Implementing offline-first features and handling content synchronization.
Using WebSockets for communication between a web app and a desktop app.
Deploying modern applications using Next.js and Vercel.

While I submitted the project, I’m excited to improve it further, add the missing features, and optimize the real-time capabilities.

Check Out the Project

If you're curious, here’s where you can see the project live or contribute to its development:

Deployed Web App: https://signage-content-web-app.vercel.app/
GitHub Repository: https://github.com/HTSagara/signage-content-app

I’m making this project open-source and would love for developers to collaborate, explore, and enhance it. Whether you're interested in improving the real-time features, optimizing the Electron app, or adding new functionality, your contributions are welcome!

Final Thoughts

This project challenged me to step out of my comfort zone and explore new tools, frameworks, and design approaches. It reminded me that development is an iterative process filled with learning opportunities. I look forward to continuing this project, improving it over time, and sharing updates along the way. 🚀

Open-Source Collaboration Progress

Henrique Sagara — Fri, 06 Dec 2024 19:16:46 +0000

Overview

Recently, I faced an interesting challenge while working on a project integrating Slack Bolt with Sanic - a framework I was previously unfamiliar with, which led to some unexpected deprecation warnings and type-related issues. I'll walk you through how I tackled the issue, the lessons I learned, and the precise code changes that resolved the problem.

What Are Sanic and Slack Bolt?

Sanic

Sanic is a high-performance, asynchronous web framework in Python. Designed to be fast, it takes advantage of Python's asyncio capabilities to handle large volumes of requests efficiently. Its minimalistic design makes it suitable for lightweight web applications, microservices, and API layers.

Slack Bolt

Slack Bolt is a framework for building Slack apps. It abstracts the complexities of Slack's APIs, allowing developers to focus on creating interactive and event-driven Slack applications. With Bolt, you can manage commands, shortcuts, events, and more with ease.

The Challenge

While implementing the integration, I encountered several warnings related to Sanic's cookie handling when running tests and handling requests. Here's an example of the warnings I saw:

DeprecationWarning: [DEPRECATION] Setting cookie values using the dict pattern has been deprecated.
DeprecationWarning: [DEPRECATION] Accessing cookies from the CookieJar by dict key is deprecated.
TypeError: Argument "path" to "add_cookie" of "BaseHTTPResponse" has incompatible type "Optional[Any]"; expected "str"

The root cause was the use of Sanic's old dict-based cookie handling syntax, which is no longer recommended as of Sanic v23.3. Instead, the new add_cookie method must be used to ensure compatibility and eliminate these warnings.

The Solution

The key change was replacing the dict-based cookie handling with the add_cookie method, ensuring that all cookie parameters passed were of the correct type.

Here’s the updated code snippet:

# Iterate over cookies and add them using Sanic's add_cookie method
for cookie in bolt_resp.cookies():
    for key, c in cookie.items():
        # Convert "expires" field if provided
        expire_value = c.get("expires")
        expires = datetime.strptime(expire_value, "%a, %d %b %Y %H:%M:%S %Z") if expire_value else None

        # Convert "max-age" if provided
        max_age = int(c["max-age"]) if c.get("max-age") else None

        # Ensure values are of the correct type before passing to add_cookie
        path = str(c.get("path")) if c.get("path") else "/"
        domain = str(c.get("domain")) if c.get("domain") else None

        # Add cookie with Sanic's add_cookie method
        resp.add_cookie(
            key=key,
            value=c.value,
            expires=expires,
            path=path,
            domain=domain,
            max_age=max_age,
            secure=True,
            httponly=True,
        )

Replaced Dict-Based Syntax: The old approach relied on direct manipulation of resp.cookies using dict syntax, which is deprecated. Instead, we used resp.add_cookie() to set cookies in a forward-compatible way.

Ensured Proper Data Types: Parameters like path and domain were sometimes None or not strings. We explicitly converted these values to strings or set defaults ("/" for path, None for domain) before passing them to add_cookie.

Handled Optional Cookie Fields: expires was parsed into a datetime object if provided, using the format "%a, %d %b %Y %H:%M:%S %Z".
max-age was converted to an integer if available.

These changes resolved all warnings and errors, ensuring the integration adhered to Sanic's modern practices.

Final Thoughts

Since I had no prior experience with Sanic, understanding its documentation was critical. Learning how Sanic handles cookies and requests helped me realize why the old syntax was problematic and how the new add_cookie method works.

Integrating Slack Bolt with Sanic turned out to be a rewarding challenge. Not only did it improve my understanding of Sanic, but it also emphasized the importance of staying up-to-date with framework best practices. If you're facing similar issues, I hope this blog post provides clarity and helps you solve your problem more efficiently.

Continue contributing to Open-Source projects

Henrique Sagara — Mon, 25 Nov 2024 14:55:08 +0000

If you read my blogs, you may know that I've been trying to gain experience by contributing to Open-Source projects.

Lately, I've been contributing to python slack-sdk. If you haven't seen my blog posts yet, you can check them out here:

Updates about slack-sdk issue

Just an update about this issue: my latest PR hasn't been merged yet. One of the maintainers wrote additional test cases to ensure that the URL-related changes don't introduce any failures. I'm currently waiting for their tests to be merged so I can re-test the code and make any necessary updates.

This has been a valuable learning experience, as it has shown me how collaborative problem-solving works in large-scale projects. The additional test cases highlight the importance of robust testing practices in Open-Source development, ensuring the stability and reliability of the codebase.

Next Steps

I'm planning to continue contributing to Slack, whether in the same project or others. My immediate priority is to get that previous PR merged. In the meantime, I'm exploring other issues to work on.

I recently discovered a large and exciting project from Slack called Nebula. For those unfamiliar, Nebula is Slack’s open-source secure global overlay network, designed to simplify connectivity between devices, even across different networks. It’s widely recognized for its reliability and innovation in creating seamless connectivity. Nebula enables devices to communicate securely as if they were on the same physical network, regardless of their actual location. However, this project requires a solid understanding of Golang, so I'll need to invest time in learning it before tackling any contributions there.

To prepare, I've started exploring Golang tutorials and small projects to build my confidence and understanding of the language. If you have any recommendations for learning Golang or working on similar projects, feel free to share them with me!

I've also come across other issues involving Python and TypeScript that I can begin exploring. These issues provide a great opportunity to strengthen my skills and continue building my portfolio of contributions.

Additionally, I'm open to contributing to Open-Source projects from other companies as well. I believe diversifying my contributions will help me grow as a developer and expose me to new tools and methodologies.

I'll continue to share my experiences, lessons learned, and progress along the way.

ReadmeGenie is Ready for You! Automating Releases with GitHub Actions

Henrique Sagara — Fri, 22 Nov 2024 21:19:35 +0000

I'm thrilled to announce the first release of ReadmeGenie! 🎉 This project has been incredibly rewarding to see it come to life on PyPI. With version 1.0.0, ReadmeGenie is now available for developers everywhere to generate clean, concise, and professional README.md files for their projects effortlessly.

What is ReadmeGenie?

ReadmeGenie is a Python-based CLI tool designed to simplify the creation of README.md files. By automating the tedious process of writing detailed documentation, ReadmeGenie helps developers focus more on coding and less on formatting.

With ReadmeGenie, you can:

Generate structured README.md files with sections like Installation, Usage, and Contributing.
Incorporate API integration to tailor your README based on your project type.
Quickly update your README files as your project evolves.

ReadmeGenie is a game-changer for developers working on open-source projects or collaborative repositories. If you want to give it a try, you can install it with the following command:

pip install -i https://test.pypi.org/simple/ ReadmeGenie==1.0.0

Check out the GitHub Repository to learn more, explore the codebase, or contribute to the project.

Automating Releases with GitHub Actions

One of the highlights of ReadmeGenie’s journey is how we’ve automated its deployment process to PyPI using GitHub Actions. Every time a new version tag is pushed to the repository, our automation pipeline builds the package, runs tests, and publishes it to PyPI. Here’s how we achieved this:

1. Version Management with Git Tags

We integrated setuptools_scm to fetch the project version directly from Git tags. This ensures that every release is versioned correctly without needing manual updates to the pyproject.toml file. By tagging a release (e.g., v1.0.0), the pipeline automatically sets the version dynamically.

2. Automated Workflows with GitHub Actions

Our GitHub Actions workflow includes the following steps:

Testing and Linting:
- Every push triggers tests using pytest and code linting with flake8.
- This ensures the project remains robust and adheres to Python best practices.
Building the Package:
- The pipeline builds the distribution files (sdist and wheel) using setuptools.
Publishing to PyPI:
- With the help of twine, the built packages are uploaded to PyPI or TestPyPI, depending on the environment.

Here’s a snippet from our GitHub Actions workflow:

name: Publish to PyPI

on:
  push:
    tags:
      - 'v*'  # Trigger workflow for version tags

jobs:
  build-and-publish:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4
      - name: Set up Python
        uses: actions/setup-python@v3
        with:
          python-version: "3.12"
      - name: Install dependencies
        run: |
          python3 -m pip install --upgrade pip build setuptools_scm twine
      - name: Build package
        run: python3 -m build
      - name: Publish package to PyPI
        env:
          TWINE_USERNAME: __token__
          TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }}
        run: python3 -m twine upload dist/*

3. Secrets Management

To ensure security, the PyPI API token is stored as a GitHub secret (PYPI_API_TOKEN) and injected into the workflow at runtime. This eliminates the need to expose sensitive information in the codebase.

What’s Next for ReadmeGenie?

This is just the beginning! 🚀 For future releases, we plan to:

Add support for more customizable templates.
Integrate advanced NLP tools to generate contextual README sections.
Suppor more GenAI tools other than Groq and Cohere.

We’re also looking forward to collaborating with the community to make ReadmeGenie even better. Feel free to contribute or report issues on our GitHub repository.

A Final Word

The journey to deploying ReadmeGenie wasn’t without its challenges, but automating the release process with GitHub Actions has been a game-changer. It ensures that every release is seamless, consistent, and reliable.

If you’re a developer who finds writing documentation tedious or repetitive, give ReadmeGenie a try. We can’t wait to see the amazing projects you’ll create with it!

Happy coding! 😊

Collaborating to Slack as an Open-Source Developer: Part 2

Henrique Sagara — Wed, 20 Nov 2024 19:32:21 +0000

Recap of Part 1

In my first blog post, I shared my journey of contributing to the Slack SDK as an open-source developer. I tackled an issue related to ensuring the base URL for API requests had a trailing slash to simplify URL construction and prevent inconsistencies. If you haven’t read it yet, I recommend starting there to get the context for this follow-up.

A New Challenge Arises

After completing my first contribution, I was eager to tackle another issue in the same project. As I prepared to start, I noticed a problem in one of the authentication tests. The issue stemmed from the trailing slash feature I had previously implemented.

Here’s what happened: the base_url now always had a trailing slash appended during initialization. However, the api_method used in some test cases also started with a /. This combination caused a double slash (e.g., https://slack.com/api//auth.test), which broke some of the API requests.

Reporting the Issue

Realizing the significance of this bug, I quickly reported it to the maintainers and opened a new issue describing the problem. To ensure transparency and provide a clear solution path, I also submitted a pull request addressing the bug. However, the maintainers decided to revert my original merge to prevent disruptions in the main branch and asked me to submit a new PR with the necessary fixes and tests for edge cases.

The Fix and New Implementation

To address the problem, I reworked the _get_url function and added additional safeguards to prevent double slashes, even when both base_url and api_method contained trailing or leading slashes.

Here’s the updated implementation:

def _get_url(base_url: str, api_method: str) -> str:
    """Joins the base Slack URL and an API method to form an absolute URL.

    Args:
        base_url (str): The base URL (always ends with '/').
        api_method (str): The Slack Web API method. e.g., 'chat.postMessage'.

    Returns:
        str: The absolute API URL, e.g., 'https://slack.com/api/chat.postMessage'.
    """
    # Strip leading slash from api_method to prevent double slashes
    api_method = api_method.lstrip("/")
    return urljoin(base_url, api_method)

Key adjustments

Strip Leading Slash: By using .lstrip("/") on api_method, the function ensures no double slashes occur during concatenation.
Test Case Enhancements: I expanded the test suite to cover scenarios such as:
- base_url with and without a trailing slash.
- api_method with and without a leading slash. Edge cases where both had slashes.

Here’s an example of the updated test:

def test_get_url_prevent_double_slash(self):
    api_url = _get_url("https://slack.com/api/", "/auth.test")
    self.assertEqual(api_url, "https://slack.com/api/auth.test", "Should prevent double slashes")

    api_url = _get_url("https://slack.com/api", "auth.test")
    self.assertEqual(api_url, "https://slack.com/api/auth.test", "Should handle base_url without trailing slash")

    api_url = _get_url("https://slack.com/api/", "auth.test")
    self.assertEqual(api_url, "https://slack.com/api/auth.test", "Should handle api_method without leading slash")

    api_url = _get_url("https://slack.com/api", "/auth.test")
    self.assertEqual(api_url, "https://slack.com/api/auth.test", "Should handle both inputs cleanly")

Reflections on Testing and Edge Cases

This experience taught me the importance of thorough testing. Even though my original implementation passed all existing tests, it didn’t account for certain edge cases, such as leading slashes in api_method.

Here are my key takeaways:

1. Unit Tests Aren’t Foolproof: While unit tests help catch many issues, they might not cover all edge cases. A feature can still have loose ends, especially if the inputs vary widely.
2. Collaborate and Communicate: Reporting bugs promptly and discussing solutions with maintainers can prevent larger disruptions. Their decision to revert my changes emphasized the importance of keeping the main branch stable.
3. Iterate and Learn: Open-source contributions are iterative. Each step is an opportunity to improve, learn from feedback, and strengthen your coding practices.

Final Thoughts

Contributing to Slack’s SDK has been an invaluable experience. This journey, from implementing a new feature to resolving its unintended side effects, highlighted the complexities of real-world software development and the collaborative spirit of open source.

If you’re considering contributing to an open-source project, don’t let the fear of making mistakes hold you back. Every bug, every fix, and every test written is a step toward becoming a better developer.

What challenges have you faced in your open-source contributions? Let’s discuss in the comments below!

Collaborating to Slack as an Open-Source Developer

Henrique Sagara — Tue, 19 Nov 2024 14:26:54 +0000

Introduction

Are you a beginner developer looking to gain more experience and wondering where to start? This blog post is for you!

One of the best ways to build your skills is by contributing to open-source projects. Recently, I tackled an issue in the SlackAPI GitHub Repo. At the time of writing, my pull request (PR) has been accepted by one of the maintainers, it has passed all the CI builds and just got merged!🙌

In this post, I’ll share my journey fixing this issue, the challenges I faced, and the lessons I learned along the way.

Issue

The issue I worked on was about handling URLs in the Slack API. Specifically, the code was not appending a trailing slash (/) to the base_url if it was missing, which could lead to inconsistent or failed API calls.

For example:

Without the trailing slash: https://slack.com/api
With the trailing slash: https://slack.com/api/

The missing trailing slash caused subtle issues that needed to be addressed to ensure reliability. My goal was to modify the code to automatically append the slash if it wasn’t provided.

You can find the original issue here: #1541.

Challenges

Finding the Right Files to Edit

As a beginner tackling a large codebase, my first challenge was finding where to make the changes. Navigating through unfamiliar directories and understanding the project’s structure took time. Reading the documentation and stepping through the code helped me locate the relevant files.

Writing the Fix

Once I identified the file (base_client.py), I drafted a solution to ensure the base_url ended with a /. However, before implementing it, I reached out to the maintainers for guidance. They pointed out that similar logic should also be applied to two other files: async_base_client.py and legacy_base_client.py. This was a great reminder of the importance of clear communication and pre-discussion in collaborative projects.

Here’s the updated code snippet for appending the trailing slash:

if not base_url.endswith("/"):
    base_url += "/"

This simple change ensured that all API calls would use a consistent base URL. I replicated this logic in the other two files as well, maintaining consistency across the different clients.

Adding Tests

Testing was a critical part of this fix. I added new tests in the test_web_client.py file to verify that the base_url was corrected if it was missing the trailing slash. For example:

Input: http://localhost:8888
Expected output: http://localhost:8888/

Here’s a simplified version of one of the tests I added:

    def test_base_url_appends_trailing_slash_issue_15141(self):
        client = self.client
        self.assertEqual(client.base_url, "http://localhost:8888/")

    def test_base_url_preserves_trailing_slash_issue_15141(self):
        client = WebClient(base_url="http://localhost:8888/")
        self.assertEqual(client.base_url, "http://localhost:8888/")

Takeaways

Importance of Testing

Before submitting your PR, test your changes thoroughly. Writing unit tests, running existing tests, and using debugging tools are crucial steps to ensure your code works as expected. By automating tests for edge cases, you can catch potential errors early and increase confidence in your solution.

Follow the Contribution Guidelines

Every open-source project has its own contribution guidelines. Make sure to read and follow them carefully, as they outline coding standards, testing requirements, and PR submission processes. For example, SlackAPI provided clear instructions on setting up the development environment and running tests.

Communication Is Key

Pre-discussing your changes with maintainers can save time and prevent unnecessary back-and-forth. In my case, it ensured I applied the fix to all relevant files, making the solution more comprehensive.
Don’t Be Afraid to Ask for Help

If you’re stuck, don’t hesitate to ask questions. Open-source maintainers and contributors are often supportive and willing to help newcomers.

Final Thoughts

Contributing to open source can feel intimidating at first, but it’s one of the best ways to grow as a developer. You’ll learn to work in a team, navigate large codebases, and write production-ready code. Plus, it feels amazing to see your work being used by others!

If you’re a beginner developer, I encourage you to explore open-source projects, find an issue that interests you, and take the plunge. The SlackAPI GitHub Repo is a great place to start!

Have you contributed to an open-source project? Share your experience in the comments below! 🚀

Implementing CI/CD for ReadmeGenie

Henrique Sagara — Mon, 11 Nov 2024 23:30:23 +0000

Why CI/CD?

Before we dive into the setup, let’s briefly cover why CI/CD is so critical:

Automated Testing: Running tests automatically ensures code is stable with every change.
Consistency: CI/CD enforces standards (linting, formatting) across the codebase.
Reliability: Automated checks and tests minimize human error and improve code reliability.
Rapid Feedback: Developers receive immediate feedback on code quality, allowing issues to be caught early.

In ReadmeGenie, we leveraged GitHub Actions as our CI/CD tool. It integrates smoothly with GitHub repositories and offers flexibility and automation through YAML configuration files.

The CI/CD Pipeline for ReadmeGenie

Our CI/CD pipeline includes the following automated steps:

Linting and Formatting Checks: We run Ruff and Black to ensure code style and consistency.
Unit Testing: We use unittest to verify that code changes don’t break existing functionality.
Coverage Analysis: We use Coverage.py to ensure that the code meets our coverage threshold before a commit is allowed.
Pre-Commit Hooks: We added hooks to enforce local quality checks before pushing changes.

Overview of the GitHub Actions Workflow

The CI workflow is defined in .github/workflows/python-app.yml. Here’s a breakdown of what each part of the workflow does:

1. Triggering the Workflow

The workflow runs on every push and pull request to the main branch. This ensures that all code changes undergo validation before merging into production.

name: Python application

on:
  push:
    branches: ["main"]
  pull_request:
    branches: ["main"]

2. Setting Up the Python Environment

We configure GitHub Actions to use Python 3.12.x, ensuring consistency with our local development environment. This step installs the specific Python version and prepares the environment for dependency installation.

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4
      - name: Set up Python 3.12.x
        uses: actions/setup-python@v3
        with:
          python-version: "3.12.x"

3. Installing Dependencies

The next step is to install project dependencies. Here, we upgrade pip and install requirements.txt file, it will install additional dependencies specified there.

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install flake8 pytest
          if [ -f requirements.txt ]; then pip install -r requirements.txt; fi

4. Running Linting and Code Quality Checks

Linting is a crucial part of our workflow, ensuring the code adheres to specified quality standards. We run flake8 with options to flag syntax errors, undefined names, and complexity issues.

      - name: Lint with flake8
        run: |
          # stop the build if there are Python syntax errors or undefined names
          flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
          # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
          flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics

5. Running Tests with Coverage Analysis

For unit tests, we use pytest to run all test cases. Additionally, we use coverage to track which lines of code are tested, ensuring that our test suite meets the defined coverage threshold of 75%.

The following commands run the tests and generate a coverage report, highlighting any gaps in test coverage. This is essential for quality assurance, as untested code is a potential source of future bugs.

      - name: Test with pytest
        run: |
          pytest
      - name: Test functions and coverage
        run: |
          coverage run --source=. -m unittest discover -s tests
          coverage report -m --fail-under=75

This coverage check ensures a high standard of code quality by enforcing that at least 75% of the codebase is covered by tests. If coverage falls below this threshold, the commit will not be allowed.

Integrating Pre-Commit Hooks

In addition to CI/CD, we set up pre-commit hooks to enforce code quality locally before any changes are pushed to the repository. These hooks:

Run Ruff for linting and Black for formatting.
Enforce a minimum coverage threshold by running the tests with coverage locally.

Here’s how we added the coverage check as a pre-commit hook in .pre-commit-config.yaml:

repos:
  - repo: local
    hooks:
      - id: check-coverage
        name: Check Coverage
        entry: bash -c "coverage run --source=. -m unittest discover -s tests && coverage report -m --fail-under=75"
        language: system

Challenges and Lessons Learned

Setting up CI/CD required a deep understanding of how different tools (flake8, pytest, coverage) interact within GitHub Actions. Here are some challenges we faced and the solutions we implemented:

Handling Divergent Local and Remote Configurations

We encountered issues with environment variable conflicts, especially in testing API integration and configuration handling. Using @patch.dict and other mocking techniques in unittest allowed us to simulate the environment effectively.

Testing Coverage and Thresholds

The biggest challenge was ensuring adequate test coverage. Using coverage.py with --fail-under=75 in both GitHub Actions and pre-commit hooks helped enforce this standard.

Future Improvements

To make the CI/CD pipeline more robust, we plan to:

Add Deployment Stages: Automate deployment to a staging or production environment after tests pass.
Automate Code Quality Badges: Add dynamic badges to display coverage, linting status, and test results in the README.
Expand Coverage Requirements: Increase the coverage threshold as we improve tests and cover more edge cases.

Takeaway

Through this project, I realized the importance of establishing robust testing and CI/CD practices early on. If I were to start again, I’d focus on writing comprehensive tests from the beginning and incrementally expanding and improving them as the project progresses. This approach would prevent missing branches or untested areas and ensure that all new code integrates smoothly into a well-covered codebase.

Implementing Unit Testing in ReadmeGenie

Henrique Sagara — Thu, 07 Nov 2024 19:23:57 +0000

In this post, I’ll walk through the journey of implementing unit testing, handling complex configuration challenges, and introducing robust code coverage in ReadmeGenie. From initial test design to setting up pre-commit hooks, this process involved a range of improvements in code quality, reliability, and developer workflow.

1. Setting Up the Testing Environment

To start, I chose unittest as the primary framework for writing and executing tests. Python’s built-in unittest provides a structured approach for defining test cases, and its integration with mock makes it ideal for testing complex configurations and API calls.

I created a dedicated test runner (tests/test_runner.py) for automatic discovery and execution of all test files in the tests/ directory:

# tests/test_runner.py
import unittest

if __name__ == "__main__":
    loader = unittest.TestLoader()
    suite = loader.discover(start_dir="tests", pattern="test_*.py")
    runner = unittest.TextTestRunner(verbosity=2)
    runner.run(suite)

This setup ensures that running python tests/test_runner.py will automatically load and run all test files, making it easy to validate the project’s overall functionality.

2. Structuring the Unit Tests

The ReadmeGenie project required comprehensive testing for several components:

Argument Parsing: Verifying correct parsing of command-line arguments and handling of default values.
Configuration and Environment Handling: Testing for proper retrieval of API keys and handling errors when they’re missing.
API Calls: Using mocks to simulate API requests to avoid real API calls in tests.
Helper Functions: Testing utility functions, such as file reading and README processing.

Each test file is named according to the module it tests (e.g., test_parse_arg.py for argument parsing and test_model.py for model functions), ensuring a clear, maintainable structure.

3. Biggest Challenge: Configuring `test_loadConfig.py`

Setting up test_loadConfig.py turned out to be the most challenging part of this project. Initially, I encountered persistent issues related to environment variables and file path checks. Since load_config() is intended to handle various configuration sources (e.g., environment variables, .env files, JSON, and TOML files), the tests required extensive mocking to simulate these environments accurately.

Errors and Solutions in `test_loadConfig.py`

The primary issues involved:

Environment Variable Conflicts: Existing environment variables sometimes interfered with mocked values. Using @patch.dict("os.environ", {}, clear=True), I cleared the environment variables within the test scope to ensure consistent results.
File Path Checks: Since load_config() checks for file existence, I used os.path.exists to simulate scenarios where configuration files were present or absent.
Mocking open and toml.load: These required precise mocking to handle cases of missing, empty, or populated configuration files. Using mock_open with patch on toml.load, I effectively simulated each situation.

After resolving these issues, test_loadConfig.py now covers three main scenarios:

Empty Configuration: Tests that an empty configuration is returned when no environment variables or files are found.
Valid Configuration Data: Tests that the api_key is correctly retrieved from the configuration file.
File Not Found: Simulates a missing file, expecting an empty configuration to be returned.

Here’s the final version of test_loadConfig.py:

import unittest
from unittest.mock import mock_open, patch
from loadConfig import load_config

class TestLoadConfig(unittest.TestCase):
    @patch.dict("os.environ", {}, clear=True)
    @patch("loadConfig.os.getenv", side_effect=lambda key, default=None: default)
    @patch("loadConfig.os.path.exists", return_value=False)
    @patch("builtins.open", new_callable=mock_open, read_data="{}")
    @patch("loadConfig.toml.load", return_value={})
    def test_load_config_empty_file(self, mock_toml_load, mock_open_file, mock_exists, mock_getenv):
        config = load_config()
        self.assertEqual(config, {})

    @patch.dict("os.environ", {}, clear=True)
    @patch("loadConfig.os.getenv", side_effect=lambda key, default=None: default)
    @patch("loadConfig.os.path.exists", return_value=True)
    @patch("builtins.open", new_callable=mock_open, read_data='{"api_key": "test_key"}')
    @patch("loadConfig.toml.load", return_value={"api_key": "test_key"})
    def test_load_config_with_valid_data(self, mock_toml_load, mock_open_file, mock_exists, mock_getenv):
        config = load_config()
        self.assertEqual(config.get("api_key"), "test_key")

    @patch.dict("os.environ", {}, clear=True)
    @patch("loadConfig.os.getenv", side_effect=lambda key, default=None: default)
    @patch("loadConfig.os.path.exists", return_value=False)
    @patch("builtins.open", side_effect=FileNotFoundError)
    @patch("loadConfig.toml.load", return_value={})
    def test_load_config_file_not_found(self, mock_toml_load, mock_open_file, mock_exists, mock_getenv):
        config = load_config()
        self.assertEqual(config, {})

4. Code Coverage Analysis

With our tests in place, we focused on measuring and improving coverage using coverage.py. By setting an 80% threshold, we aimed to ensure all critical parts of the code are tested.

Tool Configuration for Coverage

I configured coverage.py with the following settings in pyproject.toml:

[tool.coverage.run]
source = [""]
branch = true
omit = ["tests/*"]

[tool.coverage.report]
show_missing = true
fail_under = 75

This configuration includes branch coverage, highlights missing lines, and enforces a minimum 75% coverage threshold.

Pre-Commit Coverage Check

To integrate this into the development workflow, I added a pre-commit hook to ensure code coverage is checked on each commit. If coverage falls below 75%, the commit is blocked, prompting developers to improve coverage before proceeding:

- repo: local
  hooks:
    - id: check-coverage
      name: Check Coverage
      entry: bash -c "coverage run --source=. -m unittest discover -s tests && coverage report -m --fail-under=75"
      language: system

5. Current Coverage and Opportunities for Improvement

Our recent coverage report shows:

Name                   Stmts   Miss Branch BrPart  Cover   Missing
------------------------------------------------------------------
loadConfig.py              8      0      2      0   100%
logging_config.py         19      0      2      0   100%
models/__init__.py         0      0      0      0   100%
models/cohere_api.py      12      0      0      0   100%
models/groq_api.py        12      0      0      0   100%
models/model.py           64     13     16      3    78%   25-26, 44-46, 70-74, 84-87
readme_genie.py           37     16      4      1    54%   20-24, 66-86, 90
------------------------------------------------------------------
TOTAL                    152     29     24      4    79%

While coverage is strong in some areas (e.g., loadConfig.py at 100%), there are still opportunities for improvement in models/model.py and readme_genie.py. Focusing on untested branches and edge cases will be crucial to reaching our goal of 85% or higher overall coverage.

Final Thoughts

This project has taught me a lot about unit testing, mocking, and code coverage. Setting up test_loadConfig.py was a particularly valuable experience, pushing me to explore deeper levels of configuration mocking. The pre-commit hook for coverage has added a layer of quality assurance, enforcing consistent test standards.

Moving forward, I aim to refine these tests further by adding edge cases and improving branch coverage. This will not only make ReadmeGenie more robust but also lay a solid foundation for future development.

Setting Up Tools for Code Quality

Henrique Sagara — Tue, 29 Oct 2024 20:34:53 +0000

When developing ReadmeGenie, I aimed to ensure consistent code quality with an automated setup for linting and formatting. After considering several tools, I selected Ruff as the linter and Black as the code formatter. Although Ruff can also handle both linting and formatting, I decided to set up Black as a separate formatter to gain experience with the configuration of both tools. Below, I’ll share why I chose these tools, how I configured them for my project, the challenges I faced, and the lessons I learned along the way.

1. Tool Selection

Why Ruff?

Ruff is a fast linter for Python that supports various linting rules from other linters (like Flake8 and Pyflakes) and offers significant performance improvements. It’s highly customizable, which allowed me to specify a mix of rules while ensuring compatibility with Black for formatting. Ruff’s design for speed and extensibility is ideal for projects that prioritize efficiency without sacrificing quality.

Ruff Documentation: https://github.com/charliermarsh/ruff

Why Black?

Black is a Python formatter that strictly enforces one formatting style, helping reduce discussions and inconsistencies over code styling. While Ruff offers basic formatting capabilities, Black’s dedicated approach provides a few advantages:

Consistency: Black enforces a strict, standard style that minimizes debates over code formatting.
Broad Adoption: Black is widely used, making it easier to integrate into most development workflows, especially in collaborative projects.
Black Documentation: https://github.com/psf/black

2. Project Setup

To ensure that Ruff and Black worked seamlessly in ReadmeGenie, I configured them in both pyproject.toml and
.pre-commit-config.yaml, allowing developers to automatically format and lint code when making commits.

Configuration for Ruff and Black in `pyproject.toml`

This setup ensures Ruff is used solely for linting and Black for formatting:

# pyproject.toml
# Set up black as formatter
[tool.black]
line-length = 88
target-version = ["py311"]

# Set up ruff as linter only
[tool.ruff]
# Exclude directories that don’t need linting (e.g., virtual environments)
exclude = [
    "venv/",
    "__pycache__/"
]
fix = true


# Enable specific linting rules
select = ["F", "E", "W", "B", "I", "S"]  # Example codes: F=flake8, E=errors, W=warnings, B=bugbear, I=import, S=safety
# Exclude Black-compatible rules to avoid conflicts with Black's formatting.
ignore = ["E501", "E203", "E231"]  # Exclude Black-incompatible style issues

ignore: Black handles specific styling, so we excluded these rules in Ruff.
fix: Enables Ruff to fix issues where possible, leaving formatting to Black.

Adding Pre-commit Hook for Ruff and Black

Using pre-commit hooks, I configured .pre-commit-config.yaml to enforce linting and formatting on every commit:

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/psf/black
    rev: 23.1.0
    hooks:
      - id: black
  - repo: https://github.com/charliermarsh/ruff-pre-commit
    rev: v0.7.1
    hooks:
      - id: ruff

3. Running Ruff and Black from the Command Line

With the above setup, you can use the following commands:

Run Ruff:

  ruff check . --fix

Run Black:

  black .

These commands apply fixes to all Python files, ensuring consistent styling and quality checks.

4. VS Code Integration

To automate Ruff and Black on save, I added the following configuration in .vscode/settings.json:

{
  // Enable Black as the default formatter
  "python.formatting.provider": "black",
  "editor.formatOnSave": true,

  // Set up Ruff as the linter
  "python.linting.enabled": true,
  "python.linting.ruffEnabled": true,

  // Ensure only Ruff is used as the linter
  "python.linting.flake8Enabled": false,
  "python.linting.pylintEnabled": false,

  // Format and lint on save
  "editor.codeActionsOnSave": {
    "source.fixAll": true,
    "source.organizeImports": true
  }
}

This setup makes Black the default formatter and Ruff the only active linter in VS Code, allowing both to run
automatically upon saving.

5. Findings and Fixes

Once configured, Ruff and Black identified several issues:

Line Length (E501): Ruff initially flagged long lines, which Black auto-formatted.
Unused Imports and Variables: Ruff caught several unused imports and variables.
Indentation and Styling Consistency: Black applied consistent spacing and indentation, enhancing readability.

6. Challenges

One notable challenge was understanding that some styles are incompatible between Ruff and Black. For example:

Line Length (E501): Ruff initially flagged long lines exceeding 88 characters, which Black handles by wrapping lines. To prevent conflicts, I added E501 to Ruff’s ignore list. Despite this, Ruff sometimes flagged E501 errors if Black didn’t apply the expected breakpoints. These discrepancies underscored the importance of adjusting each tool’s configuration and understanding where they may overlap.

7. Lessons Learned

Using Ruff and Black together has been a great way to improve code quality. Here’s what I learned:

Consistency: Black’s opinionated style reduces ambiguity in code styling.
Automation: Pre-commit hooks save time and ensure consistent formatting.
Editor Integration: Configuring Ruff and Black to run on save within VS Code streamlined development.
Compatibility: Learning how to resolve conflicts like E501 taught me about tool configurations and helped fine-tune project workflows.

Hacktoberfest week 3- Online Auction System

Henrique Sagara — Mon, 21 Oct 2024 00:13:30 +0000

Overview

During Week 3 of Hacktoberfest, I decided to contribute to a smaller but promising project: an Online Auction System. Though the project is still in its early stages, it already shows potential for growth, and I saw an opportunity to help improve its codebase. My task was to refactor the project by reducing redundant code and improving overall structure, making it more maintainable and scalable.

My task

The core objective of my task was to identify repetitive code patterns and refactor them to make the codebase cleaner and easier to maintain. Code redundancy can lead to inconsistencies, make future updates harder, and increase the likelihood of bugs. By streamlining the code, I aimed to improve efficiency without altering functionality.

Key Changes:

1. Centralized Asynchronous Error Handling: One of the common issues in Express.js projects is repetitive try-catch blocks for handling errors in asynchronous routes. To address this, I introduced a utility function called asyncHandler that wraps all async functions, ensuring that any errors are automatically caught and handled consistently across the app.

// utils/asyncHandler.js
const asyncHandler = (fn, errorMessage = "Internal server error") => (req, res, next) =>
  Promise.resolve(fn(req, res, next)).catch((error) => {
    console.error(`Error: ${error.message}`);
    res.status(500).json({ error: errorMessage });
  });

module.exports = asyncHandler;

This change not only removed redundant try-catch blocks but also made the code cleaner and more focused on the business logic, rather than error handling.

2. Database Utility Functions: The project had several routes that directly interacted with the database, often repeating similar queries. To reduce redundancy, I created a set of reusable utility functions to handle common database queries, such as fetching a buyer's information or retrieving transactions.

// utils/dbUtils.js
const getBuyerById = async (buyerId) => {
  const query = "SELECT * FROM Buyers WHERE Buyer_ID = $1";
  const { rows } = await pool.query(query, [buyerId]);
  return rows.length > 0 ? rows[0] : null;
};

const getTransactionsByBuyerId = async (buyerId) => {
  const query = "SELECT * FROM Transactions WHERE Buyer_ID = $1";
  const { rows } = await pool.query(query, [buyerId]);
  return rows;
};

module.exports = { getBuyerById, getTransactionsByBuyerId };

Now, instead of writing SQL queries directly in each route, the codebase can rely on these utility functions, reducing duplication and making the logic easier to manage.

3. Unified Response Handling: Another area of improvement was response handling. Different routes had inconsistent patterns for sending success and error responses. To standardize this, I introduced utility functions like send404, send500, and sendSuccess.

Discussion

Since the project is still in its early stages, I noticed some missing routes like POST, DELETE, and PUT that I could have easily create. However, I decided to stick with only my Issue was asking.

Let me know in the comments, how do you usually deal with this kind of situations that you could contribute more...Do you open a new issue, do you just create it and do a Pull Request? I would love to read you experience.

Hacktoberfest week 2- Shardeum Validator Dashboard

Henrique Sagara — Fri, 11 Oct 2024 21:02:44 +0000

Introduction

For the Hacktober week 2 I choose to work on an issue from Shardeum on their Validator Dashboard. Shardeum is a highly scalable, EVM-compatible blockchain that aims to offer low-cost, decentralized, and secure services. Its architecture is built on the principle of dynamic state sharding, where the network automatically expands by adding more validator nodes as the demand for resources increases, allowing Shardeum to maintain high throughput and decentralization.

What is the Shardeum Validator Dashboard?

The Shardeum Validator Dashboard is a tool that allows node operators to monitor and manage their validator nodes. These validator nodes play a key role in maintaining the network's consensus and processing transactions securely. The dashboard provides an interface for operators to check the health of their nodes, configure settings, and ensure they are functioning correctly in the Shardeum network.

The Issue

The Validator Dashboard relies on a number of environment variables for configuration, such as port numbers, IP addresses, and node settings. However, there was no comprehensive validation in place to ensure that these environment variables were correctly set before launching the dashboard. This lack of validation led to silent failures or misconfigurations, which could cause the node to malfunction or behave unexpectedly. For example:

Invalid port numbers or IP addresses could prevent the node from communicating properly.
A weak dashboard password could expose the node to security risks.
Missing or incorrectly configured node parameters like minNodes or nodesPerConsensusGroup could lead to improper operation of the consensus mechanism.

The Solution

I created a reusable script check-variables.sh that contains functions for validating each type of environment variable. This script can be sourced by other scripts (like entrypoint.sh and installer.sh), making the validation process centralized and easier to maintain.

I hope it have a great impact in the Shardeum. I had a great time hacking in their repository. And their team was really nice providing help and information.

DEV Community: Henrique Sagara

Life After Graduation: A New Chapter

Post-Grad Life

After the Layoff

What’s Next

Building a Digital Signage Content Management System: A Challenge and a Journey

Building a Digital Signage Content Management System: A Challenge and a Journey

The Project Overview

Web Dashboard Features

Electron Display Application

Offline Capabilities

Tackling the Implementation

Learning Fabric.js and Electron from Scratch

The WebSocket Challenge

Lessons Learned

Check Out the Project

Final Thoughts

Open-Source Collaboration Progress

Overview

What Are Sanic and Slack Bolt?

Sanic

Slack Bolt

The Challenge

The Solution

Final Thoughts

Continue contributing to Open-Source projects

Updates about slack-sdk issue

Next Steps

ReadmeGenie is Ready for You! Automating Releases with GitHub Actions

What is ReadmeGenie?

Automating Releases with GitHub Actions

1. Version Management with Git Tags

2. Automated Workflows with GitHub Actions

3. Secrets Management

What’s Next for ReadmeGenie?

A Final Word

Collaborating to Slack as an Open-Source Developer: Part 2

Recap of Part 1

A New Challenge Arises

Reporting the Issue

The Fix and New Implementation

Key adjustments

Reflections on Testing and Edge Cases

Final Thoughts

Collaborating to Slack as an Open-Source Developer

Introduction

Issue

Challenges

Finding the Right Files to Edit

Writing the Fix

Adding Tests

Takeaways

Importance of Testing

Follow the Contribution Guidelines

Communication Is Key

Final Thoughts

Implementing CI/CD for ReadmeGenie

Why CI/CD?

The CI/CD Pipeline for ReadmeGenie

Overview of the GitHub Actions Workflow

1. Triggering the Workflow

2. Setting Up the Python Environment

3. Installing Dependencies

4. Running Linting and Code Quality Checks

5. Running Tests with Coverage Analysis

Integrating Pre-Commit Hooks

Challenges and Lessons Learned

Handling Divergent Local and Remote Configurations

Testing Coverage and Thresholds

Future Improvements

Takeaway

Implementing Unit Testing in ReadmeGenie

1. Setting Up the Testing Environment

2. Structuring the Unit Tests

3. Biggest Challenge: Configuring test_loadConfig.py

Errors and Solutions in test_loadConfig.py

4. Code Coverage Analysis

Tool Configuration for Coverage

Pre-Commit Coverage Check

5. Current Coverage and Opportunities for Improvement

3. Biggest Challenge: Configuring `test_loadConfig.py`

Errors and Solutions in `test_loadConfig.py`

Configuration for Ruff and Black in `pyproject.toml`