I worked on my third issue, No rate limiting for LLM API calls,
and created my third Pull Request (PR): Fix no rate limiting if llm api calls
After my first Go open-source project, I wanted my next contribution to involve a command-line tool (CLI). I found a CLI project that used an LLM, and I was interested because I had not realized before that a CLI tool could make use of a large language model (LLM). This sparked my curiosity, so I left a comment expressing my interest in fixing the bug.
The issue was that the CLI only implemented LLM usage but did not handle scenarios where many users might issue API requests in a very short time. This could cause crashes or provider rate-limit errors. I saw this as a great opportunity to join and solve the problem.
For this PR, I used the Go package "golang.org/x/time/rate," which provides a robust rate limiter to control how frequently certain actions can occur. It allowed me to regulate API calls and prevent exceeding the provider's rate limits by limiting the request frequency and handling bursts efficiently.
I set an API rate limit as a variable and improved a function so that when the CLI makes an API call, it obeys the set rate; if requests exceed the limit, it returns an error and an empty result. After updating the function, I wrote a unit test to make sure everything was working as expected. I was pleased to see it perform correctly.
Before opening my PR, I left a comment for the repository owner because I was not sure if the test file's location in the project structure would be acceptable, and there was no documentation about it. Once my unit test was approved, I submitted the PR and received a prompt response.
This experience made me feel ready to challenge myself with even bigger CLI projects in the future.
Top comments (0)