Recently, when using AI for code generation with tools like Claude Code's slash commands, I've found that automatically searching for similar examples after referencing documentation improves generation accuracy compared to just referencing documentation alone. In this article, I'll introduce this method.
Documentation-only generation often leaves room for improvement in subtle nuances
When generating code, it's well-known that referencing team-established rules such as coding conventions and database design guidelines before generation produces better accuracy than generating code from scratch. Given LLM characteristics, explicitly stating team rules rather than having no rules obviously generates something closer to those standards.
However, using documentation alone often left me feeling "it's mostly correct, but something feels off about the details." For example, the writing style might be slightly different from other parts of the project.
Automatically search for similar examples after referencing documentation
I believe the reason for getting "mostly correct but different" results is that documentation typically contains a summary of project rules and doesn't capture all the subtle nuances. Indeed, when we implement something ourselves, we often reference rule documentation first, then read through similar code in surrounding areas before making flexible decisions like "let's align with this writing style."
So I thought: what if we follow a similar flow by automatically searching for code similar to what we're about to generate after referencing documentation, and then generate code? I tried this approach, and while it's based on subjective experience, the accuracy improved.
For example, here's a slash command for generating database ALTER TABLE statements. The key point is not manually providing similar files yourself, but automatically searching for files similar to what you're about to create (step 3 in the procedure):
---
allowed-tools: Read, WebFetch, TodoWrite, Grep, WebSearch, Search
description: "Create database migration files"
---
You are a MySQL expert creating database migrations.
## Procedure
1. Understand the requirements written in $ARGUMENTS
2. Read the database design guideline @docs/db-guideline.md
3. Load 5 similar migration files from existing migration files
- Search for about 5 files in the `migrations/` directory that make similar changes and check the writing rules
- Pay special attention to the following:
- Indentation
- Which ALTER TABLE format is preferred
- Naming conventions
- Type handling
- NULL permission handling
- DEFAULT writing style
- COMMENT writing style
4. Create migration files based on guidelines and similar implementations
- Write accurate SQL following MySQL 8 syntax
## Important Notes
- **Follow conventions**: Always follow the conventions written in the documentation
- **Consistency**: Maintain consistency with existing code
- **Verification**: Confirm that the generated code matches the project's style
Now let's start generating migration files according to $ARGUMENTS.
The same concept can be applied to various cases
I think this concept of "automatically searching for similar examples after referencing documentation" can be applied to various cases. For example, when applied to blog writing where you want to generate text similar to your own style, it would look like this:
When writing articles, always refer to @WRITING.md.
Also, from the content of the article you're about to write, randomly pick several recent similar articles and:
- Use their structure as reference
- Mimic the writing style as much as possible
Articles are stored under the shibayu36.hatenablog.com/ directory.
Summary
In this article, I introduced a method to improve the accuracy of code generation and text generation by automatically searching for similar examples after referencing documentation. While this is based on subjective experience, the subtle details became much less concerning, so I encourage you to give it a try.
Top comments (0)