DEV Community

shibayu36
shibayu36

Posted on

I Created Claude Skills for Convenient Pull Request Review Operations from Claude Code

#ai #programming #github #codereview

(This is an English translation of my original Japanese article: 日本語版はこちら)

When using Claude Code or Claude Code Action to have AI autonomously review Pull Requests, I encountered several issues. To solve them, I created a Claude Skill specialized for PR review operations called github-pr-review-operation. Let me introduce it here.

The Problems

I faced these three issues:

  • When adding inline comments, the AI would comment on the wrong line
    • Claude Code Action comes with an MCP for adding inline comments, but it often placed comments about a specific line on a different line
  • Even when instructed to "consider past comments on this PR," it couldn't properly retrieve the comment list
    • It would fetch regular comments but proceed without getting inline comments
  • It couldn't properly reply to comments on the PR
    • It would reply to a specific comment by creating a separate inline comment instead

All these issues could be solved by properly using gh commands, but due to the probabilistic nature of LLMs, sometimes it worked and sometimes it didn't.

Teaching Claude Code the Operations with Claude Skills

As mentioned, the solution was to teach Claude Code how to use gh commands. Claude Skills seemed perfect for this purpose, so I created a custom Claude Skill specialized for PR review operations.

When creating Skills, it's good to use the Skill Generator that's built into Claude by default. I used the following prompt in Claude Desktop to create the base, then made various adjustments while testing it:

I want to create a skill for GitHub Pull Request reviews with the necessary operations. The required operations are:

- Get PR title and description
- Get PR file changes
- Get all comments on the PR
- Comment on the PR
- Add inline comment with line specification
- Reply to a specific comment on the PR
Enter fullscreen mode Exit fullscreen mode

The final result was a very simple structure with just SKILL.md. Using this alone solved the above issues almost 100% of the time. Very convenient!

SKILL.md:

---
name: github-pr-review-operation
description: "Skill for GitHub Pull Request review operations. Executes PR info retrieval, diff viewing, comment fetching/posting, inline comments, and comment replies using gh commands. Use for PR reviews, code reviews, and PR operations."
---

# GitHub PR Review Operation

PR review operations using GitHub CLI (`gh`).

## Prerequisites

- `gh` installed
- Authenticated with `gh auth login`

## Parsing PR URL

Extract the following from PR URL `https://github.com/OWNER/REPO/pull/NUMBER`:
- `OWNER`: Repository owner
- `REPO`: Repository name
- `NUMBER`: PR number

## Operations

### 1. Get PR Information

```bash
gh pr view NUMBER --repo OWNER/REPO --json title,body,author,state,baseRefName,headRefName,url
```

### 2. Get Diff (with line numbers)

```bash
gh pr diff NUMBER --repo OWNER/REPO | awk '
/^@@/ {
  match($0, /-([0-9]+)/, old)
  match($0, /\+([0-9]+)/, new)
  old_line = old[1]
  new_line = new[1]
  print $0
  next
}
/^-/ { printf "L%-4d     | %s\n", old_line++, $0; next }
/^\+/ { printf "     R%-4d| %s\n", new_line++, $0; next }
/^ / { printf "L%-4d R%-4d| %s\n", old_line++, new_line++, $0; next }
{ print }
'
```

Output example:
```
@@ -46,15 +46,25 @@ jobs:
L46   R46  |            prompt: |
L49       | -            (deleted line)
     R49  | +            (added line)
L50   R50  |              # Review guidelines
```

- `L number`: LEFT (base) side line number → Use with `side=LEFT` for inline comments
- `R number`: RIGHT (head) side line number → Use with `side=RIGHT` for inline comments

### 3. Get Comments

Issue Comments (comments on the entire PR):
```bash
gh api repos/OWNER/REPO/issues/NUMBER/comments --jq '.[] | {id, user: .user.login, created_at, body}'
```

Review Comments (comments on code lines):
```bash
gh api repos/OWNER/REPO/pulls/NUMBER/comments --jq '.[] | {id, user: .user.login, path, line, created_at, body, in_reply_to_id}'
```

### 4. Comment on PR

```bash
gh pr comment NUMBER --repo OWNER/REPO --body "Comment content"
```

### 5. Inline Comment (specifying code line)

First, get the head commit SHA:
```bash
gh api repos/OWNER/REPO/pulls/NUMBER --jq '.head.sha'
```

Single line comment:
```bash
gh api repos/OWNER/REPO/pulls/NUMBER/comments \
  --method POST \
  -f body="Comment content" \
  -f commit_id="COMMIT_SHA" \
  -f path="src/example.py" \
  -F line=15 \
  -f side=RIGHT
```

Multi-line comment (lines 10-15):
```bash
gh api repos/OWNER/REPO/pulls/NUMBER/comments \
  --method POST \
  -f body="Comment content" \
  -f commit_id="COMMIT_SHA" \
  -f path="src/example.py" \
  -F line=15 \
  -f side=RIGHT \
  -F start_line=10 \
  -f start_side=RIGHT
```

**Notes:**
- `-F` (uppercase): Use for numeric parameters (`line`, `start_line`). Using `-f` makes it a string and causes an error
- `side`: `RIGHT` (added lines) or `LEFT` (deleted lines)

### 6. Reply to Comment

```bash
gh api repos/OWNER/REPO/pulls/NUMBER/comments/COMMENT_ID/replies \
  --method POST \
  -f body="Reply content"
```

Use the `id` obtained from comment retrieval for `COMMENT_ID`.
Enter fullscreen mode Exit fullscreen mode

Key Implementation Detail: Enabling Correct Line Placement for Inline Comments

Getting inline comments on the correct line was the most challenging part, requiring the most adjustments after auto-generating the Skills.

Let me explain why inline comments ended up on wrong lines. When you get a diff with the gh pr diff command, you get output in git diff format. For example, an excerpt from https://github.com/shibayu36/slack-explorer-mcp/pull/20 looks like this:

diff --git a/main.go b/main.go
index 3029a7a..9383209 100644
--- a/main.go
+++ b/main.go
@@ -3,6 +3,7 @@ package main
 import (
        "context"
        "log/slog"
+       "net/http"
        "os"

        "github.com/mark3labs/mcp-go/mcp"
@@ -163,17 +164,55 @@ func main() {
                handler.SearchUsersByName,
        )

-       if err := server.ServeStdio(s, server.WithStdioContextFunc(func(ctx context.Context) context.Context {
-               ctx = WithSlackTokenFromEnv(ctx)
+       transport := os.Getenv("TRANSPORT")
+       if transport == "" {
+               transport = "stdio"
+       }
+
+       switch transport {
+       case "stdio":
+               if err := server.ServeStdio(s, server.WithStdioContextFunc(func(ctx context.Context) context.Context {
+                       ctx = WithSlackTokenFromEnv(ctx)

-               // Add session ID from ClientSession
-               if session := server.ClientSessionFromContext(ctx); session != nil {
-                       ctx = WithSessionID(ctx, SessionID(session.SessionID()))
Enter fullscreen mode Exit fullscreen mode

When adding inline comments, you need to specify a line number + side (LEFT for deleted lines or RIGHT for added lines):

gh api repos/OWNER/REPO/pulls/NUMBER/comments \
  --method POST \
  -f body="Comment content" \
  -f commit_id="COMMIT_SHA" \
  -f path="src/example.py" \
  -F line=15 \
  -f side=RIGHT
Enter fullscreen mode Exit fullscreen mode

Because of these two API interfaces, to add an inline comment after getting the diff, you need to extract line numbers from lines like @@ -163,17 +164,55 @@ func main() {, consider the - or + at the beginning of lines, determine whether side should be LEFT (deleted) or RIGHT (added), and use that combination for commenting. This extraction operation is difficult for LLMs (even with Opus 4.5), which caused the issue of inline comments being placed on wrong lines.

However, thinking about it more, if you look at GitHub's web "Files changed" page, the information needed for inline comments is immediately visible:

GitHub Files changed page showing line numbers

So I thought, why not create the same information from the CLI? That's how I created the command in section "2. Get Diff (with line numbers)" of SKILL.md:

gh pr diff NUMBER --repo OWNER/REPO | awk '
/^@@/ {
  match($0, /-([0-9]+)/, old)
  match($0, /\+([0-9]+)/, new)
  old_line = old[1]
  new_line = new[1]
  print $0
  next
}
/^-/ { printf "L%-4d     | %s\n", old_line++, $0; next }
/^\+/ { printf "     R%-4d| %s\n", new_line++, $0; next }
/^ / { printf "L%-4d R%-4d| %s\n", old_line++, new_line++, $0; next }
{ print }
'
Enter fullscreen mode Exit fullscreen mode

Using this to get the diff from https://github.com/shibayu36/slack-explorer-mcp/pull/20, you can get similar information to the web "Files changed" page in text format:

@@ -163,17 +164,55 @@ func main() {
L163  R164 |            handler.SearchUsersByName,
L164  R165 |    )
L165  R166 |
L166      | -   if err := server.ServeStdio(s, server.WithStdioContextFunc(func(ctx context.Context) context.Context {
L167      | -           ctx = WithSlackTokenFromEnv(ctx)
     R167 | +   transport := os.Getenv("TRANSPORT")
     R168 | +   if transport == "" {
     R169 | +           transport = "stdio"
     R170 | +   }
     R171 | +
     R172 | +   switch transport {
     R173 | +   case "stdio":
     R174 | +           if err := server.ServeStdio(s, server.WithStdioContextFunc(func(ctx context.Context) context.Context {
     R175 | +                   ctx = WithSlackTokenFromEnv(ctx)
L168  R176 |
L169      | -           // Add session ID from ClientSession
L170      | -           if session := server.ClientSessionFromContext(ctx); session != nil {
L171      | -                   ctx = WithSessionID(ctx, SessionID(session.SessionID()))
Enter fullscreen mode Exit fullscreen mode

Now Claude Code can easily get the information needed for inline comments from the L163 R164 | format on the left, allowing it to place comments on the correct lines.

Conclusion

In this article, I introduced a Claude Skill that's useful when having AI autonomously review Pull Requests using Claude Code or Claude Code Action. You can use it just by placing the SKILL.md described above under .claude/skills/github-pr-review-operation/. Give it a try!

Top comments (0)