From GitHub Issue to Pull Request: Running Claude Code Unattended

#agents #ai #llm #tutorial

You already run Claude Code by hand: copy issues into a prompt, watch it work, check the diff, and if something breaks halfway through, you restart it. This works fine for one task at a time, but it falls apart when you have 10 tasks simultaneously.

Claude Code is good at handling routine engineering tasks: bug fixes, dependency bumps, and small features, when the prompt is clear and the task is scoped. But when it comes to scaling, you need an infrastructure with isolated workspaces, retry logic, state that survives a restart, and tracker integration, not to waste time on babysitting.

Sortie removes the manual work. You label an issue, Sortie picks it up, creates an isolated workspace, runs the agent, retries it if it stalls, and opens a pull request when it's done. This article describes how to set the entire process from an empty directory to a GitHub issue turning into a PR without you touching the keyboard in between.

What you need

A GitHub repository you control
Export two environment variables:

ANTHROPIC_API_KEY - authenticates Claude Code
GITHUB_TOKEN - it's read by tracker.api_key: $GITHUB_TOKEN for polling/updating issues, and it's the same token gh pr create inside the after_run hook uses to open the PR, so it needs Issues: read/write, Contents: read, and Pull requests: read/write scopes on that repository, all on one fine-grained PAT.

Push access to the repository over SSH. The after_create hook below clones with git@github.com:..., so git authenticates with your SSH key, not with GITHUB_TOKEN. Verify with ssh -T git@github.com. If you'd rather stay on one credential, swap the clone URL for https://${GITHUB_TOKEN}@github.com/yourorg/yourrepo.git and give the token Contents: read/write.
In your repository, create the agent-ready label — you need it to exist before you can put it on an issue, and query_filter finds nothing without it. Creating in-progress, review, and done up front is also worth doing: GitHub does create a missing label when Sortie applies it, but it comes out in default gray, and the colors are yours to pick.
Claude Code installed on your machine. Verify with command: claude --version

The entire process takes about 15 minutes.

Installation

To install Sortie on Linux or macOS, run the command:

curl -sSL https://get.sortie-ai.com/install.sh | sh

For Homebrew:

brew install --cask sortie-ai/tap/sortie

On Windows, use the PowerShell equivalent. It runs on the PowerShell 5.1 that ships with Windows 10 and 11, as well as on PowerShell 7:

irm 'https://get.sortie-ai.com/install.ps1' | iex

That one installs into %LOCALAPPDATA%\Programs\sortie, so it needs no administrator rights, and it adds the directory to your user PATH for you. Shells you already have open keep their old environment, so reopen PowerShell before continuing.

Sortie is a single Go binary file, for which you need no database server, Docker, or Node.js. State lives in an embedded SQLite file (.sortie.db) that Sortie creates next to your workflow file — no need to create it manually.

Confirm it's on your PATH by running the command:

sortie --version

Configuring WORKFLOW.md

This file contains everything you need for Sortie, including trackers to pull tasks from, agents to run, and prompts telling them what to do. Create a directory and a file.

---
tracker:
  kind: github
  api_key: $GITHUB_TOKEN
  project: yourorg/yourrepo
  query_filter: "label:agent-ready"
  active_states:
    - agent-ready
    - in-progress
  in_progress_state: in-progress
  handoff_state: review
  terminal_states:
    - done

polling:
  interval_ms: 30000

workspace:
  root: ./workspaces

hooks:
  after_create: |
    git clone --depth 1 git@github.com:yourorg/yourrepo.git .
  before_run: |
    git fetch origin main
    git checkout -B "sortie/${SORTIE_ISSUE_IDENTIFIER}" origin/main
  after_run: |
    git add -A
    git diff --cached --quiet || \
      git commit -m "sortie(${SORTIE_ISSUE_IDENTIFIER}): automated changes"
    git push origin "sortie/${SORTIE_ISSUE_IDENTIFIER}" --force-with-lease
    gh pr create --fill \
      --head "sortie/${SORTIE_ISSUE_IDENTIFIER}" \
      --base main 2>/dev/null || true
  timeout_ms: 120000

agent:
  kind: claude-code
  command: claude
  max_turns: 5
  max_concurrent_agents: 1
  turn_timeout_ms: 1800000
  stall_timeout_ms: 300000

claude-code:
  permission_mode: bypassPermissions
  max_budget_usd: 5
  max_turns: 50

server:
  port: 8888
---

You are a senior engineer working in this repository. Your work is tracked by
an automated orchestrator (Sortie) that manages your session, retries failures,
and monitors progress.

## Task

**#{{ .issue.identifier }}**: {{ .issue.title }}
{{ if .issue.description }}

### Description

{{ .issue.description }}
{{ end }}
{{ if .issue.url }}

**Ticket:** {{ .issue.url }}
{{ end }}

## Rules

1. Read existing code and project conventions before writing anything new.
2. Keep changes minimal. Implement exactly what the task requires.
3. Run the project's lint and test commands before finishing. All checks must pass.
4. Write tests for new functionality, covering edge cases, not just the happy path.
{{ if not .run.is_continuation }}

## First run

Start by understanding the codebase structure. Check for existing patterns
and follow them. Write the implementation, add a test, and verify everything passes.
{{ end }}
{{ if .run.is_continuation }}

## Continuation (turn {{ .run.turn_number }}/{{ .run.max_turns }})

You are resuming this task. Run `git status` and check test output to understand
the current state. Continue from where the previous turn left off. Do not repeat
work already completed.
{{ end }}

The file consists of two parts. Everything between the two --- markers is YAML front matter for configuration. Everything after the closing --- is a Go text/template, rendered separately for each issue and sent to the agent as its prompt.

In this file:

query_filter: "label:agent-ready" - Sortie only takes issues with this label. If the label is missing, it ignores the issue.
active_states / in_progress_state / handoff_state / terminal_states - a status map through GitHub labels. active_states must include in_progress_state, otherwise Sortie refuses to start and sortie validate fails with a config error (enforced in code). handoff_state must not match any of active_states or terminal_states (also enforced in code).
agent.max_turns: 5 - how many turns the orchestrator runs within one session: the first turn plus up to four continuations. Defaults to 20.
claude-code.max_turns: 50 - how many internal steps Claude Code itself takes in a single invocation. These are two different fields that happen to share a name, in different sections. At 5x50, that's up to 250 steps per task.
claude-code.max_budget_usd: 5 - a cost cap per invocation, not per session. Sortie passes --max-budget-usd on every turn and Claude Code's counter starts from zero each time, so with agent.max_turns: 5 the worst case for one issue is five turns of $5, not $5 total. The cap is also checked at the turn boundary rather than mid-turn, so a single turn can overshoot it. To bound one issue, size it as budget x max_turns and lower one of the two.
permission_mode: bypassPermissions - lets the agent act without stopping to ask for approval on each step; run it against a repository you control. Setting it explicitly is the right move, but omitting it does not make the run interactive: the adapter then falls back to the deprecated --dangerously-skip-permissions, which bypasses the same checks.

Before running the config, do the following steps:

Replace yourorg/yourrepo with your repository (URL cloning) in tracker.project and hooks:after_create.
On Windows, adapt the hooks: they run through cmd.exe /C rather than sh -c, so write %SORTIE_ISSUE_IDENTIFIER% instead of ${SORTIE_ISSUE_IDENTIFIER} and drop the trailing \ line continuations.
Check the file for mistakes using the command: sortie validate ./WORKFLOW.md

Running the test issue

Create a test issue with the agent-ready label and add the description, for example, “Add a /healthz endpoint returning 200” — the more specific your description is, the more specific result you get.

Run the sortie ./WORKFLOW.md and watch the running process in the dashboard.

All git operations are Sortie's hooks:

after_create — clones the repository into an isolated workspace under its working directory.
before_run — creates the sortie/<issue-identifier> branch off origin/main.
after_run — stages, commits, pushes, and runs gh pr create --fill to open the actual PR.
After that, Sortie swaps the in-progress label for review. The agent-ready label is already gone by then — Sortie replaced it with in-progress back when it picked the issue up.

In its turn, Claude Code edits the file in the isolated workspace.

Troubleshooting

If a run doesn’t go smoothly, here’s what happens, and how Sortie recovers on its own:

Retries. A failed or stalled turn goes into a retry queue with exponential backoff, up to agent.max_retry_backoff_ms (5 minutes by default). The issue tries again on its own.
Stalls. If the agent goes quiet mid-session without events and output for longer than stall_timeout_ms (5 minutes by default), Sortie treats the session as dead, kills it, and requeues the issue for retry.
Cost control. claude-code.max_budget_usd caps spend per session, independent of how many turns it takes. Combine it with agent.max_concurrent_agents to bound total exposure across everything running at once.
State survives restarts. Retry timers, run history, and session metadata live in a SQLite file next to WORKFLOW.md. If you kill the process and start it again, nothing is lost, and nothing gets dispatched twice.

What’s next

To switch the agent, change the line: swap agent.kind: claude-code for codex, copilot-cli, opencode, or kiro, and the tracker configuration stays exactly as it is.

Jira, Linear, and Gitea work the same way on the tracker side - same file, different tracker.kind. Sortie also supports a CI feedback loop, where a failing check on the opened PR gets routed back to the agent instead of left for you to notice.

For full configuration reference and adapter details, check docs.sortie-ai.com. Source and issue tracker: github.com/sortie-ai/sortie.