How to Build a Self-Hosted AI Code Review Pipeline with Spring Boot and GitHub Webhooks
AI-assisted code review is becoming increasingly useful for catching issues before a Pull Request reaches a human reviewer.
But building an AI code-review workflow isn't simply a matter of sending source code to an LLM.
A useful system needs to handle several pieces:
GitHub Pull Requests
Webhooks
GitHub API integration
Diff extraction
AI provider integration
Structured AI responses
Finding categorization
Risk and quality assessment
Review history
Automated GitHub comments
In this article, I'll walk through a practical architecture for building such a system with Java, Spring Boot and GitHub webhooks.
- The basic architecture
The overall workflow can look like this:
GitHub Pull Request
↓
GitHub Webhook
↓
Spring Boot Application
↓
Fetch PR Changes
↓
AI Analysis
↓
Structured Findings
↓
Quality / Risk Assessment
↓
GitHub Review Comment
The important idea is that the AI model isn't directly responsible for the entire workflow.
Spring Boot acts as the orchestration layer.
It receives the GitHub event, retrieves the relevant changes, prepares the AI request, processes the response, stores the review result and optionally sends the findings back to GitHub.
- Receiving GitHub Pull Request events
GitHub webhooks allow an external application to receive events when something happens in a repository.
For example:
Pull Request opened
Pull Request updated
Pull Request synchronized
A Spring Boot endpoint can receive these events.
@RestController
@RequestMapping("/webhooks/github")
public class GitHubWebhookController {
@PostMapping
public ResponseEntity<Void> handleWebhook(
@RequestBody String payload) {
// Validate webhook
// Parse event
// Trigger review workflow
return ResponseEntity.ok().build();
}
}
In a real implementation, the webhook should also validate the GitHub signature before processing the request.
This prevents arbitrary callers from pretending to be GitHub.
- Don't send the entire repository to the AI
One of the most important design decisions is deciding what the model actually receives.
For a Pull Request review, sending the entire repository can be:
expensive
slow
unnecessary
difficult to fit into the model's context window
Instead, retrieve the Pull Request changes.
Conceptually:
Repository
↓
Pull Request
↓
Changed files
↓
Diff
↓
Relevant code
↓
AI
The GitHub API can be used to retrieve the files and changes associated with the Pull Request.
The review service can then construct an analysis request from those changes.
- Separate GitHub integration from AI integration
This is where the architecture becomes important.
I would avoid putting GitHub API calls and AI provider calls inside the same service.
Instead:
GitHubWebhookController
↓
PullRequestReviewService
↙ ↘
GitHubService AIReviewService
↓
AI Provider
For example:
public interface AIReviewService {
ReviewResult review(String codeChanges);
}
The rest of the application doesn't need to know whether the request is going to OpenAI, Gemini or a locally running Ollama model.
That's particularly useful if you want the provider to be configurable.
- Supporting multiple AI providers
A code-review tool shouldn't necessarily be locked to one AI provider.
A configuration-driven approach can look like:
ai.provider=openai
ai.model=gpt-model
And another environment could use:
ai.provider=gemini
ai.model=gemini-model
Or a local provider:
ai.provider=ollama
ai.model=local-model
The application logic should remain the same.
Conceptually:
AIReviewService
|
Provider Selection
/ | \
OpenAI Gemini Ollama
This separation also makes it easier to test the review workflow independently from a particular provider.
- Ask the AI for structured output
Another important part of an AI code-review system is the response format.
A free-form response such as:
"There might be a security problem here..."
is difficult for an application to process reliably.
Instead, define a structured review result.
For example:
{
"qualityScore": 8,
"riskLevel": "LOW",
"confidence": 0.90,
"findings": [
{
"category": "SECURITY",
"severity": "MEDIUM",
"title": "Potential credential handling issue",
"description": "Review how the credential is overridden.",
"recommendation": "Validate and restrict credential updates."
}
]
}
Now the application can use the response programmatically.
For example:
AI response
↓
Parse JSON
↓
Validate structure
↓
Store review
↓
Display dashboard
↓
Generate GitHub comment
This is much more useful than treating the model response as plain text.
- Categorizing findings
The review can classify problems into different categories.
For example:
🐛 Bugs
Potential logical errors or unexpected behavior.
🔐 Security
Potential security weaknesses, credential handling problems, authentication issues or unsafe input handling.
⚡ Performance
Potential inefficient operations, unnecessary database calls or expensive processing.
✅ Best Practices
Maintainability, readability and implementation practices that could be improved.
A finding can also have a severity:
CRITICAL
HIGH
MEDIUM
LOW
INFO
This allows developers to focus on the most important findings first.
- Adding a quality and risk assessment
Individual findings are useful, but a summary can make the review easier to understand.
For example:
Quality Score: 8/10
Risk: LOW
AI Confidence: 90%
Critical Issues: 0
Suggestions: 2
The important thing is to make these scores supporting signals, not replacements for human review.
An AI reviewer can help surface potential problems, but developers still need to inspect the actual code and decide whether a change should be merged.
- Posting the review back to GitHub
Once the analysis is complete, the application can use the GitHub API to post the review.
The workflow becomes:
PR opened
↓
Webhook
↓
Fetch diff
↓
AI analysis
↓
Structured result
↓
Save review
↓
GitHub review/comment
This makes the AI reviewer part of the existing developer workflow instead of forcing developers to open another application every time.
- Manual reviews are useful too
Webhooks are great for automation, but a dashboard can provide a manual review option.
For example:
Repository
Pull Request
↓
[ Review PR ]
↓
AI Analysis
This is useful when developers want to review a Pull Request on demand without waiting for another webhook event.
It also gives you a good place to expose review history.
- Store review history
If reviews are persisted, the application can provide historical information such as:
PR #42
Quality: 8/10
Risk: LOW
Critical Issues: 0
Suggestions: 2
Reviewed: Sep 26
Over time, this can become more useful than a single AI response.
You can start identifying trends such as:
Review History
PR #38 → 6/10
PR #39 → 7/10
PR #40 → 7/10
PR #41 → 8/10
PR #42 → 8/10
This gives developers another way to understand how their code is evolving.
- Security considerations
A self-hosted implementation still needs careful handling of credentials.
Typical sensitive values include:
GitHub Personal Access Token
GitHub Webhook Secret
AI Provider API Key
Database Credentials
These shouldn't be hardcoded in source code.
Use environment variables or an appropriate secret-management mechanism.
For example:
github.token=${GITHUB_TOKEN}
github.webhook-secret=${GITHUB_WEBHOOK_SECRET}
ai.api-key=${AI_API_KEY}
The application should also apply appropriate permissions to GitHub tokens rather than requesting more access than necessary.
- Why self-host the reviewer?
There are situations where developers may want more control over the review infrastructure.
With a self-hosted architecture:
GitHub
↓
Your infrastructure
↓
Your AI provider
Instead of:
GitHub
↓
Third-party SaaS
↓
AI provider
The exact privacy and compliance benefits depend on how the system and AI provider are configured, but self-hosting gives the developer more control over where the application, credentials and review workflow operate.
It also makes customization much easier.
You can modify:
review rules
prompts
finding categories
scoring
dashboard
AI provider
GitHub workflow
storage
authentication
- The complete workflow
Putting everything together:
GitHub Pull Request
│
▼
GitHub Webhook
│
▼
Spring Boot Application
│
▼
Fetch PR Changes
│
▼
Prepare Prompt
│
▼
AI Provider Layer
/ | \
OpenAI Gemini Ollama
\ | /
▼
Structured Review
│
┌──────────┼──────────┐
▼ ▼ ▼
Bugs Security Performance
│ │ │
└──────────┼──────────┘
▼
Quality / Risk
Assessment
│
┌──────────┴──────────┐
▼ ▼
Review Dashboard GitHub Comment
The key architectural principle is separation of concerns.
GitHub integration shouldn't know how an AI provider works.
The AI layer shouldn't know how GitHub authentication works.
And the business layer shouldn't need to change simply because you switch from one AI provider to another.
- Building CodeGuard AI
I built CodeGuard AI around this type of workflow.
It's a self-hosted GitHub AI code-review application built with:
Java 17+
Spring Boot
Spring Security
GitHub REST API
AI provider integrations
REST APIs
HTML/CSS/Vanilla JavaScript
Maven
It supports:
AI-powered Pull Request reviews
Bug detection
Security analysis
Performance analysis
Best-practice checks
Quality and risk assessment
AI confidence scoring
Automatic GitHub review comments
GitHub webhooks
Manual PR reviews
Review history
Review trends
Markdown/PDF export
OpenAI
Google Gemini
Ollama
The goal was to build a customizable source-code foundation rather than another closed review service.
Final thoughts
An AI code reviewer is more than an LLM prompt.
The interesting engineering work is around the LLM:
GitHub events → code extraction → provider abstraction → structured AI output → validation → persistence → developer-facing results → GitHub integration.
That's where Spring Boot fits particularly well.
If you're building an AI developer tool, keeping the AI layer modular and the surrounding application deterministic can make the system much easier to extend and maintain.
About the project
CodeGuard AI is available as a self-hosted Spring Boot source-code project for developers who want to build and customize their own AI-powered GitHub code-review workflow.
Build. Customize. Self-host.
Get CodeGuard AI → https://javacoder716.gumroad.com/l/codeguard-ai
Top comments (0)