Pull request review has become a bottleneck. Senior engineers spend hours skimming diffs that are 90% boilerplate, overlook the 10% that matters and still merge subtle bugs. In 2026 there is no good reason to do this by hand. Claude Sonnet 4.6 is the strongest publicly available coding model, and paired with the GitHub API it becomes an in-house reviewer that runs on every PR, costs cents and never tires.
Below we build a PR review bot on Claude Sonnet 4.6 through codexkey (https://api.codexkey.online/v1). Check current customer prices and channel availability in the catalog. The bot has to cope with provider limits and temporary errors.
Why use Claude for code review
Three reasons, ranked by importance:
- Coding ability. Sonnet 4.6 sits near the top of SWE-bench Verified and reads diffs like a senior reviewer: it tracks types across files, spots missing error handling and catches off-by-one errors a linter misses. GPT-5 and Gemini 3 are close, but for review, which rewards caution and explanation, Claude has the better signal-to-noise ratio.
- Reliable structured output. Under
tool_use, Claude follows JSON schemas with near-100% compliance. For a bot that matters: the output must parse cleanly every time. One malformed JSON in a thousand runs and someone gets paged at night. - Long context plus prompt caching. 200K tokens hold the repo style guide, architecture notes and recent commits, all placed in the system prompt. After the first call, caching makes that context nearly free. We covered the cache economics in our Claude API pricing breakdown.
Hosted services such as CodeRabbit and Greptile are a fair comparison. They are decent, but you hand over your source code, pay $30-$100 per developer a month and accept whatever prompt they wrote. Your own bot costs $0.05-$0.30 per PR, runs on your prompt, and the code never leaves your CI.
Architecture
The whole pipeline is four blocks:
GitHub PR opened/updated
↓
GitHub Action triggers
↓
Fetch diff via GitHub API
↓
Send to Claude Sonnet 4.6 (via codexkey)
↓
Parse JSON response
↓
Post review comments via GitHub API
That is all. No vector database, no embeddings, no agent loop. The diff is small (usually <50K tokens), Claude reads it once and returns structured comments. Anything more complex is over-engineering until the data says otherwise.
The prompt
The prompt is 80% of the work. Here is the template that has held up for us on Go, TypeScript and Python codebases:
You are a senior staff engineer reviewing a pull request. Your job is to
catch bugs, security issues, and design mistakes — NOT to enforce style
(linters do that) and NOT to praise good code.
SCOPE:
- Only comment on lines changed in this diff.
- Do not comment on unchanged context lines.
- Do not request changes you cannot justify with a concrete failure mode.
SEVERITY:
- blocker: will cause production incident, data loss, security breach, or
obvious correctness bug. Must be fixed before merge.
- warning: real issue but not blocking. Race conditions under unlikely
load, missing error paths, unclear naming that will bite later.
- nit: minor suggestion. Use sparingly — at most 2 per PR.
OUTPUT:
Call the `post_review` tool with a JSON list of comments. Each comment
must have:
- file: path relative to repo root
- line: line number in the new file (after the change)
- severity: "blocker" | "warning" | "nit"
- comment: 1–3 sentences. Start with the problem, then the fix.
- justification: why this matters. If you can't write this, drop the comment.
If the PR looks good, return an empty list. Do not invent issues.
REPO CONTEXT:
{cached system block: style guide, architecture notes, recent decisions}
DIFF:
{the actual diff}
Two non-obvious things make it work.
The justification field guards against false positives. Given room, Claude (like any model) will invent concerns. Requiring a written justification cuts noise by about 60% in our internal eval: the model discards weak comments before emitting them.
tool_use instead of "respond with JSON". If you define post_review as a tool with a strict input schema, the API itself rejects malformed output. You never write a JSON repair function.
Tool definition
const postReviewTool = {
name: "post_review",
description: "Post review comments on the PR. Pass an empty array if no issues found.",
input_schema: {
type: "object",
properties: {
comments: {
type: "array",
items: {
type: "object",
properties: {
file: { type: "string" },
line: { type: "integer" },
severity: {
type: "string",
enum: ["blocker", "warning", "nit"]
},
comment: { type: "string" },
justification: { type: "string" }
},
required: ["file", "line", "severity", "comment", "justification"]
}
}
},
required: ["comments"]
}
};
Set tool_choice: { type: "tool", name: "post_review" } to force the model to call it. No prose reply, nothing extra.
The bot in TypeScript
import Anthropic from "@anthropic-ai/sdk";
import { Octokit } from "@octokit/rest";
const claude = new Anthropic({
apiKey: process.env.CODEXKEY_API_KEY,
baseURL: "https://api.codexkey.online/v1",
});
const gh = new Octokit({ auth: process.env.GITHUB_TOKEN });
async function reviewPR(owner: string, repo: string, pull_number: number) {
// 1. Fetch the diff
const { data: files } = await gh.pulls.listFiles({
owner, repo, pull_number, per_page: 100,
});
// 2. Chunk by file — parallelize for large PRs
const reviews = await Promise.all(
files
.filter(f => f.patch && f.status !== "removed")
.map(file => reviewFile(file))
);
// 3. Flatten + dedupe by (file, line, severity)
const all = reviews.flat();
const seen = new Set<string>();
const unique = all.filter(c => {
const k = `${c.file}:${c.line}:${c.severity}`;
if (seen.has(k)) return false;
seen.add(k);
return true;
});
// 4. Post as a single review
await gh.pulls.createReview({
owner, repo, pull_number,
event: unique.some(c => c.severity === "blocker")
? "REQUEST_CHANGES"
: "COMMENT",
comments: unique.map(c => ({
path: c.file,
line: c.line,
body: `**[${c.severity}]** ${c.comment}\n\n_${c.justification}_`,
})),
});
}
async function reviewFile(file: any) {
const response = await claude.messages.create({
model: "claude-sonnet-5",
max_tokens: 4096,
system: [
{
type: "text",
text: REPO_CONTEXT, // style guide, arch notes
cache_control: { type: "ephemeral" },
},
{
type: "text",
text: REVIEW_INSTRUCTIONS, // the prompt above
},
],
tools: [postReviewTool],
tool_choice: { type: "tool", name: "post_review" },
messages: [
{
role: "user",
content: `File: ${file.filename}\n\nDiff:\n\`\`\`diff\n${file.patch}\n\`\`\``,
},
],
});
const toolUse = response.content.find(b => b.type === "tool_use");
return toolUse ? (toolUse.input as any).comments : [];
}
Note the cache_control on the repo context block. That is prompt caching: the first PR pays full price for those tokens, and every later PR within 5 minutes pays 10%. On a busy repo this cuts cost by 70% or more.
Handling large diffs
A 5,000-line PR blows the token budget if you send it in one piece. The strategy above, one Claude call per file, takes care of it:
- Per-file calls run in parallel. Wall time stays around 3-5 seconds even for a 30-file PR.
- Each call gets a narrow context. Claude reviews
auth.tswithoutpackage-lock.jsonin the way. - Duplicates are removed across calls. Claude sometimes flags the same pattern in two files, and the
(file, line, severity)key merges them.
For genuinely huge PRs (>100 files), add a pre-filter that drops lockfiles, generated code and vendor/. After that most "huge PRs" shrink to 10 files worth reviewing.
GitHub Actions setup
name: Claude Review
on:
pull_request:
types: [opened, synchronize]
jobs:
review:
runs-on: ubuntu-latest
permissions:
pull-requests: write
contents: read
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: "20"
- run: npm ci
- run: npx tsx scripts/review.ts
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
CODEXKEY_API_KEY: ${{ secrets.CODEXKEY_API_KEY }}
PR_NUMBER: ${{ github.event.pull_request.number }}
REPO_OWNER: ${{ github.repository_owner }}
REPO_NAME: ${{ github.event.repository.name }}
Put your codexkey key into the repo secrets as CODEXKEY_API_KEY, as in the YAML, commit the file, and the bot is live.
Cost per PR
Real numbers from our own deployment across 200 PRs:
| PR size | Tokens in | Tokens out | Cost (cached) |
|---|---|---|---|
| Small (<10 files) | ~8K | ~500 | $0.04 |
| Medium (10-30) | ~25K | ~1.5K | $0.12 |
| Large (30-80) | ~70K | ~3K | $0.28 |
These figures use Claude Sonnet 4.6 prices on codexkey with caching on. A 50-engineer team doing 100 PRs a day spends roughly $300 a month, less than one CodeRabbit Pro seat.
Your own bot or a ready-made service like CodeRabbit
| Dimension | Your bot (Claude + codexkey) | CodeRabbit / Greptile |
|---|---|---|
| Cost | $0.05-$0.30 per PR | $30-$100 per developer a month |
| Prompt control | Full | None |
| Where data lives | Your CI | Their servers |
| Customization | Anything you can prompt | Their feature set |
| Setup time | One day | One hour |
| Maintenance | Yours | Theirs |
The trade-off is real. SaaS is quicker to set up and you do not maintain it. Your own bot gives you control, a lower unit cost at scale, and room to encode your team's actual taste.
Bottom line
You can ship this in a day. Every part (Claude Sonnet 4.6 through codexkey, GitHub Actions, the Octokit SDK) is stable, well documented and cheap. The one hard part is the prompt, and the template above covers 90% of what you need.
Start simple: one prompt, one file at a time, blocker, warning and nit. Run the bot on your last 20 merged PRs and read the output. You will tune the prompt a couple of times, and by the end of the week you will have something better than most $50-a-month services.