Step 1: Parse the diff
Parse the diff
A review bot works on a pull request's diff. Before you can ask a model about it or grade what it finds, you have to turn the raw unified diff into the lines that actually changed, each with its line number in the new file, the number everyone will refer to.
harness.py gives you ask(prompt) (the model, cached) and load_diffs() (the fixtures: each has a
diff and the bugs planted in it). You write your bot in reviewer.py.
Write parse_diff(diff_text): return [{"file", "added": [{"line", "code"}]}], one entry per file, where
line is the added line's number in the new file. Read the new-file start from each @@ -a,b +c,d @@
header; an added (+) line sits at the current number and advances it, a context ( ) line advances it, a
removed (-) line does not.
reviewer.py, the file you edit77 lines
"""Your code review bot. It reads a pull-request diff, asks the model to find bugs in the added lines,
scores what it found against the bugs planted in the fixtures, drops the noise, and turns the result into a
merge decision and a review comment."""
import json
import harness
SEVERITY = {"low": 1, "medium": 2, "high": 3}
# ---------- Step 1: parse the diff ----------
def parse_diff(diff_text):
"""The added lines of a unified diff, per file. Return [{"file", "added": [{"line", "code"}]}] where
line is the added line's number in the NEW file. Track the new-file counter from each @@ hunk header;
added ('+') and context (' ') lines advance it, removed ('-') lines do not."""
# TODO (Step 1): walk the diff; set the new-file counter from each @@ header; a '+'
# line is added at the counter (then +1), a ' ' context line advances it, a '-' line does not.
raise NotImplementedError("Step 1: write parse_diff()")
# ---------- Step 2: ask the model to review ----------
def review(diff_text, ask=harness.ask):
"""Ask the model to review the diff and return its findings as
[{"file", "line", "severity", "issue"}]. Present the added lines with their numbers, require a JSON
array back, and parse it even when the model wraps it in prose or a code fence. On unparseable output
return []."""
raise NotImplementedError("review() arrives in Step 2")
def _parse_findings(text):
"""Pull the JSON array of findings out of the model's reply; [] if there is none."""
start, end = text.find("["), text.rfind("]")
if start < 0 or end < start:
return []
try:
data = json.loads(text[start:end + 1])
except json.JSONDecodeError:
return []
out = []
for d in data:
if isinstance(d, dict) and "file" in d and "line" in d:
out.append({"file": d["file"], "line": int(d["line"]),
"severity": d.get("severity", "medium"), "issue": d.get("issue", "")})
return out
# ---------- Step 3: score against the planted bugs ----------
def score(findings, bugs, window=2):
"""Precision, recall and F1 of findings against the planted bugs. A finding matches a bug when they are
in the same file and within `window` lines; each bug counts at most once. Returns
{"tp", "fp", "fn", "precision", "recall", "f1"}."""
raise NotImplementedError("score() arrives in Step 3")
# ---------- Step 4: triage the findings ----------
def triage(findings, min_severity="medium"):
"""Cut the noise: drop findings below min_severity, and when several land on the same file and line keep
only the most severe. Return the survivors, most severe first."""
raise NotImplementedError("triage() arrives in Step 4")
# ---------- Step 5: the verdict ----------
def verdict(findings):
"""The merge decision: block when any high-severity finding remains. Return
{"block": bool, "high": <count>}."""
raise NotImplementedError("verdict() arrives in Step 5")
def summary(findings):
"""A short review comment: one line per finding, most severe first, or an approval line if there are
none."""
raise NotImplementedError("summary() arrives in Step 5")diffs.jsonharness.pytry_it.py