The bot's threads get resolved, the code stays the same

The bot writes findings, the team resolves the threads in a batch, and the code under them does not change. The one thing you cannot tell is the one that matters: were the findings off the mark, or were they closed unread?

It is not the developer's laziness. Closing a thread is the simplest thing to do with a bot's finding: one click and the problem is gone, and if the project requires every thread to be resolved before merging, it also opens the merge. Some developers object to the bot in the thread, but the bot answers only if reply mode (reply.enabled) is on, and it is off by default. Reactions to comments the bot does not read. Until whoever is responsible for code quality — a team lead or the CTO — starts keeping an eye on it, threads will keep getting closed.

This symptom has three different causes, and each has its own cure. Do not guess: measure first, then find the cause, and only then change the settings. A setting picked at random can make things worse.

Measure first, through the GitLab or GitHub API

Review metrics will not help here: they are written per run — models, tokens, how many findings the judge kept and how many it dropped. The metrics have no column for «fixed or just closed». In the summary the bot says one thing about resolved threads: how many repeated findings it did not raise because their threads are closed. Which threads are closed, by whom, and whether the code under them changed is kept by GitLab or GitHub themselves, and it is their API you ask: on GitLab the merge request discussions (/merge_requests/:iid/discussions in the project API), on GitHub the pull request's reviewThreads in the GraphQL API.

The question to ask is not «was there a commit after the close» but «did the line under the thread change since the thread was opened». The usual order is to fix, push and only then resolve, so there may be no commits after the close at all, even when everything was fixed. GitLab answers the question itself: when the line under a thread changes, it writes a system note into the thread, «changed this line in version N of the diff». The script below prints the merge request's author and merge time, and under them the bot's threads: who closed them and when, and whether the line under them changed. The GitLab API returns threads only with a token, even for a public project; the read_api scope is enough.

GITLAB=https://gitlab.com
PROJECT=reviewgate-group%2Finvoices-api   # url-encoded path, or the numeric project id
MR=1
BOT=reviewgate-bot                        # the account the bot posts as

get() { curl -sf --header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
          "$GITLAB/api/v4/projects/$PROJECT/$1"; }

get "merge_requests/$MR" |
  jq -r '"MR author: \(.author.username)  ·  merged: \((.merged_at // "no")[0:16])"'

get "merge_requests/$MR/discussions?per_page=100" | jq -r --arg bot "$BOT" '
  def changed: any(.notes[]; .system and (.body | startswith("changed this")));
  [ .[] | select(.notes[0].author.username == $bot and .notes[0].resolvable)
        | .notes[0] + {changed: changed} ]
  | "bot threads: \(length)  ·  resolved: \(map(select(.resolved)) | length)",
    ( .[] | "  \(.position.new_path):\(.position.new_line)  "
          + (if .resolved
             then "closed by \(.resolved_by.username) \(.resolved_at[0:16])"
             else "open" end)
          + (if .changed then "  ·  line changed" else "" end) ),
    "resolved, line untouched: \(map(select(.resolved and (.changed | not))) | length)"'

MR author: Novohudonosor  ·  merged: no
bot threads: 6  ·  resolved: 4
  src/pricing/pricing.service.ts:52  closed by Novohudonosor 2026-09-08T12:48
  src/invoices/invoices.service.ts:72  closed by Novohudonosor 2026-09-08T12:48
  src/invoices/invoices.service.ts:73  closed by Novohudonosor 2026-09-08T12:48
  src/invoices/invoices.service.ts:69  closed by Novohudonosor 2026-09-08T12:48
  src/invoices/invoices.service.ts:77  open
  src/reports/reports.service.ts:23  open
resolved, line untouched: 4

The main thing is the last line: on the GitLab merge request of our sample repository four threads are closed, and the line under none of them changed. It is a signal, not a verdict: a fix made elsewhere is invisible to the measurement, and an edit of the same line is not yet a fix. Open the disputed threads and read them.

On GitHub the same question is one GraphQL query. GitHub sets isOutdated on a thread whose code changed after the comment, so a thread with isResolved: true and isOutdated: false is our case. Tell the bot's threads apart by the author of the first comment. The query can be sent with gh api graphql -f query='…'.

{
  repository(owner: "ReviewGate", name: "invoices-api") {
    pullRequest(number: 1) {
      author { login }
      mergedAt
      reviewThreads(first: 100) {
        nodes {
          path line isResolved isOutdated resolvedBy { login }
          comments(first: 1) { nodes { author { login } } }
        }
      }
    }
  }
}

Now look at who closed the threads — «closed by» in the script's output. If the author of the merge request closes them, that is their opinion of the findings. If one and the same person closes other people's threads — usually whoever merges, all at once right before the merge — that is a process: the threads are closed so that the merge opens. One merge request will not tell you; run the script on the last five to ten. And a caveat for GitLab: if the project resolves outdated threads automatically (Settings → Merge requests → Merge options), GitLab itself closes a thread whose line changed and records whoever pushed as the one who closed it. The script shows such a thread with «line changed», so it is not our case.

One symptom, three causes

The tax. If the project requires every thread to be resolved before merging (GitLab: the «All threads must be resolved» check; GitHub: «Require conversation resolution before merging»), a resolve tells you nothing about the finding: the thread is closed just to get the merge through. Do not look for intent in it — remove the tax. With min_severity: major the bot opens no threads for findings below major: they arrive as a folded list in the summary, and nothing is lost. Mind the side effect: muted findings do not hold the gate either. Raise this threshold above severity_gate, and the gate starts firing only from the new threshold.

Right, but not fixing it. A common case and an easy one to miss: the finding is correct, but it is about old code the author only touched or moved. Three moves:

• Resolve the thread. For the rest of the merge request the bot will not raise that finding again — how it recognises a repeat is in the next section. A new merge request gets new threads.
• dont_flag in the config — for what you do not want to hear about at all. It is text in the prompt, a request to the model rather than a filter, and it covers the whole repository. The bot reads the config from the merge request's branch, so dont_flag already works on the push that added it.
• A line in the merge request description. Both the model and the judge read it on every run: «files under legacy/ are moved as-is, do not review them». It costs nothing and lasts as long as the merge request. Editing the description does not start a run: the line takes effect from the next push.

A rule cannot be limited to paths. A rule has three fields — id, description and severity. Add globs, and the bot warns in the summary on every run that the key was not recognised and not applied, while the rule still covers the whole diff. A path can be taken out of the review entirely with ignore, but then the model does not see those files at all.

The rule is wrong. If the threads that get closed are findings marked 📐 — from your own rules — the problem is not the team: the rule is broader than you meant, or it asks for something you no longer believe in. Lower its severity, narrow the wording or delete it. A rule nobody follows produces paid findings that get closed at once, and teaches the team that closing a thread is a normal answer. And one thing about the bot to know before any argument: in a resolved thread the bot always stays silent, even with reply mode on. Reply and resolve in one go (in GitLab, the «Resolve thread» checkbox under the reply), and the bot will not answer.

What the bot does with a resolved thread

Resolving a thread does not delete the finding: the bot may find it again. What changes is how the bot treats it, and not at once: the resolve itself starts no run, and everything below happens on the next run — usually the next push.

It will not be raised again in this merge request. When the same problem comes up again, the bot looks for it among the merge request's threads by three keys in turn: the same file, the finding's name (it stands in bold at the start of the thread) and the line text; the same file and the line text; the same file and the finding's name. If a resolved thread matches, the finding is not posted. A shifted line produces no second copy: the keys look at the text of the line, not at its number. If a variable on the thread's line was renamed, the line text is different, and only the third key can match — when the model gives the finding the same name. A different name means a new thread. The third key has a flip side: one resolved thread silences every new finding with the same name in its file until the merge request ends, even findings about other lines.

A repeated finding is not posted, but it is counted. On the GitHub pull request of our sample repository we closed four of the bot's five threads without touching a line, then pushed a cosmetic commit — a renamed local variable and a doc comment — that fixes nothing. The bot re-checked both changed files, produced three findings and posted exactly one comment:

❌ Severity gate (`major`) failed: 3 blocking findings.

Findings: **3** (🔴 2 · 🟠 1); new inline comments: 1.
From your team's standards (📐): 2.
From earlier runs — already under review, not duplicated: 1;
  resolved by you, not reopened: 1.

Three blocking findings, one new comment. Of the other two, one is an older thread still open on the pull request; the other is a thread you closed, and it appears nowhere in the diff. The verdict is computed from what the run found, not from what it posted: the finding in the closed thread went into the gate too. The only place that says where it went is the line about resolved threads, and that line gives a number, not a list.

A resolved blocker does not hold the gate once its file is no longer re-checked. After a push the bot checks, by default, only the files changed since the last run. To keep a blocker from getting lost on pushes to other files, the bot carries the blocking findings of earlier runs into the gate — but only from open threads. A resolved thread is not carried. Resolve a blocking thread, push a change to another file, and the gate is green, although the code under the finding never changed. If the push touches that file and the model repeats the finding, the gate is red again, as in the example above.

The gate can go green without a code fix in other ways too. The author can edit .reviewgate/config.yml right in their merge request — raise severity_gate or put the file under ignore: the bot reads the config from the merge request's branch. It flags such an edit in the summary with the note «.reviewgate/config.yml changed in this MR». And on a re-check the model may simply not repeat the finding.

Neither GitLab nor GitHub lets you restrict who resolves the bot's threads: on GitLab the author of the merge request and anyone with the Developer role or higher can do it, on GitHub the author of the pull request and anyone with write access to the repository. The bot has no such setting either. If the gate is a real barrier for you, agree in the team who resolves blocking threads, and watch the summary for the note about a changed config.

Which repository to start with

If you are still rolling the bot out, the order matters more than the settings. Start with the repository where nobody reviews code today, not with the one where reviews are done well. Where every merge request already gets a careful read from a person, the bot's findings compete with the colleagues'. Where nobody reads, the bot is the only one reading the code, and its threads are the only feedback the author gets.

A team that does not review will meet the bot with silence too — nobody argues, nobody answers, threads get closed on the way to the merge. There the gate is the only thing that gives a finding weight. Set severity_gate to the level you would genuinely stop a merge for, and leave everything below it as information. Check that a merge cannot pass without a green status: on GitLab that takes the «Pipelines must succeed» check, on GitHub the «Require status checks before merging» rule with the bot's check in branch protection. And remember how the gate is bypassed without a code fix — that is covered in «What the bot does with a resolved thread».

Next

• Rules your linter cannot check— a rule people keep closing is a rule to rewrite
• Arguing with a finding— what to do instead of closing the thread
• Config reference— min_severity, dont_flag, severity_gate and reply mode in full