DEV Community

Breach Protocol
Breach Protocol

Posted on • Originally published at groundtruth.day

NeurIPS rebuttal week ended with authors, reviewers and chairs all reporting the same silence

The NeurIPS 2026 author-facing discussion window closed on 3 August with authors, reviewers and at least one area chair independently describing the same breakdown: rebuttals posted, reminders sent, program chairs contacted, and no reply from anyone. The conference redesigned this year's cycle specifically to make rebuttal discussion consequential, which makes the reported silence a failure of the new mechanism rather than ordinary conference grumbling.

Key facts

  • The window: authors, reviewers and area chairs were to discuss papers from 27 July to 3 August, after which authors are excluded while reviewers and chairs deliberate through 10 August. Decisions arrive 24 September. Official schedule
  • The obligation: the 2026 handbook requires reviewers to read responses and at minimum acknowledge one that did not change their view, and requires area chairs to initiate discussion immediately.
  • The scale, from the last published year: NeurIPS 2025 handled 21,575 valid submissions with 20,518 reviewers, 1,663 area chairs and 199 senior area chairs. The conference has published no 2026 totals.
  • Primary sources: the NeurIPS 2026 handbook, dates page, and area chair pilot announcement.

Peer review at machine-learning conferences works on a rebuttal cycle. Reviewers post criticisms, authors get a short window to respond — correcting misreadings, adding requested experiments, conceding what needs conceding — and reviewers are then supposed to reconsider. The entire value of the rebuttal depends on someone reading it. If nobody returns, the exercise is a formality that consumes a week of authors' lives.

That is what multiple independent threads describe. One reports no response from all reviewers and the area chair, persisting after public comments, reviewer reminders and contacting the program chairs. Another reports the same across multiple papers. From the other side, a reviewer describes receiving the second-phase notification late and facing five month-old assignments to reconstruct from scratch. What makes this more than seasonal complaining is that the accounts come from all three seats in the process and describe the same break.

NeurIPS built this year's cycle to prevent exactly this. Under the area chair pilot, chairs must issue an initial meta-review before rebuttal that, for most papers, names the specific concerns that could change the outcome — a guidance document rather than a verdict. The final meta-review must then state whether the response addressed those concerns and explain the decision. That design also explains a second confusion authors reported: an initial meta-review with no accept-or-reject recommendation is working as intended, though it still fails the pilot's purpose if it gives an author no usable path through rebuttal.

Two competing explanations deserve weight before concluding that reviewers simply stopped caring.

The first is an interface failure. At the start of the window, reviewers reported that rebuttals were not visible to them, and later posts describe email notifications arriving only after authors posted comments. The organisers issued a clarification on 31 July, mid-window, stating that "Rebuttal" and "Official Comment" are equivalent and instructing authors to answer initial meta-reviews via comments so that reviewers could see them. A clarification issued four days into a seven-day window is itself evidence that the handoff was not clean.

The second is rational triage. Reviewers and an area chair argue that rebuttals frequently do not change a settled assessment, that rereading a paper properly is real work, and that this batch landed over a weekend. One chair says the papers they handled already had clear outcomes. That is the charitable reading, and it conflicts with the handbook's explicit acknowledgment requirement — and it is weakest exactly where the conference says discussion matters most, on borderline papers.

There is also a structural observation worth making, though it is an inference from policy design rather than a published finding. NeurIPS's enforcement is front-loaded: reviewer-authors lose access to their own reviews until they submit their initial reviews, and the handbook reserves desk-rejection sanctions for gross negligence. That gating creates powerful pressure to deliver phase one — and expires before the discussion window. The predicted result is precisely the reported pattern: fewer missing initial reviews, and no comparable immediate incentive for the costly second read.

Timing compounds it. ARR's August cycle opens 3 August, the exact day NeurIPS removes authors from the conversation, with reviewer registration on 5 August. ARR's May cycle already ran 17,087 submissions against 10,636 reviewers and 1,424 qualified chairs, and organisers responded by increasing reciprocal assignments and lowering the reviewer qualification threshold while warning it could harm quality. The two systems are separate, but they draw on the same finite pool of researcher attention.

One thing this is emphatically not is an AI story about machines judging science. NeurIPS is running an opt-in randomised experiment where eligible reviewer-paper assignments receive no assistance, open-ended model assistance, or structured assistance, with chairs blind to condition rating the resulting review quality — and the protocol explicitly forbids the tool from writing reviews or replacing reviewer judgement. We covered it when it launched: NeurIPS is running a randomized experiment on AI-assisted review. No evidence connects it to the silence. The uncomfortable juxtaposition is that the conference is measuring whether AI can improve human reviewing at the precise moment its human discussion layer is failing to close, and better first-pass reviews cannot make someone return a month later, digest a rebuttal and own a decision.

The necessary caveat: 2026 reviews are not public until decisions, so nobody outside the conference can measure how widespread this is. These are testimonies from participants, not a prevalence estimate, and NeurIPS has published no 2026 submission, reviewer or assignment-load figures against which to judge the strain.


Originally published on Ground Truth, where every claim is checked against the primary source.

Top comments (0)