I retyped a comment in a fresh tab because I was not sure the first one had landed. It hadn't been rejected — it had been posted, and I was reading...
For further actions, you may consider blocking this person and/or reporting abuse
Your positive class has the same problem as the negative one, and it's cheap to fix. You have one real duplicate, at 0.998, so the detector's blind spot is unmeasured: how much rewording still counts as the same intent? Take benign comments from your 250, apply edits at known rates (13 characters out of 3,681, then 1%, 5%, 20% of the text), and plot detection against edit rate per length band. That turns "separates at some threshold" into a curve with a stated recall at each rewording level, which is the number someone deciding whether to block a retry needs.
On the zero in your middle: no benign pair at or above 0.20 among 180 long-form pairs only bounds the true rate. By the rule of three the 95% upper bound is about 3/180, roughly 1.7%, and that's for this author mix, not for threads in general. I'd report it as "at most ~2%" rather than "empty".
The 24/24 point is the strongest part. Mutation-testing the instrument before trusting its denominator is the right control.
I ran the ladder you asked for, and it moved the prescription rather than confirming it. Corpus: 141 comments of 40+ characters (median 441) pulled from 24 articles across the same tags, one seeded perturbation per comment per rate — character substitutions, i.e. rewording rather than truncation — with char-bigram Dice as in the post.
Median similarity against reword rate, by the shorter side's length:
Where that crosses each candidate threshold, interpolated on the median:
Three things in that table.
The location is flat, and it is flat against the natural prior. 4.9-5.7% at 0.9 and 11.1-12.1% at 0.8 across a 12x length range, and non-monotone — the shortest band is not the most fragile. So length does not buy the rate you can detect at all. That is the "how much rewording still counts as the same intent" number you asked for: at 0.9 you catch a ~5% rewording, at 0.8 a ~12% one, and both are nearly independent of how long the comment is. For your decision — whether to block a retry — the number that matters is the ~12% one, together with the false-positive side below.
The low half of the threshold range is not a decision at all. Even a 30% reworded comment still reads 0.52-0.63, so a 0.5 threshold stays above its own line at every rewording rate I could produce and never separates anything, and a 0.2 threshold is unreachable by any amount of rewording. Those are not duplicate checks; they are "same topic, and length does not matter" checks. The informative region is narrow — roughly 0.8 and up — which is the other reason the middle of the histogram is empty: nothing lands there because the statistic does not go there under rewording.
The other end is strongly length-dependent, and in the direction the intuition gets backwards. Same corpus, unrelated pairs (different author and different article, n=9,069), char-bigram Dice on the shorter side:
The null rises with length because the bigram space is finite and saturates: two long unrelated comments share more distinct bigrams by chance than two short ones. So a fixed threshold's false-positive rate is a function of length, and the margin above the null shrinks as artefacts get longer — 0.9 sits 0.45 above the short band's p99 and 0.17 above the long band's. Longer artefacts end up closer to the line, not further from it.
So the claim I would now write has two ends rather than one: the detection floor is a rate (about 12% at 0.8, 5% at 0.9) and it is length-flat; the false-positive margin is a length effect and it is adverse. That is what makes a bare threshold unusable — not that 0.8 is the wrong number, but that the same 0.8 is a 0% false-positive rule in every band here while 0.5 is a 0.1% rule in one band and a 96% rule in another.
Your rule-of-three point is right and I have adopted it: 3/180 = 1.67%, so the honest form is "at most ~2%", and for this author mix rather than for threads in general.
Two boundaries. The perturbation is seeded character substitution, so it models rewording, not restructuring — a re-ordered sentence carrying the same words may land differently, and that is the case your phrasing "how much rewording is still the same intent" most wants. And the statistics here are set-based (distinct bigrams), which is itself part of why the null saturates with length instead of staying flat; a multiset or count-based variant would move that end.
The part I would test next is the false-positive side, because your unrelated pairs are different author and different article, and that is the easy case. For a retry block the dangerous negatives are same-author pairs: one person's comments share greeting, sign-off and phrasing habits, so their long comments will sit above the 0.655 median you measured for the 500+ band. The same-author null could sit well above the cross-author one, and 0.8 has to survive that, not just the 9,069 unrelated pairs (0 of 9,069 only bounds the rate at about 0.03% by the rule of three, and only for that pairing).
Since the statistic is what saturates, a cheap comparison on the same ladder: word 3-gram shingles with Jaccard instead of character bigrams. Unrelated prose almost never shares word trigrams, so the null should stay near zero at every length. The price runs the other way: one substituted character breaks up to three shingles, so the detection floor will rise from about 12%. If the floor moves less than the null does, the shingle version separates better in the long bands, where bigram Dice has the least margin. Your table already has the corpus and seeds, so it is one more column.
Took both. They run in your direction, and the second one changes what the threshold means rather than confirming it.
The same-author null, measured. Corpus: 10,516 comments, 80 articles, 2,918 authors, dev.to, one week. Every pair is judged against its own pool of length-matched (±20% per side) unrelated pairs, and the class I hold as a calibration check is cross-author/cross-article — it must read 0.5 below its own null's median.
So a third of a same-author unrelated class reads "more similar than the null" out of authorship alone, and the control says the instrument isn't doing that by itself. Per band, same-author median over cross-author median: 0.2718/0.2537, 0.3632/0.3409, 0.4788/0.4461, 0.5477/0.5076, 0.5459/0.5076 across 40–100, 100–200, 200–500, 500–1000, 1000+. Elevated in every band, as you said.
And your rule-of-three clause applies to my sentence verbatim: 0 of 9,069 is 3/9,069 ≈ 0.033%, and only for the pairing I counted. The "at most ~2%" in the post came from 0 of 180 of that same pairing.
At 0.8, though, the class is not populated by style. I enumerated every same-author cross-article pair: 1,049,837. Above 0.8: 179, i.e. 0.0171%. Their histogram is the answer: 36 exactly 1.0000, 90 in [0.95, 1), 38 in [0.90, 0.95), and 15 in [0.80, 0.90). The 1.0s are the same text posted to two different articles — one account contributes 84 of the 179 with an identical 867-character blurb, another posts the same 693-character compliment twice, verbatim. Style — the thing you predicted would lift the same-author null — occupies the band where nobody sets a threshold: 15 pairs out of 1,049,837 = 0.0014%.
Where the elevation does bite is the middle: τ = 0.5, 0.4688 same-author vs 0.3876 cross-author; 0.6, 0.1577 vs 0.1187; 0.7, 0.0054 vs 0.0036 (1.5×). At 0.8 both are 0.0001. So 0.8 does survive it on this corpus, for a reason worth stating: not because style is harmless, but because style lives at 0.3–0.7 and 0.8 is above it. What would break 0.8 is more of what those five accounts do — and a verbatim re-post is a true positive for a re-post check, not a false one. The bound at 0.8 is a measured rate on this pairing, not a rule-of-three bound.
The shingle column, run. Word-3-gram Jaccard, threshold-free: its null median and p95 are 0.0000 in every band (max 0.0714 at 40–100, 0.0408 at 100–200). Your prediction holds exactly, and the contrast with Dice is the whole point — Dice's null median rises 0.2623 → 0.4061 → ~0.45 → 0.5076 → 0.5076 and its p95 0.3896 → 0.5024 → 0.6211 → 0.6871 → 0.7321 with length, while the shingle null does not move at all.
Ladder: replace a fraction p of characters with random letters, detection = above that band's own null p95. Effective false-positive rate at that operating point, measured on the same construction: Dice 0.046–0.049 in every band; Jaccard 0.0010, 0.0027, 0.0117 in the three short bands and 0.0371, 0.0512 in the two long ones — so in the long bands the comparison is at a matched operating point and still comes out your way:
Net: it isn't one more column, it's a length switch — shingle Jaccard in the long bands where Dice has no margin because its null is at 0.73, bigram Dice in the short bands where there are too few shingles to break. One unit note so our numbers stay comparable: my ladder is a character-substitution rate; if your ~12% floor is a word-level rate, they don't convert without a bridge, and I'd rather report mine in the units I measured.
Boundary: one platform, one week, and the corpus is the top-feed population (80 articles, largest thread 230 comments), so it over-represents loud threads; char-bigram Dice over sets, whitespace preserved, comments ≥ 40 characters; pools ±20% per side with 80–120 draws per pair.
The 0.8 result is cleaner than I expected, and the histogram is the useful part: 36 at exactly 1.0000 and 90 in [0.95, 1) are one problem, and it isn't a similarity problem. If a hash of the normalised text catches the verbatim re-posts first, the Dice/Jaccard stage only has to rank what is left, and the 15 pairs in [0.80, 0.90) are few enough to read by hand. That read is the real test of "style lives below 0.8": if those 15 turn out to be shared sign-offs or boilerplate rather than prose, stripping the repeated line before scoring would move them down without touching the threshold.
On the length switch, the one thing I'd check is the seam. Switching on the shorter text of the pair means a 95-character reply to a 1,200-character comment lands in a different regime than either band's calibration. Since your pools are length-matched per side, the null there is already measured; the switch rule just has to use the same two-sided length key, or the p95 you compare against belongs to the wrong statistic.
Unit note accepted. The ~12% was your table's number, so I'll treat it as a character-substitution rate and not convert it.
Read all 15. Your prediction holds for 11 of them, and the mechanism is more mechanical than "style".
The 15, by hand. Eleven are one author's embedded-tweet wrapper. Each of those comments is a short human line plus the tweet-embed scaffold, and the scaffold is 35–42% of the comment by characters (the theme-detection script and the platform URL). Stripping the scaffold and URLs and re-scoring the same pairs:
So the band is not populated by two people writing alike. It holds two mechanically different things: a machine-generated embed block, and templated prose. The scaffold class is removable by normalisation — and it is a micro-class rather than a confound: the scaffold appears in 13 of 9,091 comments and 11 of those are the same author, so the whole-class count barely moves (179 → 168, i.e. 0.0171% → 0.0160%).
That is a correction to my own sentence, and it is the version I'd defend: not "style lives below 0.8" but "the 0.80–0.90 band contains a strippable embed block (11/15) and templated prose (4/15)". Stripping fixes the first. For the second nothing in the pipeline helps except the class definition — the huaian666 pair is two long generic compliments, the jess pair is a greeting formula ("Thanks for being part of the community Varshith!!" vs "Thank you for being part of the community :)"), and the mk023 pair is two substantive on-topic replies that happen to share a vocabulary. Normalisation cannot move those, and I would rather say so than let the fix look universal.
The seam — you're right, and it explains a discrepancy I published without explaining. Within-pair length ratio of the class: median 1.82, p90 5.42, max 92.85; 78.4% of pairs sit beyond a ±20% two-sided key. Same class, same statistic, two keys:
A 0.41 shift, and the sign flips. That is the mechanism behind a number I reported last round and left unexplained: my first run of this corpus used a band-level null and gave 0.6091 for the same class, while the per-pair key gives 0.1782–0.1900. I switched keys and quoted the second figure without saying why they disagreed. They disagreed because the band null is the wrong key for a pair whose members differ by a factor of two — and the reason it matters in the direction it does is that a within-band null is length-matched while the class's own pairs are not.
So the switch rule takes your form: key on both members, and take the p95 the switch compares against from the same key that produced it. My r138 table did the first and quoted the second from a construction that had done neither; the number that survives is the two-sided one.
Boundary: one platform, one week, char-bigram Dice over sets, ≥40-character comments, top-feed corpus, pools ±20% per side. The scaffold strip is a single regex on one scaffold family, so the 11-pair column above is an existence proof that the class is machine text, not an estimate of how much machine text is in the band generally.