DEV Community: Martin Yovchevski

A similarity score is a starting point, not a verdict

Martin Yovchevski — Sun, 01 Nov 2026 19:26:30 +0000

When a document-review tool reports a percentage, it is tempting to treat that number as the answer. But a similarity score can only describe matches found under a particular set of rules and against a particular collection of sources. Whether a passage is properly attributed, common terminology, a quotation, or a problem needs context.

I work on Unplag.bg, a Bulgarian platform for text review and academic integrity. One design question matters to me: how can a report make the next human decision easier without pretending to make that decision itself?

Start with the evidence

A useful report should let a reviewer move from an overall figure to the actual matched passages and their sources. Seeing the surrounding text helps distinguish a referenced quotation from a passage whose attribution needs further review. A source link is useful only if the reviewer can inspect the relevant material.

Make the boundaries visible

A report is shaped by the sources available to the system, the text extracted from the document and the matching rules used. No match is not proof that every possible source has been checked. Likewise, a match is not proof of misconduct. Where possible, a report should explain which sources were checked and note extraction problems instead of hiding those limitations behind one number.

Leave room for human judgement

A teacher may need to consider the assignment, quotation conventions and an opportunity for the student to explain their work. A responsible workflow supports that review and documents its reasoning. It does not equate text similarity with plagiarism or a potential AI signal with proof of authorship.

For developers, this means thinking beyond a single threshold: show the evidence, communicate limitations, and make it possible to review context. These are design principles, not claims about the accuracy of any particular system.

Generated with AI based on the author’s topic and instructions; not reviewed by the author before publication. I am Martin Yovchevski (Мартин Йовчевски), co-founder and technical lead of Unplag.bg.

Three text-processing tests to run before comparing Bulgarian documents

Martin Yovchevski — Sun, 01 Nov 2026 18:20:20 +0000

When a comparison system misses a passage, the matching algorithm is not the only place to look. It may be comparing a different sequence of characters from the one a reader sees. Before interpreting a similarity result, test what happens to the text on its way into the matcher.

I work on Unplag.bg as a co-founder and technical lead. The examples below are general test ideas, not a description of Unplag's internal implementation or measurements of its performance.

1. Equivalent Unicode representations

The Cyrillic letter й can be represented as a single code point or as и followed by a combining breve. They can look the same while a direct string comparison returns false. Normalizing both strings to NFC before comparison is one possible policy:

import unicodedata

composed = "й"
decomposed = "и\u0306"
assert composed != decomposed
assert unicodedata.normalize("NFC", composed) == unicodedata.normalize("NFC", decomposed)

This is a test for equivalent encodings, not a general spelling correction. The normalization policy should be documented and applied consistently to both sides. Keep the original text for display and check that highlights can be mapped back to it.

2. Lookalikes across writing systems

Latin a and Cyrillic а look similar but are different characters. The word папа written entirely in Cyrillic is not the same character sequence as пaпa, where the two a-shaped characters are Latin. A test suite should include both. A warning about mixed scripts can help a reviewer notice unexpected input; it is not evidence of deliberate manipulation. Mixed scripts occur legitimately too, for example in product names and quotations. Avoid silently replacing every lookalike: that can change meaning and create false matches. Unicode's security guidance discusses confusables and their limitations.

3. Extraction before matching

Use a small set of documents with a known reference transcription: a digital PDF, a scan requiring OCR, and a DOCX file. Compare the extracted text with the reference and inspect whitespace, punctuation, page breaks, hyphenation and Cyrillic characters. A failure here should be visible to a reviewer; it should not quietly become a low similarity score. Include examples with correctly attributed quotations and common phrases, because a match still requires contextual interpretation.

These checks cover input quality, not the accuracy of a full plagiarism or authorship assessment. A useful report should show the matching passages and sources, and leave the final judgment to people following the institution's procedure.

References

Disclosure: this article was generated with AI assistance for Martin Yovchevski's professional profile. Martin is co-founder of Unplag.bg. It describes general test ideas, not results from an experiment.

Designing text-review tools around evidence

Martin Yovchevski — Sun, 01 Nov 2026 18:18:31 +0000

A similarity score is easy to display. A review interface is harder to design: it must help a person find relevant evidence, understand its limits, and make a decision that can be explained later.

At Unplag.bg, where I work as co-founder and technical lead, this question matters because educational institutions need more than a single number. Here are practical design questions I consider useful when building systems for reviewing documents.

Keep the source close to the match

A highlighted passage should make it clear what was compared and where the corresponding source can be found. The reviewer needs enough surrounding text to understand a quotation, a reference, or a common phrase. A link to the source is a starting point for inspection, not proof that the passage is inappropriate.

Preserve the original and expose extraction problems

Document processing starts before comparison. PDF extraction, formatting, and character encodings can change what a system reads. For Bulgarian, Cyrillic text, mixed scripts, and word forms add practical cases to test. A reviewer should be able to return to the original document when extracted text looks wrong.

Separate different kinds of evidence

Text overlap and a signal about possible AI-generated writing answer different questions. Neither should be presented as a definitive conclusion about intent or authorship. Label what each signal measures, explain its limitations, and keep decisions with the people responsible for review.

Make the review understandable later

A useful workflow lets an authorised reviewer trace the relevant passages, sources and decisions. This supports consistency without pretending that every case has the same rules. Institutions can then apply their own policies and give students a fair opportunity to respond.

Designing for context is a product choice: it makes the limits of automation visible and gives educators a better basis for judgment.

Originally published on my Hashnode blog. Prepared with AI assistance. I am co-founder of Unplag.bg.