Scaling "Sage": Building Nepal's First Home-Grown Anti-Plagiarism System

Albert Maharjan — Tue, 14 Jul 2026 06:39:34 +0000

Date: 2026-07-14
Project: Sage Anti-Plagiarism System, Nepal's first in-house anti-plagiarism project, built at Softwarica College of IT & E-Commerce for academic integrity.
System: plagiarism similarity search over a ~48.8M-sentence embedding corpus
Vector engine: Qdrant 1.18.2 (384-dim embeddings, cosine distance, 8 shards, 52 segments)

TL;DR

Over one year, report generation went from 62.5 s to 4.67 s (13.4× faster) while RAM fell from 40–60 GB to 11 GB. The decisive release (v4) put a 4-bit-quantized index and the navigation graph in RAM at the same time, so it got faster and lighter together. A three-regime load test then found a ceiling of about 1,300 queries/second, degrading gracefully with zero errors at 10,000 concurrent queries, and about 90 ms of real-user latency for a 100-sentence submission. The 4-bit compression costs almost nothing: 99% verdict agreement and 0.3% score error versus full precision (§4).

1. Objective

Two questions about the production similarity-search backend:

How hard can it be pushed? Find the throughput ceiling and confirm it degrades gracefully rather than failing under load.
What does a real user wait? Measure the search latency of one submission with no contention.

The report also records how the system reached its current performance over the past year, so the stress-test numbers can be read against that trajectory.

2. One year of optimization evolution

Benchmarked on a fixed reference submission of 508 sentence embeddings (4,495 words), measuring end-to-end plagiarism-report time.

Version	Date	Key change	RAM	Report time	Speedup vs v2
v2	Jul 28, 2025	Migrated Milvus to Qdrant	40 GB	62.53 s	1.0×
v3	Mar 26, 2026	8-shard collection	60 GB	37.41 s	1.7×
v4	Jun 10, 2026	TurboQuant 4-bit + HNSW graph in RAM	11 GB	5.01 s	12.5×
v5	Jul 13, 2026	Parallelized search operation	11 GB	4.67 s	13.4×

v2 to v3: 8 shards let queries fan out in parallel and cut report time about 40%, but holding more index resident pushed RAM to 60 GB, the wrong direction for scaling.
v3 to v4: compressing vectors to 4 bits and pinning both the vectors and the navigation graph in RAM gave a 7.5× speedup and an 82% RAM reduction together (60 GB to 11 GB). Architecture in §3.
v4 to v5: parallelizing search removed the last 7% at no RAM cost. This stress test exercises the search component that v5 parallelized. It measures that component's throughput and latency, not full end-to-end report time (scope in §5).

Read the trajectory as a real-world improvement record, not a controlled ablation. (1) v4 changed two things at once, 4-bit quantization and graph-in-RAM, so no single release's gain is attributable to one change. (2) The corpus grew over the year, so the 13.4× figure blends optimization with a changing data baseline.

3. Current architecture: TurboQuant and HNSW-in-RAM together

At full precision the vectors are too large to hold in RAM. The v4/v5 design makes search fast anyway by running two optimizations at once, arranged as three memory tiers. (TurboQuant is a rotation-based, training-free quantization shipped in Qdrant 1.18; 4-bit gives 8× compression at 1–2 pp recall versus scalar quant.)

Tier 1, navigation graph, in RAM. A hierarchical proximity graph (32 edges per node, built with a 512-neighbour construction pass) decides which vectors to examine. Candidate selection is a pure in-memory operation with no disk access.
Tier 2, 4-bit vectors, in RAM. Every vector is compressed from 32-bit to 4 bits per dimension and pinned in RAM, then used to score candidates during traversal. These are fast approximate distances with no disk on the hot path.
Tier 3, full-precision vectors, on disk. The exact 32-bit vectors are read only to rescore the handful of top candidates, restoring full accuracy without paying RAM for them.

4-bit:   48.8M × 384 × 4 bits   ≈  9.4 GB  →  ~10 GB resident
f32:     48.8M × 384 × 4 bytes  ≈  75 GB   (84.8 GB measured on disk, incl. overhead)
                                          ─────────────
                                          ~8× smaller in RAM

Why both, not either

	TurboQuant only	HNSW-in-RAM only	Both (current)
Choose candidates	graph on disk, slow	RAM, fast	RAM, fast
Score candidates	RAM (4-bit), fast	full vectors on disk, slow	RAM (4-bit), fast
Final accuracy	approximate	full	full (disk rescore)
RAM footprint	vectors + graph on disk	graph + ~85 GB (won't fit)	~11.5 GB total

The whole hot loop, candidate selection and candidate scoring, runs in RAM inside about 11.5 GB, while the bulky exact vectors stay on disk and are touched only for the final rescore. That is what let v4 be 7.5× faster and 5.5× lighter than v3 at the same time. If either tier lived on disk, the hot loop would hit disk on every query and latency would be much worse.

Live memory footprint

	Disk	RAM
Vectors	84.8 GB (full precision)	10.0 GB (4-bit)
Index / id tracking	~1.0 GB	~1.7 GB
Payload	1.7 GB	negligible (on disk)
Total	87.4 GB	~11.5 GB

The 4-bit approximation's accuracy is validated separately in §4 (99% verdict agreement, 0.3% score error), so the compression is effectively free at the 0.80 threshold.

4. Search quality: does 4-bit quantization hurt accuracy?

A compressed search is only useful if it returns the right matches. The 4-bit retrieval quality was measured against an f32 near-exact ground truth: the same in-RAM graph searched with quantization ignored at deep breadth (ef 256, 4× production's ef 64), on the live collection, read-only. 100 sampled real queries, top_k up to 10.

The right metric. Plagiarism detection does not depend on which exact corpus point is returned. The corpus is dense with near-duplicate sentences, so many points sit at nearly identical similarity. What matters is whether the similarity score is preserved and whether it crosses the 0.80 cosine threshold. Cosine 0.998 and 0.999 give the same verdict, and which one wins is irrelevant.

Metric	Value	Meaning
Threshold agreement (same flag @ 0.80)	0.99	Same plagiarism verdict on 99% of queries
Top-1 similarity error (mean \|Δcosine\|)	0.0033	Best-match score within 0.3% of exact
Prod's #1 is a true top-10 neighbour	0.89	Genuinely close match about 9 times in 10
recall@1 (exact same point id)	0.52	Confounded by ties, see below
recall@10 (id set overlap)	0.74	Confounded by ties, see below

Strict ID recall is misleading here. recall@1 = 0.52 looks low, but it is a near-duplicate measurement artifact. When the true neighbour is at 0.9991 and the runner-up at 0.9989, 4-bit resolution cannot separate them, so it returns the "wrong" id while returning a point that is equally similar. The two report-governing metrics, 99% threshold agreement and 0.3% score error, are the ones that matter.

Conclusion: the 4-bit compression is effectively free for this workload. The v4 speed-and-memory win costs no meaningful accuracy at the 0.80 threshold.

Scope. Measured on vector-search top-1 agreement as a proxy for the production recommend-by-id operation (same index, same quantization). The ground truth is near-exact (deep-ef graph, not a true full scan), so it may marginally overstate agreement, though at 99% and 0.3% the margin is not decision-relevant. Recall was not re-measured across submission types or thresholds other than 0.80.

5. Methodology

Scope. This test isolates the vector-search operation only, not full report generation (which also downloads and converts the file, splits sentences, embeds, inserts, runs AI-text detection, and runs a reverse check). Search is about 0.39 s of the 4.67 s total (§7), so these numbers bound a sub-10% slice of end-to-end time: the search tier's capacity, not the whole pipeline.

The load test drives the exact production search operation, a batched "find the most similar sentences to these" request. Each request carries real sentence identifiers drawn from the live corpus and asks for the single nearest match per sentence, identical in shape to how a submission is scored.

The harness:

Samples real sentence identifiers into a reuse pool. The recommend-style query needs existing corpus points, not synthetic vectors.
Issues batched search requests, each covering a configurable batch. One request equals one submission-shaped scoring call.
Runs requests concurrently across a configurable worker count.
Records per-request latency and reports throughput two ways, requests/second and queries/second (requests × batch), plus p50/p95/p99/max.

#	Workers	Batch (sentences/req)	Total requests	Intent
1	1	100	20	True single-submission latency, no contention
2	55	32	1000	Moderate concurrent load
3	100	100	1000	Deliberate overload

Limitations. Each config ran once, with no repeats and no warm-up, so cold vs warm-cache state is uncontrolled and single-sample p99/max should be read as indicative, not precise. Numbers reflect this host and corpus state on the test date.

6. Results

Workers	Batch	Queries/s	Requests/s	p50/req	p95	p99	max
1	100	1055	10.6	90 ms	200 ms	200 ms	200 ms
55	32	1295	40.5	1319 ms	1645 ms	1945 ms	2311 ms
100	100	1339	13.4	7358 ms	8276 ms	9135 ms	10090 ms

Zero errors across all runs, including 10,000 concurrent queries in flight (run 3). The collection stayed healthy throughout.

The throughput ceiling is about 1,300 queries/second. It stays flat (1295 to 1339) while workers and batch swing 2 to 3×, which is the mark of a hard server-side capacity wall. Per-query cost is 1/1300, about 0.77 ms.

7. Analysis

Saturation-bound. Past the ceiling, throughput holds constant while latency grows linearly with concurrency, the behaviour of a saturated queue. Little's Law (throughput ≈ concurrency ÷ mean latency) holds on every run:

Run	Check	Predicted req/s	Measured
55 × 32	55 ÷ 1.298 s	42.4	40.5 ✓
100 × 100	100 ÷ 6.926 s	14.4	13.4 ✓

The agreement confirms the server is the bottleneck, not the client and not the network. Adding workers past the ceiling buys no throughput, only queue.

Latency is driven by submission size, then by backlog. At a fixed 0.77 ms/query, submission latency = (sentence count × 0.77 ms) + queue wait.

100 sentences, no contention: 77 ms + overhead, about 90 ms measured ✓.
508-sentence reference: about 0.39 s of pure search, a small slice of the 4.67 s report.
Run 3's 7.4 s p50 is not "a submission takes 7 s". It is 100 simultaneous 100-sentence submissions (10,000 queries) draining through a 1,300/s pipe, so about 10,000 ÷ 1,300 ≈ 7.5 s to clear the batch. Self-inflicted overload, not a real scenario.

Degradation is safe. No errors, timeouts, or crashes from 1 to 10,000 concurrent queries. The system slows down but never fails. RAM stays flat under load: the hot tiers are fixed-size and pinned, so concurrency spends CPU, search threads, and the disk-rescore path, not memory. This is why v5's parallelization added throughput without moving the 11 GB footprint.

8. Capacity model and recommendations

Capacity: about 1,300 queries/second. Divide by average submission size for the sustainable rate:

Avg submission size	Sustained submissions/s before queueing
50 sentences	~26
100 sentences	~13
508 sentences (reference)	~2.6

Latency: about 0.77 ms/sentence plus queue. Sub-100 ms for a 100-sentence document until sustained load nears the ceiling, then linear growth with backlog.

Operating guidance:

Real traffic is almost certainly far below 1,300 q/s, so no action is needed today.
Keep sustained concurrency where p99 stays acceptable. The 55-worker point was already into queue territory (p99 about 1.9 s).
If large-submission latency ever hurts UX and the server is not saturated, split one submission into several concurrent smaller requests. Same server cost, lower wall-clock.

Levers to raise the ceiling (the cheap wins, quantization, HNSW-in-RAM, sharding, and parallelization, are already spent):

Cut disk-rescore cost. The one remaining per-query disk touch. Faster storage, or tuning rescore oversampling (fewer f32 reads per query for a small recall trade), is the highest-impact lever now.
Scale out. Add replicas or a second node. Replication is currently single-copy.
Narrow search breadth. Fewer candidates per query: faster, small recall cost, no re-indexing.
More RAM. Holding f32 vectors resident would remove the rescore hit, but about 85 GB will not fit the 11.5 GB envelope. That is a hardware decision, not a tuning one.

DEV Community: Albert Maharjan