Date: 2026-07-14
Project: Sage Anti-Plagiarism System, Nepal's first in-house anti-plagiarism project, built at Softwarica College of IT & E-Commerce for academic integrity.
System: plagiarism similarity search over a ~48.8M-sentence embedding corpus
Vector engine: Qdrant 1.18.2 (384-dim embeddings, cosine distance, 8 shards, 52 segments)
TL;DR
Over one year, report generation went from 62.5 s to 4.67 s (13.4× faster) while RAM fell from 40–60 GB to 11 GB. The decisive release (v4) put a 4-bit-quantized index and the navigation graph in RAM at the same time, so it got faster and lighter together. A three-regime load test then found a ceiling of about 1,300 queries/second, degrading gracefully with zero errors at 10,000 concurrent queries, and about 90 ms of real-user latency for a 100-sentence submission. The 4-bit compression costs almost nothing: 99% verdict agreement and 0.3% score error versus full precision (§4).
1. Objective
Two questions about the production similarity-search backend:
- How hard can it be pushed? Find the throughput ceiling and confirm it degrades gracefully rather than failing under load.
- What does a real user wait? Measure the search latency of one submission with no contention.
The report also records how the system reached its current performance over the past year, so the stress-test numbers can be read against that trajectory.
2. One year of optimization evolution
Benchmarked on a fixed reference submission of 508 sentence embeddings (4,495 words), measuring end-to-end plagiarism-report time.
| Version | Date | Key change | RAM | Report time | Speedup vs v2 |
|---|---|---|---|---|---|
| v2 | Jul 28, 2025 | Migrated Milvus to Qdrant | 40 GB | 62.53 s | 1.0× |
| v3 | Mar 26, 2026 | 8-shard collection | 60 GB | 37.41 s | 1.7× |
| v4 | Jun 10, 2026 | TurboQuant 4-bit + HNSW graph in RAM | 11 GB | 5.01 s | 12.5× |
| v5 | Jul 13, 2026 | Parallelized search operation | 11 GB | 4.67 s | 13.4× |
- v2 to v3: 8 shards let queries fan out in parallel and cut report time about 40%, but holding more index resident pushed RAM to 60 GB, the wrong direction for scaling.
- v3 to v4: compressing vectors to 4 bits and pinning both the vectors and the navigation graph in RAM gave a 7.5× speedup and an 82% RAM reduction together (60 GB to 11 GB). Architecture in §3.
- v4 to v5: parallelizing search removed the last 7% at no RAM cost. This stress test exercises the search component that v5 parallelized. It measures that component's throughput and latency, not full end-to-end report time (scope in §5).
Read the trajectory as a real-world improvement record, not a controlled ablation. (1) v4 changed two things at once, 4-bit quantization and graph-in-RAM, so no single release's gain is attributable to one change. (2) The corpus grew over the year, so the 13.4× figure blends optimization with a changing data baseline.
3. Current architecture: TurboQuant and HNSW-in-RAM together
At full precision the vectors are too large to hold in RAM. The v4/v5 design makes search fast anyway by running two optimizations at once, arranged as three memory tiers. (TurboQuant is a rotation-based, training-free quantization shipped in Qdrant 1.18; 4-bit gives 8× compression at 1–2 pp recall versus scalar quant.)
- Tier 1, navigation graph, in RAM. A hierarchical proximity graph (32 edges per node, built with a 512-neighbour construction pass) decides which vectors to examine. Candidate selection is a pure in-memory operation with no disk access.
- Tier 2, 4-bit vectors, in RAM. Every vector is compressed from 32-bit to 4 bits per dimension and pinned in RAM, then used to score candidates during traversal. These are fast approximate distances with no disk on the hot path.
- Tier 3, full-precision vectors, on disk. The exact 32-bit vectors are read only to rescore the handful of top candidates, restoring full accuracy without paying RAM for them.
4-bit: 48.8M × 384 × 4 bits ≈ 9.4 GB → ~10 GB resident
f32: 48.8M × 384 × 4 bytes ≈ 75 GB (84.8 GB measured on disk, incl. overhead)
─────────────
~8× smaller in RAM
Why both, not either
| TurboQuant only | HNSW-in-RAM only | Both (current) | |
|---|---|---|---|
| Choose candidates | graph on disk, slow | RAM, fast | RAM, fast |
| Score candidates | RAM (4-bit), fast | full vectors on disk, slow | RAM (4-bit), fast |
| Final accuracy | approximate | full | full (disk rescore) |
| RAM footprint | vectors + graph on disk | graph + ~85 GB (won't fit) | ~11.5 GB total |
The whole hot loop, candidate selection and candidate scoring, runs in RAM inside about 11.5 GB, while the bulky exact vectors stay on disk and are touched only for the final rescore. That is what let v4 be 7.5× faster and 5.5× lighter than v3 at the same time. If either tier lived on disk, the hot loop would hit disk on every query and latency would be much worse.
Live memory footprint
| Disk | RAM | |
|---|---|---|
| Vectors | 84.8 GB (full precision) | 10.0 GB (4-bit) |
| Index / id tracking | ~1.0 GB | ~1.7 GB |
| Payload | 1.7 GB | negligible (on disk) |
| Total | 87.4 GB | ~11.5 GB |
The 4-bit approximation's accuracy is validated separately in §4 (99% verdict agreement, 0.3% score error), so the compression is effectively free at the 0.80 threshold.
4. Search quality: does 4-bit quantization hurt accuracy?
A compressed search is only useful if it returns the right matches. The 4-bit retrieval quality was measured against an f32 near-exact ground truth: the same in-RAM graph searched with quantization ignored at deep breadth (ef 256, 4× production's ef 64), on the live collection, read-only. 100 sampled real queries, top_k up to 10.
The right metric. Plagiarism detection does not depend on which exact corpus point is returned. The corpus is dense with near-duplicate sentences, so many points sit at nearly identical similarity. What matters is whether the similarity score is preserved and whether it crosses the 0.80 cosine threshold. Cosine 0.998 and 0.999 give the same verdict, and which one wins is irrelevant.
| Metric | Value | Meaning |
|---|---|---|
| Threshold agreement (same flag @ 0.80) | 0.99 | Same plagiarism verdict on 99% of queries |
| Top-1 similarity error (mean |Δcosine|) | 0.0033 | Best-match score within 0.3% of exact |
| Prod's #1 is a true top-10 neighbour | 0.89 | Genuinely close match about 9 times in 10 |
| recall@1 (exact same point id) | 0.52 | Confounded by ties, see below |
| recall@10 (id set overlap) | 0.74 | Confounded by ties, see below |
Strict ID recall is misleading here. recall@1 = 0.52 looks low, but it is a near-duplicate measurement artifact. When the true neighbour is at 0.9991 and the runner-up at 0.9989, 4-bit resolution cannot separate them, so it returns the "wrong" id while returning a point that is equally similar. The two report-governing metrics, 99% threshold agreement and 0.3% score error, are the ones that matter.
Conclusion: the 4-bit compression is effectively free for this workload. The v4 speed-and-memory win costs no meaningful accuracy at the 0.80 threshold.
Scope. Measured on vector-search top-1 agreement as a proxy for the production recommend-by-id operation (same index, same quantization). The ground truth is near-exact (deep-ef graph, not a true full scan), so it may marginally overstate agreement, though at 99% and 0.3% the margin is not decision-relevant. Recall was not re-measured across submission types or thresholds other than 0.80.
5. Methodology
Scope. This test isolates the vector-search operation only, not full report generation (which also downloads and converts the file, splits sentences, embeds, inserts, runs AI-text detection, and runs a reverse check). Search is about 0.39 s of the 4.67 s total (§7), so these numbers bound a sub-10% slice of end-to-end time: the search tier's capacity, not the whole pipeline.
The load test drives the exact production search operation, a batched "find the most similar sentences to these" request. Each request carries real sentence identifiers drawn from the live corpus and asks for the single nearest match per sentence, identical in shape to how a submission is scored.
The harness:
- Samples real sentence identifiers into a reuse pool. The recommend-style query needs existing corpus points, not synthetic vectors.
- Issues batched search requests, each covering a configurable batch. One request equals one submission-shaped scoring call.
- Runs requests concurrently across a configurable worker count.
- Records per-request latency and reports throughput two ways, requests/second and queries/second (requests × batch), plus p50/p95/p99/max.
| # | Workers | Batch (sentences/req) | Total requests | Intent |
|---|---|---|---|---|
| 1 | 1 | 100 | 20 | True single-submission latency, no contention |
| 2 | 55 | 32 | 1000 | Moderate concurrent load |
| 3 | 100 | 100 | 1000 | Deliberate overload |
Limitations. Each config ran once, with no repeats and no warm-up, so cold vs warm-cache state is uncontrolled and single-sample p99/max should be read as indicative, not precise. Numbers reflect this host and corpus state on the test date.
6. Results
| Workers | Batch | Queries/s | Requests/s | p50/req | p95 | p99 | max |
|---|---|---|---|---|---|---|---|
| 1 | 100 | 1055 | 10.6 | 90 ms | 200 ms | 200 ms | 200 ms |
| 55 | 32 | 1295 | 40.5 | 1319 ms | 1645 ms | 1945 ms | 2311 ms |
| 100 | 100 | 1339 | 13.4 | 7358 ms | 8276 ms | 9135 ms | 10090 ms |
Zero errors across all runs, including 10,000 concurrent queries in flight (run 3). The collection stayed healthy throughout.
The throughput ceiling is about 1,300 queries/second. It stays flat (1295 to 1339) while workers and batch swing 2 to 3×, which is the mark of a hard server-side capacity wall. Per-query cost is 1/1300, about 0.77 ms.
7. Analysis
Saturation-bound. Past the ceiling, throughput holds constant while latency grows linearly with concurrency, the behaviour of a saturated queue. Little's Law (throughput ≈ concurrency ÷ mean latency) holds on every run:
| Run | Check | Predicted req/s | Measured |
|---|---|---|---|
| 55 × 32 | 55 ÷ 1.298 s | 42.4 | 40.5 ✓ |
| 100 × 100 | 100 ÷ 6.926 s | 14.4 | 13.4 ✓ |
The agreement confirms the server is the bottleneck, not the client and not the network. Adding workers past the ceiling buys no throughput, only queue.
Latency is driven by submission size, then by backlog. At a fixed 0.77 ms/query, submission latency = (sentence count × 0.77 ms) + queue wait.
- 100 sentences, no contention: 77 ms + overhead, about 90 ms measured ✓.
- 508-sentence reference: about 0.39 s of pure search, a small slice of the 4.67 s report.
- Run 3's 7.4 s p50 is not "a submission takes 7 s". It is 100 simultaneous 100-sentence submissions (10,000 queries) draining through a 1,300/s pipe, so about 10,000 ÷ 1,300 ≈ 7.5 s to clear the batch. Self-inflicted overload, not a real scenario.
Degradation is safe. No errors, timeouts, or crashes from 1 to 10,000 concurrent queries. The system slows down but never fails. RAM stays flat under load: the hot tiers are fixed-size and pinned, so concurrency spends CPU, search threads, and the disk-rescore path, not memory. This is why v5's parallelization added throughput without moving the 11 GB footprint.
8. Capacity model and recommendations
Capacity: about 1,300 queries/second. Divide by average submission size for the sustainable rate:
| Avg submission size | Sustained submissions/s before queueing |
|---|---|
| 50 sentences | ~26 |
| 100 sentences | ~13 |
| 508 sentences (reference) | ~2.6 |
Latency: about 0.77 ms/sentence plus queue. Sub-100 ms for a 100-sentence document until sustained load nears the ceiling, then linear growth with backlog.
Operating guidance:
- Real traffic is almost certainly far below 1,300 q/s, so no action is needed today.
- Keep sustained concurrency where p99 stays acceptable. The 55-worker point was already into queue territory (p99 about 1.9 s).
- If large-submission latency ever hurts UX and the server is not saturated, split one submission into several concurrent smaller requests. Same server cost, lower wall-clock.
Levers to raise the ceiling (the cheap wins, quantization, HNSW-in-RAM, sharding, and parallelization, are already spent):
- Cut disk-rescore cost. The one remaining per-query disk touch. Faster storage, or tuning rescore oversampling (fewer f32 reads per query for a small recall trade), is the highest-impact lever now.
- Scale out. Add replicas or a second node. Replication is currently single-copy.
- Narrow search breadth. Fewer candidates per query: faster, small recall cost, no re-indexing.
- More RAM. Holding f32 vectors resident would remove the rescore hit, but about 85 GB will not fit the 11.5 GB envelope. That is a hardware decision, not a tuning one.
Top comments (0)