Over the last few years my team has owned the QA function for an edtech platform, a food delivery app, and a non-profit system. Same service model each time. Wildly different problems.
Here's what actually transferred between them, and what didn't.
The setup
In all three cases we ran managed QA, meaning we owned the whole testing function, not just supplied testers. Test strategy, the team, execution, environments, and the SLAs we were held to. Ownership sat with us, not the client.
That part stayed constant. Almost nothing else did.
Edtech: the problem was scale, not features
The edtech platform served 5,000 concurrent users, and load was the thing that broke it. Functional bugs mattered, but performance under real concurrency mattered more, because a slow platform during peak class hours is a churned customer.
What we learned:
- Load testing isn't a phase, it's a heartbeat. We ran it continuously, not once before release. The bottlenecks that mattered only showed up under sustained concurrency, not in a one-off test.
- Response time is a feature. After isolating the bottlenecks, response time dropped 65%. Nobody wrote a "make it 65% faster" ticket. It came out of treating performance as a first-class test target.
- We caught 150+ critical bugs before the first production deploy, and the ones that would have hurt most were all concurrency-related, not logic errors.
Food delivery: the problem lived in the APIs
The food delivery app's pain was different. The user-facing flow looked fine. The APIs underneath it were slow, and in a delivery app latency compounds across every order, driver update, and status change.
What we learned:
- Test where the latency compounds, not where it's visible. The UI wasn't the problem. Targeted API performance testing was, and it drove a 60% latency reduction.
- Coverage numbers only mean something when they're on the right surface. We pushed test coverage to 95%, but the number mattered because it was concentrated on the ordering and delivery paths, not spread thin across the whole app.
- Mobile plus API is a different testing discipline than web. Treating it like web testing would have missed the defects that actually cost money.
Non-profit: the problem was permissions
The non-profit platform had almost no performance pressure. Its risk was entirely in access control. 900 role-based permission configurations, and every wrong combination was a data exposure waiting to happen.
What we learned:
- Some systems are all edge cases. There was no "happy path" worth much here. The whole risk surface was permission combinations, so that's where the entire effort went.
- Systematic beats clever. We tested all 900 configurations methodically and resolved every one before launch. No shortcut would have been safe.
- Low-traffic doesn't mean low-risk. This was the lowest-load system of the three and by far the highest-stakes per defect.
What actually transferred
Three engagements, three completely different failure modes. The thing that carried across all of them wasn't a tool or a test type. It was owning the risk model.
- The risk lives somewhere different every time. Scale, then APIs, then permissions. Walking in with a fixed checklist would have missed the real threat in at least two of the three.
- Match the testing to the risk, not to a template. The edtech playbook would have been useless for the non-profit. The value was figuring out where each system actually breaks, then pointing everything there.
- Ownership is what makes that possible. When you own the outcome instead of a task list, you're free to put the effort where the risk is. That's the whole argument for managed QA over just renting testers.
If you're weighing managed QA against staff augmentation or an in-house build, that ownership question is the one that decides whether it works. I wrote up the full breakdown of the models, the SLAs worth demanding, and when to skip managed QA entirely in a complete guide to managed QA services.
Top comments (0)