You are the first technical hire, or the fractional CTO, or just the person in the company who has shipped something before. A founder hands you three agency proposals and asks which one. The proposals are marketing documents and contain almost no technical information.
Here is what I actually check, in order, and what each answer tells you.
1. Ask to walk one change from ticket to production
Not a demo of the product. A walkthrough of the pipeline on a project they already have running. Screen share, twenty minutes.
What you are looking for:
- Is there a CI pipeline, and does it run tests on every pull request, or is it a deploy button?
- How long does the pipeline take? Anything over fifteen minutes on a small project means people will start skipping it.
- Is there a staging environment that matches production in shape, or is staging someone's laptop?
- Are migrations automated and reversible, or does someone SSH in?
- Who approves a merge? Is there a required reviewer, or is it self-merge with a rubber stamp?
A firm that cannot produce this walkthrough within a day does not have a standard pipeline. Every project is bespoke, which means yours will be too, and the cost of that lands on you in month nine.
2. Ask who owns correctness on machine-generated code
This is the 2026 question and it separates firms more sharply than anything else on this list.
Every firm now uses assistants. The differences are entirely downstream of that. Ask:
- What does review look like when the first draft was generated? Is there a different standard than for hand-written code?
- Do they require tests written before or independently of the generated implementation? Generating both from the same prompt in the same pass gives you a test suite that agrees with the bug.
- Do they use retrieval over project-specific context — decisions, schemas, prior ADRs — or is every prompt context-free? Context-free generation across five engineers produces five inconsistent interpretations of the same domain.
- What has gone wrong with their AI workflow, and what did they change?
The last one is the tell. A firm with no failure story either has not been watching or is managing you.
3. Ask for an architecture decision record from a real project
Redacted is fine. You want to see whether they write down why, not just what.
A good ADR states the decision, the alternatives considered, the constraints that mattered, and — critically — the conditions under which the decision should be revisited. That last part is the difference between documentation and archaeology.
If they have never written one, the reasoning behind every choice in your codebase will live in one person's head, and that person will roll off your project eventually.
4. Check the exit cost before you check the entry price
This is where I have seen the most damage, and it is entirely contractual.
- IP assignment on payment, explicitly including subcontractor output. Subcontracting is normal; unassigned subcontractor IP is a live problem during diligence.
- Repositories under the client organisation from day one. Not migrated at the end. If the code lives in the vendor's GitHub org, the vendor controls the schedule of any disagreement.
- Cloud, DNS, app store and payment provider accounts in the client's name with client billing.
- A handover document as a named deliverable with acceptance criteria, not a goodwill gesture.
- No fee for handover or documentation. If there is one, the whole relationship is priced on lock-in.
5. Sanity-check the architecture proposal against the stage
A seed-stage product being sold microservices, event sourcing, a service mesh and multi-region deployment is a red flag with no innocent explanation. Either the firm is padding, or they have never built at this stage, or they are pattern-matching from enterprise work.
The correct architecture for a pre-product-market-fit company is boring: one well-structured application, one database, a background job runner, and clear internal module boundaries so you can split later if you ever need to. Anything more is paying interest on a loan you have not taken out.
Conversely, look for the things that are worth doing early and get skipped: structured logging, error tracking, a migration story, secrets not in the repository, and product event instrumentation before the fourth feature. Our notes on custom software development cover where this line sits by stage.
6. Read the estimate for the missing line items
Feature-based estimates hide the expensive parts. Check specifically for:
- Environments, CI, and deployment automation
- Third-party integrations, priced with a wider band for vendors the team has not used before
- Admin and internal tooling — someone has to refund a customer and fix bad data
- Code review time, which is now a larger share of total effort than it was
Zero review time in an estimate describes a process that does not exist.
7. Interview the actual technical lead
Not the account manager. Ask them to describe a hard technical decision on a previous project and what they would do differently now. You are listening for whether they can articulate a trade-off, and whether they have updated on anything.
Then ask: what do your engineers do when they disagree with a client decision? The answer tells you whether the founder is buying judgement or compliance. Compliance is cheaper and much worse.
The two-page report
Write it up for the founder in two pages: pipeline maturity, review discipline, exit cost, architecture fit, estimate completeness, lead quality. Score each one to five. Weight exit cost and review discipline double, because those are the two that are nearly impossible to fix later.
The lowest bid usually does not win this. That is the point.
Full-length version with engagement models, pricing bands and IP checklist: How to Choose a Software Development Company for Startups in 2026. If the product itself has an AI component, the evaluation questions in our LLM integration guide are worth adding to this list.
Frequently Asked Questions
What should I ask to see before signing with an agency?
A live walkthrough of one code change from ticket to production on an existing project, one redacted architecture decision record, and their code review policy for machine-generated code. All three are cheap for a competent firm to produce and revealing when they cannot.
How do I evaluate an agency's use of AI coding tools?
Ask who owns correctness on generated code, whether tests are written independently of the implementation, whether they use retrieval over project context, and what has gone wrong with their workflow. Independent test authorship is the strongest single signal.
What architecture is appropriate for a pre-PMF startup?
One well-structured application, one database, a job runner, and clear internal module boundaries. Microservices, event sourcing and multi-region deployment at seed stage indicate padding or pattern-matching from enterprise work.
What contractual terms matter most technically?
IP assignment including subcontractors, repositories under your own organisation from day one, infrastructure accounts in your company name, and a handover document as a named deliverable. Any fee attached to handover means the engagement is priced on lock-in.
Which line items are usually missing from agency estimates?
Environments and CI, third-party integrations, admin and internal tooling, and code review time. Review in particular is now a larger share of effort than it was, and an estimate showing none of it is describing a process that does not exist.
Top comments (0)