DEV Community: Sonal Jain

What a Maintenance Retainer Should Actually Cover

Sonal Jain — Tue, 15 Sep 2026 08:00:22 +0000

A maintenance retainer should cover four things in writing: keeping the system running, keeping it patched, a fixed pool of small changes, and a named person accountable when it breaks. Most retainers I have inherited covered the first one loosely and left the rest to goodwill, and the arguments started in month three when a "small change" turned out to be a week and a security patch turned out to be nobody's job.

What does "keeping it running" actually mean?

More than uptime. It means monitoring that a person actually reads, backups restored on a schedule, certificate renewals, hosting bills reviewed monthly, disk and queue alerts wired to a human, and a log of every deploy. Those tasks belong in a written checklist with a cadence beside each one.

A backup that has never been restored is a hope with a file name. I ask every team taking on a retainer to run one restore test in the first month, and I have lost count of the times that test found a backup job which had been silently failing for weeks.

Vague retainers get resented from both sides: the client feels they are paying for nothing, and the team feels they are doing invisible work. The checklist settles both, because it shows what "running" costs and what it includes.

Who pays for the updates nobody planned?

The retainer, if it is written properly. Framework upgrades, library patches, operating system updates and third-party endpoints being deprecated are not features and not optional. A system left alone for a year has quietly become an older system with known holes in it.

AI features add a second list. Model versions get retired, prompts tuned over months start behaving differently, and token costs creep as usage grows. That upkeep is part of the real bill of automation, so every AI integration we hand over now leaves with a line in the retainer covering it, and a monthly review of the AI spend next to the hosting spend. A client who discovers that drift from their own finance team is a client who thinks the feature is broken.

How big is the small-change pool, and what counts as small?

A monthly block of hours, with unused hours rolling over one month and no further. Small means a change one person can finish and test in under a day, with no new screens and no data model changes. Anything larger is a change request with its own estimate.

Counts as small, inside the pool	Counts as a change request
Reword an invoice email template	A new approval step with its own screen
Add a column to an existing report	A report that needs a new field in the data model
Fix a call to a third-party endpoint that moved	Swapping the third-party provider entirely
Apply a security patch and redeploy	Migrating to a new framework major version

The rollover rule stops the client paying for air and stops the team carrying a growing debt. The definition matters more than the number of hours. Most retainer disputes I have mediated were about whether something counted as maintenance, and every one of them would have been settled by a sentence written before the contract was signed.

What does accountable actually look like?

A named person, response times by severity, and a monthly report. Severity one, meaning the system is down or data is at risk, gets a human within an agreed window around the clock. Severity two, a broken feature with a workaround, gets a response within a working day. Everything else joins the queue.

The named person changes as people change, but there is always one, and the client always knows who. The monthly report is thirty minutes and one page: what broke, what we patched, hours used from the pool, the bills, and anything we recommend before it becomes urgent. It is the least glamorous meeting on my calendar and the one that most reliably turns a maintenance client into a client for the next build.

The first month of any retainer I take on runs in the same order:

Run one restore test from a real backup, before anything else is agreed.
Write the running checklist, with a cadence beside every task.
Put the small-change definition into the contract, in one sentence.
Name the accountable person and the severity response windows.
Book the monthly report meeting for the whole year, in advance.

At Shanti Infosoft I would rather lose a retainer negotiation over the pool definition than win one that both sides misunderstand. The misunderstanding always costs more than the hours. If yours is up for renewal and you want those four sections read by someone who has argued about all of them, my calendar is here.

What does your current maintenance agreement say happens when a library you depend on stops being supported?

Sonal Jain is delivery lead at Shanti Infosoft, a CMMI Level 5 software firm that has supported systems for 700+ companies.

Sharing a Codebase With the Client's In-House Team

Sonal Jain — Sun, 13 Sep 2026 09:00:45 +0000

When a client's own developers work in the same codebase as ours, the project succeeds or fails on agreements made in the first week: whose conventions apply, who reviews whose work, who owns each environment, and how a disagreement gets settled. Skip those and you get two polite teams quietly breaking each other's work and blaming the other one in private.

Whose rules apply?

The client's, if they have any. It is their codebase and they will live in it long after the vendor leaves, so we adopt their branch strategy, their linting, their pull request template and their release process. Where they have no rules, we propose ours and write them down as shared rules.

Setting that up takes about a day:

Sixty minutes walking through their PR process, led by whoever actually merges.
A list of who can merge to the main branch. It is usually shorter than anyone thought, and usually one person who is also the busiest on their team.
A written note of which conventions we are adopting, sent to both teams the same afternoon.
Agreement on review turnaround, in hours, before the first pull request is opened.

From the day that note goes out they are the shared rules rather than the vendor's rules, which is a small wording difference that changes how people argue about them later.

Who reviews whose work?

Both directions, always, with an agreed turnaround, and twenty-four working hours is the number I usually push for. The common failure is lopsided review, where the client's team reviews everything the vendor ships and nobody reviews what the client's team ships, usually out of politeness.

That politeness is expensive. It means half the codebase is unreviewed and the other half is slow. Reviews across the fence also do something no document can: they transfer knowledge in both directions, every day, without a handover meeting.

What about AI-generated code from either side?

It gets the same rule as everything else, agreed in week one. Both teams are using coding assistants and both are shipping faster, so review capacity becomes the constraint. Every change gets a human reader who did not write it, whatever produced the first draft.

If neither side's review process has caught up with that speed, you end up sharing a growing pile of code nobody has read closely, and the invoice for that arrives as the security bill for vibe-coding, typically after launch, addressed to whoever is holding the codebase at the time.

So anything touching authentication, payments, permissions or personal data gets a second reader as well. Same rule for both teams, which is what makes it enforceable. I raise it in week one now, rather than discovering the gap in month three.

How do you handle the human side?

Four agreements: one backlog so there is a single definition of priority, one shared channel so nobody forwards messages between camps, blameless incident notes written the same way whichever team caused the break, and a named escalation pair who talk before anything becomes a twelve-person thread.

Week-one question	A good answer	What it looks like unanswered
Whose conventions apply?	Theirs, written down and shared	Two styles, two linters, noisy diffs
Who reviews whose work?	Both ways, inside a working day	Half the codebase unreviewed, half of it slow
Who owns each environment?	One named owner per environment	Staging "borrowed" in the middle of a demo
How is a disagreement settled?	A named escalation pair	A thread with twelve people on it

I also watch for the small signs of drift: reviews left open for three days, an environment borrowed without a message, a decision made in a meeting the other team was not in. Each one is minor. Together they are how two teams become two sides.

At Shanti Infosoft I treat a shared codebase as one team with two payrolls, and I say that sentence in the kickoff so both groups hear it at the same time. It is the first thing I set up on any embedded software development engagement, and most of the friction disappears once nobody is waiting to see who gets blamed first.

If you share a codebase with a client's team today, could you say in one line who reviews what, and would they give the same answer?

Sonal Jain leads delivery at Shanti Infosoft, a CMMI Level 5 company whose 80+ engineers regularly work embedded with client teams.

Sharing a Codebase With the Client's In-House Team

Sonal Jain — Sun, 13 Sep 2026 08:01:43 +0000

Whose rules apply?

Setting that up takes about a day:

Sixty minutes walking through their PR process, led by whoever actually merges.
A list of who can merge to the main branch. It is usually shorter than anyone thought, and usually one person who is also the busiest on their team.
A written note of which conventions we are adopting, sent to both teams the same afternoon.
Agreement on review turnaround, in hours, before the first pull request is opened.

From the day that note goes out they are the shared rules rather than the vendor's rules, which is a small wording difference that changes how people argue about them later.

Who reviews whose work?

What about AI-generated code from either side?

How do you handle the human side?

Week-one question	A good answer	What it looks like unanswered
Whose conventions apply?	Theirs, written down and shared	Two styles, two linters, noisy diffs
Who reviews whose work?	Both ways, inside a working day	Half the codebase unreviewed, half of it slow
Who owns each environment?	One named owner per environment	Staging "borrowed" in the middle of a demo
How is a disagreement settled?	A named escalation pair	A thread with twelve people on it

If you share a codebase with a client's team today, could you say in one line who reviews what, and would they give the same answer?

Sonal Jain leads delivery at Shanti Infosoft, a CMMI Level 5 company whose 80+ engineers regularly work embedded with client teams.

When the Third-Party API Is the Riskiest Part of Your Project

Sonal Jain — Fri, 11 Sep 2026 09:00:08 +0000

On most projects the riskiest component is one we did not build. A payment gateway, a courier's tracking API, a government identity check, a maps provider, an SMS service. Our own code we can fix at midnight. A vendor's sandbox that behaves differently from production, or a go-live approval queue that takes three weeks, we cannot. So every third-party integration on my projects gets treated as a risk item from the first week, with its own owner and its own fallback.

Why do third parties bite so often?

Because a third-party service sits outside the plan's control and inside its critical path. The same failure modes repeat: documentation for a version the vendor no longer runs, a permissive sandbox followed by a strict production endpoint, unread rate limits, a vendor-side compliance review, and a deprecation email nobody opened.

None of these are exotic. All of them are invisible on a Gantt chart, because "integrate payment gateway, five days" looks like any other task. The compliance review is the one that hurts most, because it starts only after you submit the finished integration, which is to say after the point where the plan assumed you were done.

What do I check in week one?

Five questions, answered before the integration is scheduled: do we hold sandbox credentials today, how long does production approval actually take, what are the rate limits and the price at real volume, what does the user see when the service is down, and who at the vendor answers the phone.

Do we have sandbox credentials in our hands today, or has someone promised them?
How long does the vendor's production approval take, measured from someone who has done it recently rather than from the vendor's own page?
What are the rate limits and the price at our real volume, and at three times that?
What happens to the user when the service is down, and has anyone designed that screen?
Who at the vendor answers the phone, by name, when it breaks?

I do not let the work be scheduled until all five have answers. The approval question is the one that moves dates. Payment providers and identity services in particular have review steps that can add weeks, and those weeks belong at the start of the plan, running in parallel with the build, rather than at the end where they usually get discovered.

Isn't buying an integration supposed to reduce risk?

It trades one risk for another. Buying moves the engineering work onto a vendor who has solved the problem many times, which is worth paying for. It also moves your timeline onto that vendor's approval queue and your uptime onto their status page, neither of which you control.

I keep an eye on how the build-versus-buy line has moved, because for a growing set of small integrations, building a thin version ourselves is now faster than getting through a vendor's onboarding, and it leaves the client owning something they can change. That is a delivery decision as much as an engineering one. If the vendor's approval takes longer than the build of the feature it enables, that belongs on the risk register with a date, and the client should hear about it in week one.

What does the fallback look like?

Three habits. Build against a stub in the first sprint, so the rest of the product does not wait. Integrate for real the day credentials land, never in the final sprint. And design a degraded mode for every integration, so the product still does something useful when the service is down.

Integration	What can hold the date	The fallback we build
Payment gateway	Vendor-side compliance review before go-live	Stubbed checkout so the rest of the flow ships
Courier tracking	Rate limits discovered at real volume	Order saves, label prints later, user is told
Identity check	An approval queue measured in weeks	A manual review path for the first cohort
SMS or maps	An outage on the provider's status page	A visible message instead of a dead screen

Integrating early matters more than it sounds, because a real integration always teaches you something the documentation did not. The degraded mode is a design task rather than an error handler: if the courier API is down, the order still saves and the label prints later, with a message that says so.

At Shanti Infosoft the integration list is reviewed in every weekly status report until each item has cleared production approval. It is the dullest recurring line on a web development project and the one I refuse to drop, because dull is what keeps the launch date attached to reality.

Which third-party service on your project could hold your launch date hostage, and does anyone own the relationship?

Sonal Jain runs project delivery at Shanti Infosoft, a CMMI Level 5 team that has delivered for 700+ companies.

When the Third-Party API Is the Riskiest Part of Your Project

Sonal Jain — Fri, 11 Sep 2026 08:01:06 +0000

Why do third parties bite so often?

What do I check in week one?

Do we have sandbox credentials in our hands today, or has someone promised them?
How long does the vendor's production approval take, measured from someone who has done it recently rather than from the vendor's own page?
What are the rate limits and the price at our real volume, and at three times that?
What happens to the user when the service is down, and has anyone designed that screen?
Who at the vendor answers the phone, by name, when it breaks?

Isn't buying an integration supposed to reduce risk?

What does the fallback look like?

Integration	What can hold the date	The fallback we build
Payment gateway	Vendor-side compliance review before go-live	Stubbed checkout so the rest of the flow ships
Courier tracking	Rate limits discovered at real volume	Order saves, label prints later, user is told
Identity check	An approval queue measured in weeks	A manual review path for the first cohort
SMS or maps	An outage on the provider's status page	A visible message instead of a dead screen

Which third-party service on your project could hold your launch date hostage, and does anyone own the relationship?

Sonal Jain runs project delivery at Shanti Infosoft, a CMMI Level 5 team that has delivered for 700+ companies.

Why I Tie Milestone Payments to Working Software

Sonal Jain — Wed, 09 Sep 2026 08:00:08 +0000

I structure milestone payments around things the client can click, and never around calendar dates alone. When a payment falls due because "month two ended", the invoice becomes a negotiation about how finished the work is. When it falls due because "vendor onboarding works end to end on staging and your tester has run the scenarios", the invoice is a receipt. Same money, completely different conversation.

Why do date-based milestones go wrong?

Because both sides end up arguing about a percentage. A date-based milestone falls due whether or not anything is demonstrable, so the only evidence available at invoice time is an opinion about progress. Dates also remove the pressure to have something working by the first of the month.

Property	Date-based milestone	Acceptance-based milestone
Falls due when	Month two ends	Vendor onboarding works end to end on staging
Evidence	A percentage-complete figure	The eight UAT scenarios, ticked by their tester
The invoice is	A negotiation	A receipt
It rewards	Being on the calendar	Having something to show

"We're at sixty percent" is a sentence that means something different to an engineer and a finance manager, and I have watched a discussion about whether a project was at sixty or seventy percent complete last longer than the sprint it was describing. Nobody wins that conversation. The vendor sounds defensive and the client feels sold to.

What does a good milestone look like?

A good milestone has four properties. It is demonstrable, so the client sees it work rather than reads that it works. It carries a short written acceptance list. Payment follows sign-off within a fixed number of days. And it is small enough to land every three to five weeks.

I check every proposed milestone against those before it goes near a contract:

Can the client click it? If the only proof is a status update, it is not a milestone.
Is the acceptance list written down, short, and specific enough to argue about now instead of later?
Does payment follow sign-off inside a fixed window, so acceptance and payment are one event rather than two?
Does it land inside five weeks? A milestone that takes a quarter is a project with one payment at the end.

A real one reads like this: "Milestone 2: vendor onboarding, from invitation to first product listed, complete on staging with email verification. Acceptance: the eight onboarding scenarios in the UAT list pass. Payment: 25 percent, within ten business days of sign-off." Nothing in there is open to interpretation, which is exactly the point.

What do buyers expect now?

Proof before promises. Buyers increasingly want to see software working before they commit money to the next stage, and they walk away from vendors who can only offer slides. Milestone structure applies that instinct to a contract, so every payment becomes a small request to show the work.

The way buyers shop for AI agents in 2026 is a good picture of it, and a vendor confident in their delivery should welcome the scrutiny, because it makes the invoice conversation short.

On AI work I go one step further. The first paid milestone is often the boring one: the evaluation set agreed, the data access working, the failure cases listed, a baseline number measured on real examples. Clients sometimes push back on paying for a milestone with no feature in it. My answer is that it is the milestone that decides whether the features are worth building, and I would rather they pay for that answer early than for a beautiful feature that turns out to be measuring the wrong thing.

What if the client's finance team needs dates anyway?

Then each milestone carries both. A target date for cash-flow planning, and an acceptance list for payment. If acceptance slips for the vendor's reasons, the vendor carries the delay and says so. If it slips because client testing has not happened, a deemed-acceptance clause takes over after an agreed window.

Ten business days after delivery is the number I usually write, so a payment cannot be held hostage by an unopened staging link. Both sides know the rules before the first invoice, which is the only time anyone reads them calmly.

At Shanti Infosoft I would rather spend an extra hour on the milestone wording during contracting than an extra week arguing about percentages in month four, and it is the hour I defend hardest in any IT consulting agreement we sign. The hour is cheaper, and the client remembers the week.

When did your last invoice conversation take longer than it should have, and what would the milestone have needed to say to make it a receipt?

Sonal Jain leads delivery at Shanti Infosoft, a CMMI Level 5 software team that has shipped for 700+ companies.

What Happens to Your Project When the Lead Developer Resigns

Sonal Jain — Mon, 07 Sep 2026 08:00:33 +0000

When the lead developer resigns mid-project, the client's real fear is that the knowledge walks out with them. It usually does, unless continuity was being built from day one. So there are two halves to this: what I do in the forty-eight hours after the notice lands, and what I do on every project so those forty-eight hours hurt less.

What happens in the first two days?

The client hears it from the delivery lead within a day, with a plan attached: who the successor is, when they start pairing, what the departing developer will hand over, and by when. Internally, the same two days go into freezing risky work and booking the handover sessions with recordings turned on.

Never a bare "unfortunately". A client can absorb bad news with a plan attached. They cannot absorb bad news that arrives as a rumour.

I learned that the hard way. Early in my career I sat on a resignation for a week, hoping to have the replacement fully briefed before saying anything. The client found out from a LinkedIn update. Everything I said after that sounded like damage control, and it was. Now the rule is simple: the client knows before the team's own farewell lunch is booked.

What does a real handover contain?

Four things, and a document dump is not one of them. A recorded walkthrough of the ugliest code, a written list of what only that developer knows, three tickets paired with the successor driving, and a decision log that explains why the system ended up the way it is.

Record a walkthrough of the parts the developer would be embarrassed to explain, because that is exactly where the successor will get stuck.
Write the "things only I know" list, produced by one question: what would break if you vanished tomorrow and nobody could call you?
Pair on the next three tickets, with the successor driving and the leaver watching.
Hand over the decision log, so the successor knows why as well as how.

A hundred pages written in the last week by someone with one foot out of the door is a hundred pages nobody reads. The "things only I know" list is usually a single page, and it is the most valuable artefact in the whole handover.

Can continuity be built before anyone resigns?

Yes, and it costs little compared with the alternative. Two people touch every critical module, even if the second only reviews. Every pull request is read by someone who did not write it. Runbooks exist for anything that runs on a schedule. The decision log is written weekly rather than reconstructed later.

Habit	Without it	With it
Second reviewer on every module	The handover starts from zero	Someone can already read the code
A live "things only I know" list	Written in the last panicked week	Updated while the knowledge is fresh
Weekly decision log	Reconstructed from memory and chat scrollback	Read in one afternoon by the successor
Runbooks for scheduled jobs	Discovered when a job fails at 2am	Handed over with everything else

Those habits sound like process for its own sake until the day they save a project. They are the same habits I see in governing code rather than writing it, the idea that the value of code is in how well it survives its author, and it has only become more true now that so much code is generated fast and reviewed slowly.

How do you tell the client honestly without frightening them?

Separate the fact from the fear. The fact is that a person is leaving. The fear is that the project is now at risk. So the message answers the fear first, saying what changes this week and why, then states the fact, then the plan, then offers a call.

"Nothing about the timeline changes this week, and here is why" is the first line. Most clients relax at that sentence and read the rest as information rather than as a warning. The ones who do not relax want the successor's name and the pairing dates, which is why both are already in the message.

At Shanti Infosoft I keep a one-line rule for delivery leads on every software development engagement we run: any developer who is the only person who understands a module is a risk on the register, and it gets a date by which that stops being true.

If your lead developer handed in notice tomorrow, how many days would it take before your client noticed a difference?

Sonal Jain is the delivery lead at Shanti Infosoft, a CMMI Level 5 software company with more than 80 engineers on staff.

Why I Run a Pre-Mortem Before Every Kickoff

Sonal Jain — Sat, 05 Sep 2026 09:00:22 +0000

A pre-mortem is a forty-five-minute meeting, held before kickoff, where the team pretends the project has already failed and writes down why. I run one on every project now. It surfaces the risks that a normal planning session never hears, because "imagine it went wrong" gives people permission to say things they would not say as "I have a concern".

How does it actually run?

A pre-mortem runs in three moves: one framing sentence, five minutes of silent writing, then a round-robin readout. The framing is "it is six months from now, the project has failed badly, and we are explaining what happened." The whole thing takes forty-five minutes.

The full sequence:

Read the framing sentence. No context-setting, no reassurance, nothing else.
Five minutes of silent writing. Everyone lists their reasons alone, before anyone speaks.
Round-robin, one reason per person per turn, until the lists are empty.
Group them on a board and pick the five that would hurt most.
Give each of the five an owner and a tripwire before anyone leaves the room.

The silence in step two is the part people want to skip and the part that matters most, because the moment someone senior talks first, the room converges on their view. I run a lighter, twenty-minute version with the client too, usually in the second week, once there is enough trust to say "here is how we think this could go wrong".

What comes out that planning misses?

Specifics. A planning session produces risks like "timeline pressure". A pre-mortem produces sentences describing something a person actually watched happen once, named in enough detail to be plannable, including the uncomfortable ones about how the team itself behaves when the schedule gets tight.

The planning session says	The pre-mortem says
Timeline pressure	We said yes to the extra reporting module in week two and it ate the buffer
Integration risk	The courier API's documentation was wrong and we found out in UAT
Access dependencies	The client's IT team never gave us production access and we built the migration blind
Adoption risk	The model was accurate but the agents on the floor kept using the old form

On one pre-mortem the most-voted failure reason was that first row, close to word for word. Everyone in the room had watched it happen on a previous project. Nobody had said it in the planning meeting.

Do AI projects need their own failure list?

Yes. AI projects fail in ways a room that has mostly shipped conventional software will not name unprompted: data that was never approved for use, an evaluation set nobody built, a working feature nobody owned once the build team left, and users who were never asked whether they wanted it.

So before an AI build I seed the exercise with that list, and I send round the frank teardown of what broke when someone tried to run a company on AI-agent departments a few days beforehand, because it is a catalogue of failures that look nothing like a missed deadline. People read it and arrive with their own version.

How does a list become protection?

Through tripwires. A risk with an owner is a hope. A risk with a tripwire is a plan, because a tripwire names an observable signal and a date: "if we do not have production-like data by day fifteen, Sonal escalates to the sponsor that week." Never "we will monitor it".

A specific thing, on a specific day, done by a named person. The tripwires go into the risk section of the weekly status report, so the client sees them and sees them being watched. Half the time the tripwire never fires and the risk quietly retires. The other half, we act in week three instead of week nine.

The exercise has also become the first thing I bring to a new software consulting engagement, ahead of the plan, because the plan reads better once the room has said out loud what could sink it.

At Shanti Infosoft the pre-mortem sits between the estimate and the kickoff on every project plan, and I treat skipping it the way I would treat skipping the estimate. Both are ways of choosing to be surprised later.

If your current project failed six months from now, what is the reason your team already knows and has not said out loud?

Sonal Jain heads project delivery at Shanti Infosoft, a CMMI Level 5 firm whose 80+ engineers ship client software week in, week out.

Why I Run a Pre-Mortem Before Every Kickoff

Sonal Jain — Sat, 05 Sep 2026 08:01:21 +0000

How does it actually run?

The full sequence:

Read the framing sentence. No context-setting, no reassurance, nothing else.
Five minutes of silent writing. Everyone lists their reasons alone, before anyone speaks.
Round-robin, one reason per person per turn, until the lists are empty.
Group them on a board and pick the five that would hurt most.
Give each of the five an owner and a tripwire before anyone leaves the room.

What comes out that planning misses?

The planning session says	The pre-mortem says
Timeline pressure	We said yes to the extra reporting module in week two and it ate the buffer
Integration risk	The courier API's documentation was wrong and we found out in UAT
Access dependencies	The client's IT team never gave us production access and we built the migration blind
Adoption risk	The model was accurate but the agents on the floor kept using the old form

Do AI projects need their own failure list?

How does a list become protection?

The exercise has also become the first thing I bring to a new software consulting engagement, ahead of the plan, because the plan reads better once the room has said out loud what could sink it.

If your current project failed six months from now, what is the reason your team already knows and has not said out loud?

Sonal Jain heads project delivery at Shanti Infosoft, a CMMI Level 5 firm whose 80+ engineers ship client software week in, week out.

The Month-Two Checkpoint: When to Pull the Plug on an AI Pilot

Sonal Jain — Thu, 03 Sep 2026 09:00:27 +0000

Every AI pilot I run has a checkpoint around week eight where we decide, in writing, whether to continue, change direction, or stop. The decision itself is rarely hard. What makes it possible is that the criteria were written down in week one, before anyone had a favourite outcome to defend.

Why week eight?

Week eight is late enough for real users to have hit the system with real data, and early enough that stopping is still cheap. By then the demo glow has faded, the first bill has arrived, and the intended users have either adopted the tool or found reasons not to.

Sunk cost has not hardened into pride yet. Wait until month five and the pilot has a budget line, a name in a board deck and a team whose next quarter depends on it. Nobody stops those. They just get quietly extended.

What are the criteria, and who writes them?

The criteria are written by the delivery lead and the sponsor together at kickoff, on one page, in four groups: the outcome and the number that counts as working, adoption without chasing, cost per unit and its direction, and how many of the starting unknowns are still unknown.

Criterion	The question it asks	Example stop condition
Outcome	What number counts as working?	Fewer than a third of drafts go out unedited by week eight
Adoption	Are people using it without being chased?	Weekly active users flat or falling for three weeks
Unit cost	What does one unit cost, and which way is it moving?	Cost per handled ticket still above the manual baseline
Unknowns	How many week-one unknowns are still unknown?	The biggest question at kickoff is still unanswered

A support drafting pilot might set its outcome line at "the agent's draft goes out with no edits at least half the time". Putting the stop conditions next to the success conditions is the whole trick. In week eight nobody argues about them, because they were agreed by the same people in week one, before there was anything to be attached to.

What does the conversation sound like when it isn't working?

Direct and early. The checkpoint meeting is booked on day one, so it exists before anyone is nervous about it, and it opens with the same one-page criteria. The recommendation is stated plainly, with the measured number, the agreed floor, and the direction of travel.

The running order I use:

Read the week-one page out loud, success conditions and stop conditions together.
Put the measured numbers next to them, with the trend rather than the latest value.
State a recommendation, whether that is continue, narrow or stop, before opening the floor.
Write the decision and its date on the same page, and name who owns the next one.

In practice step three sounds like this: "We are at 28 percent unedited, we agreed a third was the floor, and the trend is flat. My recommendation is to narrow it to refund queries only, where it is at 61 percent, and review again in four weeks."

I take seriously the industry warnings that a large share of AI agent projects will be dead by 2027, and my reading of that number is that most of those projects should have died in month two, cheaply and with everyone's dignity intact, rather than in month fourteen with a budget attached. Killing a pilot at the checkpoint is a good outcome. It is the reason the checkpoint exists.

What does a pivot usually look like?

Smaller, almost always. A pivot after a failed checkpoint narrows the scope to the slice where the numbers were already good, drops the autonomy level so the tool suggests instead of acts, adds a human checkpoint in front of it, or changes the data the pilot runs on.

That last one comes up more than people expect, because the pilot ran on the data that was easy to get rather than the data the job needed. Whichever it is, the pivot gets its own week-one page and its own checkpoint. A pilot that pivots without new criteria is a pilot that has learned to avoid being measured.

The first hour of any AI consulting engagement I run now goes on stop conditions rather than on the roadmap, because the roadmap is the easy part and nobody disagrees with it yet.

At Shanti Infosoft I would rather run four small pilots that each end with a clear decision than one long pilot that ends with a shrug. The decisions are the deliverable, even the ones that say stop. If a pilot of yours is overdue a checkpoint, my calendar is here.

If your current AI pilot were being judged next week, what would it be judged against, and who agreed to that?

Sonal Jain runs delivery at Shanti Infosoft, a CMMI Level 5 team that has built software for more than 700 companies.

The Month-Two Checkpoint: When to Pull the Plug on an AI Pilot

Sonal Jain — Thu, 03 Sep 2026 08:01:25 +0000

Why week eight?

What are the criteria, and who writes them?

Criterion	The question it asks	Example stop condition
Outcome	What number counts as working?	Fewer than a third of drafts go out unedited by week eight
Adoption	Are people using it without being chased?	Weekly active users flat or falling for three weeks
Unit cost	What does one unit cost, and which way is it moving?	Cost per handled ticket still above the manual baseline
Unknowns	How many week-one unknowns are still unknown?	The biggest question at kickoff is still unanswered

What does the conversation sound like when it isn't working?

The running order I use:

Read the week-one page out loud, success conditions and stop conditions together.
Put the measured numbers next to them, with the trend rather than the latest value.
State a recommendation, whether that is continue, narrow or stop, before opening the floor.
Write the decision and its date on the same page, and name who owns the next one.

What does a pivot usually look like?

The first hour of any AI consulting engagement I run now goes on stop conditions rather than on the roadmap, because the roadmap is the easy part and nobody disagrees with it yet.

If your current AI pilot were being judged next week, what would it be judged against, and who agreed to that?

Sonal Jain runs delivery at Shanti Infosoft, a CMMI Level 5 team that has built software for more than 700 companies.

How I Vet an AI Tool Before It Touches a Client Project

Sonal Jain — Tue, 01 Sep 2026 08:00:10 +0000

Before any AI tool or so-called agent goes anywhere near a client project, I want to watch it fail in front of me. Sales pages tell you what it does on a good day. My job is to know what it does on a bad one, who is accountable when it is wrong, and what it costs at ten times the demo volume. Those answers decide whether it gets in, and most tools do not.

What is the first test?

Ugly inputs. I keep a folder of them, and every tool gets the folder before it gets a meeting. A scanned invoice photographed at an angle. An email written half in Hindi and half in English. A spreadsheet with merged cells and a total row sitting in the middle of the data.

There is also a support ticket that runs as one long sentence with no punctuation. Nothing confidential, all real in shape.

What I am watching for is how it fails, since every tool fails somewhere. Does it say "I'm not sure" or does it produce a confident wrong answer with a straight face? A tool that flags uncertainty can be put in front of a human. A tool that bluffs cannot, whatever its accuracy on the vendor's own examples.

Is it an agent, or a form with a chat window?

A lot of what is sold as an "AI agent" is a chatbot stapled to a form. That is fine as a product, and it changes what we can honestly promise a client, so I ask four blunt questions before the demo starts. The answers sort the two categories inside ten minutes.

Question I ask	A real agent	A form with a chat window
What does it produce?	An action inside a system	Text for a person to act on
Who approves?	A checkpoint I can make mandatory	Nobody, because nothing happens
Is there a log?	Every step, in plain language	The conversation, and that is all
Wrong at 2am, then what?	It stops and escalates	It waits to be read

The distinction matters more than the demo does. I lean on the way agent-washing, real agents versus fake ones is laid out because it gives me a vocabulary for the vendor call: what does the thing decide on its own, and what happens when that decision is wrong at two in the morning.

What about cost and control?

Four questions, asked before anyone on my side falls in love with the tool. They are dull questions and they are the ones that decide whether a tool can carry client work, because a tool that fails any of them becomes somebody's problem to unwind eighteen months later.

What does it cost at ten times our expected volume? AI pricing is gentle at pilot scale and brutal at production scale.
Where does the data go, and can the client's legal team live with that answer?
Can we get our data and our configuration out if we leave?
What happens when the vendor swaps the underlying model, which they will, without telling us?

The last one has bitten more projects than any bug. A prompt tuned for months quietly starts behaving differently, and nobody on the vendor's side considers it a change worth announcing.

What is the rule once a tool passes?

It still does not get to act alone. Every AI tool enters a client workflow with a human checkpoint in front of it, and it earns autonomy over weeks by being measured: how often did the human change the output, and how often did that change matter?

Only when those numbers are boring for a sustained stretch do we discuss removing the checkpoint, and even then the client decides.

At Shanti Infosoft the vetting sits with delivery rather than with whoever found the tool first, because the person who found it is usually excited and excitement is a poor reviewer. The same folder of ugly inputs goes into every AI integration we scope, before anyone writes a line of glue code.

One more thing I have learned to ask on the vendor call: "Show me a customer who turned it off, and tell me why." The good vendors have an answer. The rest change the subject. If you want a second pair of eyes on a tool your team is about to sign for, book a slot.

Which AI tool on your current project has never been shown a genuinely ugly input?

Sonal Jain leads delivery at Shanti Infosoft, a CMMI Level 5 company that has shipped for 700+ companies, and she keeps the ugly-input folder herself.