DEV Community: FARHAN HABIB FARAZ

When The Bot's Personality Falls Apart Under Pressure

FARHAN HABIB FARAZ — Mon, 31 Aug 2026 05:40:32 +0000

A persona that holds up fine through the first two or three exchanges of a call often starts drifting the moment the conversation gets difficult, an angry caller, a repeated question, a request the bot has to decline. The calm, professional, on brand voice that worked perfectly in the easy part of the call can quietly slip into something else entirely once the interaction gets harder, sometimes overly apologetic, sometimes stiff and defensive, sometimes oddly casual in a way that clashes with how the same bot sounded a minute earlier.

This happens because most persona instructions are written and tested against calm, cooperative sample conversations, since that is what most testing naturally looks like. The instruction describes a tone, professional, warm, patient, and that description holds up perfectly well as long as nothing in the conversation is pushing against it. The failure only becomes visible under actual pressure, which is precisely the condition most persona testing does not naturally include, because it is far more comfortable to test a system against polite sample dialogue than against a genuinely frustrated, repetitive, or hostile caller.

Under pressure, a model without explicit instruction for exactly this scenario tends to reach for whatever response pattern feels most immediately appropriate to the emotional tone of the moment, rather than the persona defined for the deployment as a whole. A frustrated caller pushes the model toward an apologetic, placating register. A repetitive caller pushes it toward a flatter, more mechanical register. Neither of those drifts is necessarily wrong in isolation, but neither one is the persona that was actually designed and approved for the deployment, and the inconsistency itself, a bot that sounds like a different entity depending on how the caller is behaving, damages trust more than a stable but imperfect tone would.

The fix requires treating persona consistency as something that has to be explicitly reinforced specifically under difficult conditions, not just defined once at the top of the system prompt and assumed to hold automatically. This means writing distinct guidance for how the established persona specifically behaves when a caller is frustrated, when a question is being repeated, when a request has to be declined, rather than trusting that a single general tone description will naturally extend itself correctly into those harder moments. A persona described as calm and professional needs an explicit instruction for what calm and professional specifically sounds like when someone is upset, because the model's default instinct in that moment is not automatically going to match the intended definition.

The other necessary piece is giving the model explicit permission to maintain firm boundaries without breaking persona to do so, since a common failure pattern is a bot becoming either overly accommodating or noticeably colder the moment it has to say no to something. The instruction has to make clear that declining a request or holding a boundary is fully compatible with the established warm, professional tone, rather than something that requires switching into a different, more clinical register to accomplish. Without that explicit permission, models tend to treat firmness and warmth as being in tension with each other, when in a well designed persona they are not meant to be.

Written by Mohammad Farhan Habib Faraz
Senior Prompt Engineer and Prompt Team Lead at PowerinAI
www.powerinai.com

When The Bot Sounds Confident Even When It Shouldn't

FARHAN HABIB FARAZ — Mon, 31 Aug 2026 05:33:54 +0000

A model asked something it does not have reliable information for will very rarely say so plainly. Instead it produces an answer with exactly the same tone, pacing, and structural confidence as a fully grounded response, and a caller has no way to tell the difference just from how it sounds. This is one of the most consistently underestimated failure modes in voice specifically, because in a text interface a hedge or a citation gap is at least visually inspectable. On a phone call, tone is the only signal a caller has, and an ungrounded guess delivered smoothly sounds exactly as trustworthy as a verified fact delivered the same way.

The mechanism behind this is not unique to voice, it is the same underlying tendency any language model has toward generating plausible sounding completions regardless of whether the underlying information is actually solid. What makes it a bigger problem specifically in a call bot context is that a caller cannot pause and independently verify anything mid conversation the way someone reading a chat response might quietly do on their own. Whatever gets said on the call is what the caller walks away believing, often immediately acted on, before there is any real opportunity to catch a subtly wrong answer.

The fix requires treating confidence as something the system has to earn explicitly for each response, rather than something it defaults to automatically. This means building an internal distinction into the system prompt between information the model actually has grounded access to, retrieved directly from a knowledge base or confirmed earlier in the call, versus anything it would otherwise be inferring or generating from general pattern completion. Anything falling into that second category needs to trigger a different response posture entirely, not a confident direct answer, but an explicit acknowledgment of uncertainty paired with a path forward, something like I don't have that confirmed on my end, let me connect you with someone who can verify that for you, rather than a smoothly delivered guess dressed up in the same tone as everything else.

Getting this right in practice means being unusually explicit in the instructions about what counts as grounded versus inferred, because left ambiguous, a model will often treat its own general world knowledge as functionally equivalent to information actually provided for that specific deployment, especially on questions that sound like they should have a standard, common sense answer. The instruction has to draw that line clearly rather than trusting the model to intuit it, specifying exactly which categories of question must be answered strictly from provided source material and explicitly forbidding the model from filling any gap in that material with plausible sounding general knowledge instead.

The tone of the uncertainty acknowledgment matters almost as much as the fact of including it at all. Delivered flatly or apologetically, an admission of not knowing something can read as the system being broken or unhelpful. Delivered as a brief, matter of fact redirect toward the right resource, it reads instead as a competently handled boundary, and callers generally respond far better to a confidently delivered I don't have that, but here's how we get it than to either a confidently wrong guess or an awkward, hesitant non answer.

Written by Mohammad Farhan Habib Faraz
Senior Prompt Engineer and Prompt Team Lead at PowerinAI
www.powerinai.com

When The Bot Forgets What It Just Confirmed Thirty Seconds Ago

FARHAN HABIB FARAZ — Sun, 30 Aug 2026 04:26:29 +0000

A caller confirms their name, spells out an account reference, or agrees to a specific option, and two exchanges later the bot asks for the exact same thing again, as if the earlier confirmation never happened. Nothing about the individual response is wrong in isolation. The system genuinely believes it needs that information because, from its immediate vantage point in the conversation, it does not have clear enough access to what was already settled.

This failure is easy to dismiss as a memory limitation, and sometimes it genuinely is a technical context window issue, but a large share of the time the actual cause is structural rather than a hard capacity limit. Confirmed facts get treated with the same weight as ordinary conversational turns, sitting somewhere in the middle of a growing transcript, competing for attention with small talk, clarifying questions, and filler acknowledgments. By the time several more exchanges have happened, a genuinely important confirmed detail has the same visual and structural weight in the model's context as a passing pleasantry, and it becomes easy for a system prompt that is not explicitly designed around this to lose track of which pieces of information were actually locked in versus just mentioned in passing.

Callers experience this specific failure as uniquely frustrating compared to other conversational errors, more than a wrong answer, more than a mishearing, because it reads as the system not paying attention at all, even when every other part of the interaction has gone smoothly. Repeating a confirmed detail signals inattention in a way that other kinds of mistakes do not.

The fix that works reliably is separating confirmed information from the general flow of conversation at a structural level, rather than trusting that important details will naturally stay salient inside a long transcript. This means explicitly instructing the system to maintain a running state of confirmed facts, treated as fixed and non negotiable for the remainder of the call, distinct from the ordinary back and forth of the conversation itself. Once something has been explicitly confirmed by the caller, it gets referenced from that separate running state, not re-derived by scanning back through the conversation history and hoping it gets noticed again.

The instruction also needs an explicit rule against re-asking anything already present in that confirmed state, phrased as a hard constraint rather than a soft preference, because a soft instruction like try to remember confirmed details competes poorly against the model's general tendency to double check things it is uncertain about. A hard rule, treat information in the confirmed state as settled and never request it again during this call, holds up far more reliably under the pressure of a long, meandering conversation than a gentler version of the same idea.

The other piece that matters is making sure the confirmed state gets updated the moment something is confirmed, not at the end of a longer exchange, because a confirmation that happens early in a call and only gets recorded much later leaves a window where the same failure can still occur despite the fix technically being in place.

Written by Mohammad Farhan Habib Faraz
Senior Prompt Engineer and Prompt Team Lead at PowerinAI
www.powerinai.com

When The Bot Answers The Wrong Question Because It Answered Too Fast

FARHAN HABIB FARAZ — Sun, 30 Aug 2026 04:19:40 +0000

A caller asks something with two parts folded into one sentence, and the bot answers only the first part cleanly, missing the second half entirely, not because it lacked the information but because it stopped listening for intent the moment it had enough to generate a plausible response.

This shows up constantly in real conversational flows, more in voice than in text, because spoken questions rarely arrive as clean, single, well formed requests. A caller says something like can you check my balance and also tell me when the payment is due, and the model frequently responds only to the balance check, treating the second clause as either background noise or something to address later that never actually gets addressed, because the conversation has already moved forward by the time anyone notices the gap.

The root cause is a subtle one. Models generate a response as soon as a reasonably complete answer becomes available, and a single-intent answer to a multi-intent question often looks complete on its own terms, fully formed, grammatically fine, directly responsive to at least part of what was asked. Nothing about the output signals that something was dropped, which makes this a particularly hard failure to catch just by reading transcripts casually, since each individual response reads as correct in isolation.

The system prompt fix that helps most is an explicit instruction to parse every user turn for multiple distinct intents before generating any response at all, and to treat an incomplete address of a multi-part question as an incorrect response, not a partially correct one. Framing it that way matters, because a model told to be helpful and respond to what the user asked will often consider a partial answer good enough, since it did respond to something the user said. Framing an incomplete answer as flatly wrong, rather than partially right, changes how aggressively the model checks for a second or third clause before committing to a response.

The harder part is handling cases where addressing every intent in one turn would make the response too long or unwieldy for a voice context specifically, where a caller cannot skim the way a text reader can. The instruction set needs a secondary rule for exactly this situation, acknowledging every intent detected even when not every part can be fully answered in the same breath, something like a short bridging line that confirms the second question was heard and will be addressed right after the first, rather than silently dropping it. That small acknowledgment is often what determines whether a caller feels heard or feels like they need to repeat themselves, even when the actual information delivered ends up being identical either way.

Written by Mohammad Farhan Habib Faraz
Senior Prompt Engineer and Prompt Team Lead at PowerinAI
www.powerinai.com

The Silence Problem Nobody Designs For Until A Real Call Goes Quiet

FARHAN HABIB FARAZ — Tue, 25 Aug 2026 06:41:55 +0000

Voice AI demos always sound perfect because demo networks are perfect. The moment you deploy a call bot onto a real GSM network, handling real calls from real phones in real signal conditions, an entire category of failure shows up that never appears in a controlled test environment, and none of it is really about the AI's intelligence at all.

The Gap That Has Nothing To Do With The Model

Every voice AI system assumes a reasonably continuous audio stream. The model receives speech, processes it, responds. That assumption holds up fine in testing, where connections are stable and the environment is controlled. It falls apart the moment real GSM network conditions enter the picture, dropped packets, brief connection instability, backend lookups that take longer than expected on a live call, database queries that occasionally lag under real traffic.

None of those are AI problems in the traditional sense. They are infrastructure realities that the AI layer still has to gracefully handle, because from the caller's perspective, there is no difference between the model thinking and the network stalling. Both produce the same experience, silence on the other end of the line, and silence on a phone call reads very differently to a human than silence in a text chat ever does. A pause of even a few seconds in a text conversation is unremarkable. That same pause on a live call feels like the line dropped, and callers respond to that uncertainty by hanging up, repeating themselves, or getting audibly frustrated, none of which is really about whether the eventual answer would have been correct.

Turning Dead Air Into Something That Feels Intentional

The fix was not making backend lookups faster, though that mattered too. It was accepting that some delay is unavoidable and needed to be filled with something that kept the caller anchored in the conversation rather than left wondering if anything was still happening.

The system prompt got built around explicit acknowledgment phrases triggered specifically during known-delay conditions, a geolocation lookup taking a beat longer than expected, a database confirmation step running slightly slow. Rather than letting the model simply go silent while a backend process completed, it was instructed to bridge that gap verbally, something in the register of let me check that and confirm for you, one moment please, followed naturally by a closing acknowledgment once the result came back, appreciate your patience, once the information was ready to deliver.

That sounds like a small, almost trivial addition. In practice, it changed how the entire interaction felt. A caller hearing an active acknowledgment during a pause interprets that pause as the system working. A caller hearing nothing during the exact same pause interprets it as the system failing. The underlying delay was often identical in both cases. Only the caller's read of it changed, and that read is what actually determines whether they stay on the line calmly or start repeating themselves out of anxiety.

Getting the timing of those bridging phrases right took real iteration. Too early, and the acknowledgment fires before the caller even notices a gap, which reads as oddly hesitant. Too late, and the caller has already started to worry before the reassurance arrives. The instruction set needed a fairly specific threshold, tied to the actual expected duration of each known delay-prone operation, rather than a single generic timeout applied everywhere.

When The Transcript Itself Turns Against You

A separate and genuinely harder problem showed up around call transcription quality. Live GSM audio occasionally produces transcripts that partially garble into fragments that read as a different language entirely, sometimes real foreign language artifacts, sometimes just phonetic noise that a transcription layer misreads as belonging to a different language model. A bot that treats a corrupted transcript as reliable input will confidently try to process nonsense as if it were a real user utterance, which produces exactly the kind of unpredictable, wrong-sounding response that erodes caller trust fastest.

Handling this meant building explicit low-confidence detection into how the system treated incoming transcripts, rather than assuming every transcript was equally trustworthy input. When a transcript came back looking fragmented, inconsistent with the expected language of the call, or simply nonsensical relative to conversational context, the instruction set treated that as a signal to ask for clarification rather than attempt to reason from corrupted input, something as simple as I want to make sure I understood that correctly, could you repeat that for me. That single fallback did more to protect call quality than any amount of trying to improve transcription accuracy itself, because it accepted that some transcription failures were simply going to happen and built a graceful recovery path around that reality instead of pretending the input layer would always be clean.

Fixing Geolocation Without Asking The Caller Anything

A third recurring issue involved geolocation, needed for certain call flows to route or personalize responses correctly, but unreliable to pull directly in a fair number of real call conditions. Rather than build a flow that interrupted the conversation to explicitly ask a caller where they were calling from, which adds friction and feels like an odd question in the middle of an otherwise natural call, the more elegant fix used the international dialing prefix already present in the caller's phone number itself. The country code sitting at the front of virtually every incoming number carried enough signal on its own to infer general location context, without requiring a single additional question or interrupting the flow of the conversation at all.

That fix is a good example of a pattern that shows up constantly in this kind of work, the best solution to a missing-data problem is often not asking the user to supply the missing data directly, it is finding a piece of information you already have access to that can substitute for it without the caller ever noticing a gap existed in the first place.

The Actual Lesson

A meaningful share of what makes a voice AI system feel reliable has very little to do with how intelligent the underlying model is, and everything to do with how gracefully the system handles the messy, unpredictable conditions of a real phone network, dead air during real backend delays, corrupted transcripts, missing location data. None of those are solved by a smarter model. They are solved by treating the system prompt as responsible not just for what the bot says when everything is working, but for what it says and does in the specific moments when something in the pipeline underneath it is not.

Specific client deployments and network infrastructure details remain confidential given the nature of this work. Happy to discuss the general approach to handling real-world voice AI reliability issues with anyone building similar call bot systems through the proper channel.

Written by Mohammad Farhan Habib Faraz
Senior Prompt Engineer and Prompt Team Lead at PowerinAI
www.powerinai.com

Building A Prompt Template That Works Without You In The Room

FARHAN HABIB FARAZ — Tue, 25 Aug 2026 06:33:16 +0000

Building a working tender documentation system for yourself is one project. Turning that same system into a template the rest of the team can pick up and use correctly, without needing to ask you what a particular instruction actually means, is a completely different project wearing the same clothes.

The Gap Between Personal Use And Handoff

A prompt template that only you use can carry a lot of implicit knowledge safely, because the missing context lives in your head and gets filled in automatically every time you run it. An instruction that says something like ensure the response addresses compliance requirements directly means something very specific to the person who wrote it, shaped by dozens of past examples of what counting as directly actually looks like in practice. That same instruction, handed to someone on the team who was not present for any of those past examples, is just as likely to be interpreted in a way that is defensible on its own terms and still wrong relative to what was actually meant.

The template worked perfectly for months before it needed to be handed off, which made the gap invisible until the moment it actually mattered. The first time someone else on the team ran it independently and produced a response that technically followed the instructions but missed the actual intent behind them, the problem was not that the instructions were poorly written in any obvious sense. It was that they had been written for an audience of one, and that audience had context nobody else on the team had access to.

What Actually Needs To Be In A Handoff Ready Template

Fixing this meant rewriting a significant portion of the template with a different question in mind at every step, not does this instruction produce the right output when I run it, but does this instruction contain enough of the reasoning behind it that someone without my accumulated context could apply it correctly to a new tender they have never seen before.

That meant replacing instructions that stated a rule without stating why the rule existed. An instruction like keep responses concise and avoid excessive marketing language got expanded to explain specifically what excessive marketing language actually looks like in a tender context, and specifically why it costs points with an evaluator rather than just sounding unprofessional in the abstract. Once the reasoning was visible inside the instruction itself, someone applying it to an unfamiliar tender had something to reason from, rather than just a rule to follow blindly and hope it generalized correctly.

It also meant building in explicit examples of edge cases the template needed to handle, not because examples make output better in general, but because a person unfamiliar with the domain needs at least one concrete anchor showing what correct application looks like before they can reliably extend that pattern to a new, slightly different situation on their own.

The Part That Was Hardest To Accept

The most uncomfortable part of this process was realizing how much of what made the original template work well was not actually in the instructions at all, it was in my own judgment about when to deviate from the template, when a particular tender's requirements were unusual enough that the standard structure needed to bend. That judgment is exactly the hardest thing to transfer into a written template, because it is inherently situational rather than rule based.

The honest resolution was not pretending that judgment could be fully captured in written instructions, because it cannot. Instead, the template got restructured to explicitly flag the situations most likely to require that kind of judgment call, telling the person using it here is a category of tender structure where the standard approach often needs adjustment, and here is what to watch for, rather than trying to write a rule precise enough to cover every possible variation. Naming the boundary of what the template could reliably handle on its own turned out to be more useful than pretending that boundary did not exist.

Why This Matters Beyond One Template

Once this template had been through that rebuilding process, the same discipline started getting applied earlier in the development of every subsequent template built for the team, rather than being treated as a separate step that happens only after something has already been in personal use for a while. Building with handoff in mind from the beginning, rather than retrofitting it in after the fact, produces noticeably fewer of these gaps, because the instinct to ask does this make sense to someone without my context becomes part of writing the instruction the first time, rather than a correction applied after watching it fail.

The Actual Lesson

A prompt template that works well for its original author and a prompt template that is actually ready to hand to a team are not the same artifact, even when they produce identical output in testing. The difference lives entirely in whether the reasoning behind each instruction is visible enough for someone without the author's accumulated context to apply it correctly to a situation the author never specifically anticipated.

Specific tender template content and team workflow details remain confidential given the nature of this work. Happy to discuss the general approach to building prompt systems designed for team handoff with anyone building similar reusable templates through the proper channel.

Written by Mohammad Farhan Habib Faraz
Senior Prompt Engineer and Prompt Team Lead at PowerinAI
www.powerinai.com

When The Fix Isn't The Prompt At All, It's What The Prompt Is Reading From

FARHAN HABIB FARAZ — Tue, 25 Aug 2026 06:28:58 +0000

The instinct every time a deployed bot gives a wrong or off answer is to open the system prompt and start tightening instructions. Sometimes that is exactly the right move. Often it is treating the symptom while leaving the actual cause untouched, because the real problem was never in the instructions telling the model how to behave, it was in the material the model was pulling from when it tried to follow those instructions.

The Habit That Forms Because It Usually Works First

System prompt tweaking is the fastest lever to pull, and it genuinely resolves a large share of behavioral issues, tone problems, scope creep, inconsistent formatting. That fast success rate builds a habit of reaching for the same lever every time something goes wrong, because it has worked often enough to feel like the default diagnosis rather than one diagnosis among several.

The habit becomes a problem specifically with a certain category of failure, where a bot gives a confident, well formatted, properly toned answer that is simply factually wrong relative to the actual current state of whatever it is answering about. Tightening the system prompt in response to that kind of failure usually does very little, because the instructions governing how the model should behave were never the issue. The model behaved exactly as instructed. It just behaved that way using outdated, incomplete, or poorly structured source material.

Learning To Ask The Right Question Before Touching Anything

The diagnostic habit that actually needed to replace the reflexive prompt edit was asking, before changing anything, whether the failure was a behavior problem or a knowledge problem. A behavior problem looks like the model doing something it was not told to do, or doing something in a tone or format nobody asked for, even when the underlying facts it used were correct. A knowledge problem looks like the model doing exactly what it was told, in exactly the right tone, using information that was wrong, missing, or structured in a way that made it hard to retrieve accurately.

Those two categories require completely different fixes, and treating a knowledge problem as a behavior problem produces a particular kind of frustrating outcome, several rounds of system prompt tightening that each feel like they should help and none of which actually move the accuracy number, because the prompt was never broken in the first place.

Once that diagnostic question became a deliberate first step rather than an afterthought, a noticeable share of issues that used to trigger immediate prompt editing instead led to opening the knowledge base itself, and finding the actual problem sitting there, a document that had been updated in one place but not another, information structured as a dense paragraph that made a specific fact difficult for retrieval to surface cleanly, or a genuine gap where the model was being asked to answer something the knowledge base simply never covered and had been quietly filling with a plausible sounding guess instead.

Why Knowledge Base Fixes Are Less Satisfying And More Necessary

There is a reason system prompt tweaking remains the more attractive first move even after learning this lesson. Editing a system prompt is fast, contained, and produces an immediate testable result. Restructuring a knowledge base is slower, often unglamorous, and requires actually going through source material line by line to find the specific piece that is wrong or poorly structured, rather than adjusting a few lines of instruction and rerunning a test case.

That difference in effort is exactly why the wrong diagnosis is so tempting under deadline pressure. A quick prompt edit that does not actually fix anything still feels like progress was made, because something visibly changed. Sitting with a knowledge base for an hour, tracing exactly which document is feeding a wrong answer, feels slower and less immediately rewarding, even when it is the only path that actually resolves the issue.

Committing to doing the slower, correct diagnosis anyway, resisting the pull toward the faster feeling but often ineffective prompt edit, turned out to be one of the more consistently underrated disciplines in this kind of work. It means occasionally telling a client or a deadline that the fix will take longer than a quick prompt adjustment, because the actual problem lives somewhere the prompt cannot reach.

A Rough Way To Tell Them Apart Quickly

Over enough cycles of this, a rough heuristic emerged that speeds up the initial diagnosis without requiring a full investigation every time. If the same underlying question, asked in slightly different phrasings, produces inconsistent answers, that inconsistency usually points toward a knowledge base retrieval problem, because the model is pulling from different fragments depending on phrasing rather than reliably finding one clear source. If the answer is consistent across phrasings but consistently has the wrong tone, wrong format, or wrong scope, that points toward the system prompt. That rough split does not replace actually checking, but it meaningfully speeds up deciding which direction to check first.

The Actual Lesson

A wrong answer from a deployed system is not automatically a prompting failure, even though the prompt is usually the first and easiest place to look. Learning to separate a behavior problem from a knowledge problem before making any change, and being willing to accept that the correct fix is sometimes the slower one, is a large part of what actually separates a system that gets marginally patched from one that gets genuinely fixed.

Specific client systems and knowledge base structures remain confidential given the nature of this work. Happy to discuss the general diagnostic approach to separating prompt issues from knowledge base issues with anyone maintaining similar deployed AI systems through the proper channel.

Written by Mohammad Farhan Habib Faraz
Senior Prompt Engineer and Prompt Team Lead at PowerinAI
www.powerinai.com

When A Product Review Shoot Needs You To Be Honest And Complimentary At The Same Time

FARHAN HABIB FARAZ — Tue, 25 Aug 2026 06:26:12 +0000

Directing product review content for Computer Jagat sits in an uncomfortable middle ground that most people underestimate until they are actually the one writing the script. The audience trusts a review because it reads as honest assessment. The client relationship depends on the content also being fair to the product being reviewed. Those two pressures do not automatically conflict, but the moment a product has a genuine, noticeable weakness, they absolutely do, and figuring out how to write through that tension turned out to be one of the more delicate parts of the job.

Where This Actually Shows Up

Most products being reviewed are genuinely fine, and most review scripts do not encounter this tension in any serious way. The difficult cases are the ones where a product has one clearly weak feature sitting next to several genuinely strong ones. A script that simply omits the weak point entirely produces content that reads as promotional rather than reviewed, and audiences, especially the kind of audience that reads or watches tech reviews specifically because they want an honest read before spending money, are unusually good at detecting that omission even when it is never stated outright. A missing criticism in a review that otherwise sounds thorough is often more noticeable than a criticism that is actually there.

A script that leads with the weak point, or gives it disproportionate weight relative to its actual importance, produces content that damages a legitimate product's reception over something that might matter to only a small fraction of its actual use cases, and does so in a way that feels unfair given everything else the product does well.

Neither extreme is honest, in the fullest sense of the word, even though the first one looks more obviously compromised. Full honesty requires proportion, not just inclusion, and getting that proportion right is a genuinely different skill than simply deciding whether to mention something at all.

The Approach That Actually Holds Up

What works is treating the weak point the way a genuinely fair, technically competent reviewer would treat it in their own independent judgment, not the way marketing would want it framed and not the way a maximally critical reviewer would frame it either. That means placing it accurately within the structure of the review rather than at the very beginning where it would dominate the viewer's first impression, or burying it at the very end where it reads as an afterthought nobody was meant to notice.

It also means being specific rather than vague about the actual scope of the weakness. A vague criticism, something like performance could be better in certain situations, reads as evasive and actually damages credibility more than a specific one does, because vagueness signals that something is being softened. A specific, scoped criticism, describing exactly which situation the weakness shows up in and how it compares to competing products in that same specific situation, reads as genuinely informed rather than either overly harsh or suspiciously gentle.

The script writing discipline that comes out of this is being precise enough about the actual mechanism of a weakness that the criticism sounds like it comes from real testing rather than a formulaic negative section that every review script includes out of habit. A specific, well reasoned criticism placed proportionately within an otherwise fair review builds more trust in the positive parts of the same review than a review with no criticism at all ever could, because the presence of real, specific critical judgment is what makes the audience believe the praise elsewhere is not just being handed out uniformly.

Why This Requires Direct Conversation With The Client, Not Just Instinct

Getting the proportion right is not something that can be decided unilaterally by whoever is writing the script, because the client relationship genuinely matters and cannot be treated as irrelevant to how the content gets framed. The approach that works involves a direct, explicit conversation with the client before the script gets finalized, being upfront that a specific weak point exists and needs to be addressed for the review to read as credible, and working through together how that gets framed rather than either hiding it from the client until it appears in finished content or deciding unilaterally to either downplay or foreground it without their input.

Most clients, when approached this way, respond better than expected, because a client who understands their own product usually already knows where its weak points are, and a reviewer who demonstrates enough technical understanding to identify that weak point accurately, rather than manufacturing a generic complaint, tends to earn more trust from that client rather than less, even when the conversation itself is slightly uncomfortable.

The Actual Lesson

A product review that tries to be purely promotional loses the credibility that makes it worth producing in the first place, and a product review that tries to be maximally critical damages a legitimate client relationship over disproportionate framing of something that may genuinely be a minor consideration. The actual skill is proportion, giving a real weakness exactly the weight it deserves relative to everything else being reviewed, communicated with enough specificity that it reads as genuine judgment rather than either a formality or an attack, and worked through directly with the client rather than decided about them from a distance.

Specific client names, products, and review details remain confidential given the nature of this work. Happy to discuss the general approach to balancing credibility and fairness in sponsored or client based review content with anyone working in similar review or content production contexts through the proper channel.

Written by Mohammad Farhan Habib Faraz
Senior Prompt Engineer and Prompt Team Lead at PowerinAI
www.powerinai.com

Reusing A Prompt System Across Clients Without Turning It Into A One Size Fits All Failure

FARHAN HABIB FARAZ — Tue, 25 Aug 2026 06:21:11 +0000

Building a custom GPT for one ministry client teaches you something specific about that ministry. Building the third or fourth one for a different government or enterprise client teaches you something much harder, which is how much of what worked the first time was actually general, and how much of it only worked because it happened to fit that particular institution.

The Temptation That Causes The Most Damage

After the first successful deployment, the obvious next move is treating that system prompt as a proven template and adapting it lightly for the next client. Swap the knowledge base, adjust a few tone instructions, change the scope boundaries to match the new domain, and ship it faster than building from scratch. That instinct is not wrong exactly, but acting on it without first separating what was actually general from what was incidentally specific to the first client produces a second deployment that quietly inherits assumptions nobody meant to carry forward.

The clearest example of this showed up around scope boundary language. The refusal and redirection instructions built for the first ministry deployment had been carefully tuned against that specific institution's culture, a fairly formal, procedurally strict environment where a firm, precise boundary read as competent and appropriate. Carrying that same boundary language into a private enterprise deployment, where the internal culture was considerably less formal and staff expected a more conversational tone even when the bot was declining to answer something outside its scope, produced a tool that technically enforced the correct scope but felt oddly cold and bureaucratic to an audience that had no institutional reason to expect that register.

Nothing about that was a bug in the traditional sense. The logic was sound, the boundary was correctly enforced, and it still felt wrong, because the tone calibration underneath the logic had been implicitly trained against one specific institutional culture and never actually generalized.

Learning To Separate The Skeleton From The Skin

What actually needed to happen, and what took a couple of deployments to fully recognize, was treating every system prompt built for a client as having two distinct layers that needed to be developed and maintained separately. One layer is structural, the actual logic governing what the tool is allowed to answer, how it handles uncertainty, how it escalates outside its scope, the verification discipline preventing it from generating unsupported claims. That layer, once genuinely proven to work, really does generalize well across different institutional contexts, because the underlying reasoning about safe and reliable behavior does not change much based on who is asking.

The second layer is register, meaning tone, formality, warmth, how directly versus how gently a boundary gets communicated, how much personality the responses are allowed to carry. That layer does not generalize at all, and treating it as though it does is exactly what produced the mismatch in the enterprise deployment. It has to be rebuilt, or at minimum substantially recalibrated, against the specific culture of whichever institution the tool is being handed to next.

Once that separation became explicit, template reuse actually started working the way it was supposed to. The structural layer, the part that took the most careful engineering and the most testing to get reliably safe, could genuinely carry forward with confidence. The register layer got treated every single time as something requiring fresh calibration specific to that client, informed by actual conversations with people who understood how that particular institution communicated internally, rather than assumed from what had worked previously.

Why Skipping This Step Costs More Than It Saves

The pressure to move fast on a new client deployment makes it tempting to skip dedicated register calibration and just ship the adapted template, especially when the structural logic underneath is already proven and testing feels like it should mainly be about scope and knowledge base accuracy. That shortcut produces a tool that works correctly and still generates a steady undercurrent of mild dissatisfaction from users who cannot always articulate exactly why the tool feels slightly off, because the actual gap is not in what the tool says, it is in how the tool says it relative to what that specific environment expects.

Catching that gap early, through direct conversations with people inside the new client's organization about how their teams actually talk to each other, turned out to be far cheaper than catching it later through a slow accumulation of quiet user disengagement that is much harder to diagnose after the fact.

The Actual Lesson

A reusable prompt system is not one template that gets lightly reskinned for each new client, it is a stable structural core deliberately paired with a register layer that gets rebuilt from scratch every time, informed by the specific culture of whoever is going to be talking to it. Treating those two layers as one thing is the fastest way to make a genuinely proven system quietly underperform the moment it leaves the context it was originally built for.

Specific client deployments and system architecture details remain confidential given the nature of this work. Happy to discuss the general approach to building reusable prompt systems across institutional clients with anyone doing similar multi client deployment work through the proper channel.

Written by Mohammad Farhan Habib Faraz
Senior Prompt Engineer and Prompt Team Lead at PowerinAI
www.powerinai.com

Measuring Whether A Training Actually Worked Is Harder Than Delivering It

FARHAN HABIB FARAZ — Tue, 25 Aug 2026 06:17:05 +0000

Every PIAI Academy session ends with participants nodding, taking notes, sometimes even applauding. None of that tells you whether the training actually changed how they work three weeks later. Building actual measurement into training design, rather than treating a smooth session as proof of success, turned out to be one of the more neglected pieces of building out the academy's programs.

Satisfaction Is Not The Same Thing As Capability

The easiest signal to collect after a session is also the least useful one. A feedback form asking whether participants found the session valuable will almost always come back positive, because a well delivered session, regardless of how much actual capability it built, tends to feel valuable in the moment. Good pacing, an engaging trainer, a room that laughed at the right points, all of that produces genuine positive sentiment on a feedback form without necessarily producing someone who can walk away and independently apply what they were shown.

Relying on that signal alone for a while produced a comfortable but misleading picture, sessions consistently scored well, and it was tempting to treat that as confirmation the material was working. What eventually forced a harder look was noticing that participants who had rated a session highly were, weeks later, still making the same category of prompting mistake the session had specifically been designed to fix. The session had been enjoyable and had not actually closed the gap it was built to close.

Building A Signal That Actually Tests Capability

The fix meant separating the question of whether a session felt good from the question of whether it worked, and building a distinct way to measure the second one. That meant introducing a short practical exercise near the end of a session, not a quiz about definitions, but an actual small task requiring participants to apply the core technique just taught to a new situation they had not seen during the session itself.

That single addition changed what sessions revealed almost immediately. A participant could follow along attentively for the entire session, nod at the right moments, and then visibly struggle the instant they had to apply the technique independently to even a slightly unfamiliar scenario. That gap, between following an explanation and being able to reproduce the underlying judgment without guidance, is exactly the gap a satisfaction survey can never see, because it only exists at the moment of independent application.

Seeing that gap consistently across sessions was uncomfortable at first, because it meant material that had been scoring well on feedback forms was not actually landing the way the numbers suggested. It was also the only way to find out, and every subsequent redesign of a module got built around closing specifically the gaps that practical exercise revealed, rather than around what the feedback forms had been quietly implying was already fine.

Why This Mattered More For Government Sessions Specifically

The stakes of this gap are higher in a government training context than almost anywhere else the academy operates, because a ministry participant who leaves a session with a false sense of confidence, believing they understood a technique they actually only followed passively, is more likely to apply that technique incorrectly in a real institutional context afterward, without anyone catching the misunderstanding until it shows up somewhere consequential.

That risk pushed the practical exercise requirement harder in government sessions specifically, even though it takes more session time and slows the pacing that participants sometimes visibly want to move past quickly. The tradeoff, a slightly less smooth feeling session in exchange for an honest signal about whether the room actually absorbed the material, was worth accepting deliberately rather than optimizing purely for how good the session felt while it was happening.

The Longer Term Follow Up That Closed The Loop Further

Even the in session practical exercise only tests immediate application, not retention. The next layer that got added, though harder to execute consistently, involved a brief follow up check some weeks after a session, a short prompt or scenario sent to participants asking them to apply the same core technique again, cold, without the session's scaffolding present. That follow up consistently surfaced a further gap between what a practical exercise measured during the session and what actually stuck once the immediate context of the training had faded.

That longer feedback loop is slower and harder to run consistently across every session, but it is the only real test of whether training produced lasting capability rather than a temporary demonstration that decayed the moment the room emptied.

The Actual Lesson

A training session that feels successful and a training session that actually builds capability are measured by completely different signals, and defaulting to the easier signal, how the room felt, produces a comfortable but inaccurate picture of whether the work is actually succeeding. Building in a genuine test of independent application, even at the cost of a smoother feeling session, is the only way to find out whether a training program is actually doing its job or just performing well.

Specific session results, participant performance, and training outcomes remain confidential given the nature of this work. Happy to discuss the general approach to training evaluation design with anyone building capacity programs that need to demonstrate real impact through the proper channel.

Written by Mohammad Farhan Habib Faraz
Senior Prompt Engineer and Prompt Team Lead at PowerinAI
www.powerinai.com

Maintaining A Government Custom GPT After The Training Ends Is A Different Job Than Building It

FARHAN HABIB FARAZ — Tue, 25 Aug 2026 04:43:30 +0000

Building a custom GPT for a ministry training program gets treated as a finished deliverable the moment the training session ends and everyone walks out satisfied. What actually happens afterward, watching what the tool needs six months into real use versus what it needed on launch day, turned out to be a completely different set of problems than the ones solved during initial development.

The Assumption That Breaks First

Deployment day testing happens against the world as it exists on deployment day. The knowledge base reflects current procedures, current terminology, current organizational structure. The system prompt gets tuned against that snapshot, and by every reasonable measure at that moment, it works well.

The quiet assumption underneath that success is that the ministry's procedures, terminology, and structure will hold still. They do not. Procedures get revised. Terminology shifts, sometimes subtly, in ways that would not even register as a change to someone inside the institution but that matter enormously to a retrieval system trained against the old phrasing. Organizational responsibilities move between departments. None of this is dramatic or sudden. It accumulates quietly, and a tool that was accurate on day one can become gradually less accurate without any single obvious moment where it broke.

The failure mode this produces is particularly hard to catch, because the tool does not fail loudly. It keeps answering confidently, using the knowledge base it was given, while that knowledge base slowly drifts further from current reality. Nobody notices until someone acts on an answer that used to be correct and no longer is.

Why This Requires A Different Kind Of Attention Than Building Did

Initial development gets scoped, resourced, and reviewed carefully, because everyone involved understands it is the deliverable the whole engagement is judged against. Maintenance does not naturally get that same attention, because it does not feel like a deliverable in the same way. There is no training session to prepare for, no room full of participants to satisfy. It is quiet, ongoing, unglamorous work, and it is exactly the kind of work that gets deprioritized under the pressure of the next new project needing attention.

The approach that actually holds up treats maintenance as a scheduled discipline rather than a reactive one. Rather than waiting for someone to notice an answer has gone stale, which usually means waiting for a mistake to actually happen first, the more reliable version of this work involves periodic deliberate review of the knowledge base against current source material, checking specifically for the kind of quiet drift that would never trigger an obvious error message but would gradually degrade answer quality.

That periodic review is not the same task as the original build, and treating it as a smaller, faster version of the original build undersells what it actually requires. The original build starts from a mostly blank knowledge structure and populates it. Maintenance starts from an existing structure that people currently trust, and has to identify exactly which parts of that trusted structure have quietly become wrong, without disrupting the parts that are still accurate. That is a more delicate task than building from scratch, not a lesser one.

The Harder Problem Underneath The Obvious One

Beyond simply keeping facts current, there is a second layer that matters more over time, which is that the persona and scope boundaries built during initial development were tuned against the specific audience and specific concerns present during that original training program. As the tool continues being used past the training window, by people who were not in the original room and were not part of the original scoping conversation, the edges of that original scope get tested in ways nobody anticipated at launch.

A boundary response that felt exactly right for the original training audience can feel oddly rigid or oddly permissive to a different set of users encountering the tool for the first time, months later, with different expectations shaped by their own role. That mismatch does not show up as a technical bug. It shows up as gradually eroding trust in the tool from a segment of users nobody was specifically designing for the first time around, and it is easy to miss because it does not generate an obvious complaint, it just generates quiet disuse.

The Actual Lesson

A custom GPT deployed into an institution is not a finished artifact the moment training ends, it is the start of an ongoing relationship between the tool and an environment that keeps changing underneath it. Treating the launch as the finish line, rather than as the point where a different, quieter kind of work actually begins, is the single most common reason a genuinely well built tool ends up trusted less a year later than it was on day one, without anyone being able to point to a specific moment where it went wrong.

Specific ministry deployments, maintenance schedules, and system details remain confidential given the nature of this work. Happy to discuss the general approach to long term maintenance of institutional AI tools with anyone managing similar deployments through the proper channel.

Written by Mohammad Farhan Habib Faraz
Senior Prompt Engineer and Prompt Team Lead at PowerinAI
www.powerinai.com

How Do You Know A Trainer Is Actually Ready, Not Just Trained

FARHAN HABIB FARAZ — Tue, 25 Aug 2026 04:40:00 +0000

Handing someone the material and watching them deliver it once is not the same thing as knowing they are ready to run a session alone in front of a room that might push back. That distinction sounds obvious stated plainly, but it took real mistakes in how I evaluated readiness, specifically with Adiba and Kevin, before the difference between having covered the material and being ready to own it became something I could actually assess rather than just assume.

The Mistake Of Treating Rehearsal As Proof

The early version of readiness evaluation looked at whether someone could deliver the material correctly when walking through it with me present. That is a genuinely low bar, and I did not fully appreciate how low it was until watching what happened when the same material got delivered without that safety net in the room.

A rehearsal with the person who wrote the material sitting nearby changes the entire psychology of the delivery. Any hesitation gets silently backstopped, because both people in the room know that if something goes sideways, there is an immediate correction available. That backstop does something subtle to how a trainer paces themselves and how much genuine ownership they take over an explanation, because part of their attention is still oriented toward the safety net rather than fully toward the room.

What actually separates readiness from rehearsal is removing that backstop deliberately, before the real deployment, not during it. Watching Kevin and Adiba each run through a full session with me present but silent, refusing to intervene even when I could see a moment where I would normally have jumped in to clarify something, revealed far more about actual readiness than any number of walkthroughs with active correction available.

The Specific Gap That Rehearsal Hides

The particular failure mode that stayed invisible during supported rehearsal was handling a question the material had not anticipated. Delivering prepared content is one skill. Fielding an unexpected question live, without breaking stride or defaulting to a vague non answer, is a completely different skill, and it is exactly the skill that determines whether a trainer holds a room or loses it the first time someone asks something the slide deck did not cover.

During supported sessions, an unanticipated question would sometimes get quietly redirected to me, almost by reflex, because I was sitting right there and it felt natural to let the person with more direct experience answer it. That reflex, on both sides, meant the actual skill of handling the unexpected was never really being tested during rehearsal, no matter how many times the core material got run through successfully.

The fix was structuring an evaluation session specifically designed to surface that gap, rather than assuming it would show up naturally. That meant deliberately bringing harder, less anticipated questions into a mock session myself, playing a more difficult version of a real participant than any real audience had actually been so far, specifically to see whether the trainer could hold their footing without an easy way to hand the question off.

What Actually Signals Readiness

The signal that mattered was not whether every answer given during that harder mock session was perfect. It was whether the trainer could recognize the edge of their own knowledge in real time and handle that edge gracefully, either by reasoning toward a defensible answer using the same underlying logic the material was built on, or by clearly and confidently acknowledging the boundary and committing to follow up, rather than either freezing or improvising something ungrounded just to fill the silence.

Kevin's readiness showed up specifically in how he handled a deliberately difficult hypothetical about a scenario the training material had never directly covered, working through it out loud using the same reasoning framework the material taught rather than defaulting to a memorized answer that did not quite fit. Adiba's showed up in a different way, in how comfortable she became telling a mock participant that a specific question was outside what that session was scoped to answer, without that boundary sounding like a weakness or a dodge.

Neither of those moments would have been visible in a standard supported rehearsal, because a supported rehearsal never actually puts someone in the position of needing to demonstrate either skill.

Why This Changed How Every Trainer Gets Evaluated Since

Once that gap became visible, readiness evaluation stopped being about whether someone could deliver the material and became specifically about whether they could handle the material breaking a little, live, without a backstop. Every trainer evaluation since has deliberately included that harder, unsupported layer, because skipping straight from successful rehearsal to a live deployment was exactly the gap that had gone untested before.

The Actual Lesson

Competence with prepared material and readiness to own a room are not the same measurement, and evaluating only the first one creates a false sense of security about the second. The real test of whether someone is ready is not whether they can deliver what they were given. It is whether they can hold their footing at the exact point where what they were given runs out.

Specific evaluation sessions and individual performance details remain confidential given the nature of this work. Happy to discuss the general approach to trainer readiness evaluation with anyone building or scaling a training program through the proper channel.

Written by Mohammad Farhan Habib Faraz
Senior Prompt Engineer and Prompt Team Lead at PowerinAI
www.powerinai.com