DEV Community

Precision AI OS
Precision AI OS

Posted on

Human in the loop AI for client work: where to put the approval step

A reminder goes to a client who paid last week. A client is told the wrong filing deadline. A draft engagement letter leaves the office with last year's rates still in it. Each one costs a partner an afternoon of apology and rework, and the client's trust takes longer to rebuild than the email took to send. So when we say human in the loop AI, we mean something specific: a person approves particular actions before they happen. Not a dashboard someone checks when they remember. Not a review screen that appears after the send. Approval that sits in the path of the action and stops it until a named person says yes.

That definition matters because a lot of the disappointment with agents in accounting and law practices isn't a model problem. It's a placement problem. The approval sits in the wrong place, or nowhere, or everywhere.

Place it nowhere and something leaves the building that should not have. Place it everywhere and people stop reading. Both failures cost money. The first costs clients. The second costs the automation, because the firm quietly stops using it and goes back to doing the work by hand.

Classify by reversibility and audience

We sort every action an agent can take on two axes. Can we undo it? Who sees it? A draft that stays inside the firm and can be deleted is a different animal from an email that leaves the building with a partner's name on it. A calendar hold is different from a payment instruction.

The default we have landed on is simple. If the action is client-facing, it waits for approval. If it is irreversible, it waits for approval. If it is both, it waits for approval from the person who is accountable for the outcome, not just anyone with a login.

  • Internal and reversible: let it run, keep a log.
  • Internal but awkward to unwind: let it run with a short undo window and a notification.
  • Client-facing: approve before send, every time.
  • Money, filings, deadline changes, permissions: approve, and name the role that owns it.

This is a policy, not a prompt. It lives in configuration that a partner or ops lead can read and change, not in the system message of an agent. If the policy is buried in a prompt, nobody can audit it and nobody can explain it to a client. It also has to be versioned. When the rules change, you want to know which items ran under the old ones, and the people who own the rules should be able to read the diff.

One more distinction helps. Some actions are safe to reverse technically but expensive to reverse socially. Sending a corrected invoice is easy. Un-sending the wrong one is not. We treat social irreversibility as real irreversibility.

Make the review worth doing

Approval screens fail in a predictable way. The reviewer has thirty items, each with a green button, and no reason to slow down. Click, click, click. The step exists, the signature exists, and the judgment does not.

So the screen has to give the reviewer three things at once: the draft that will go out, the source data it came from, and a plain statement of what happens on approval. If a reminder email is about to go to a client, show the invoice status next to it. If the status says paid, the reviewer should see that without opening another tab. We also put the consequence in plain language: this sends the attached statement to three named recipients at 9am tomorrow. Reviewers decide better when the result is stated rather than implied.

Editing has to be as easy as approving. If the reviewer has to reject, go back to the source, regenerate, and start again, they will approve a draft they know is mediocre. Let them fix a line, then send. Log the edited version as the version of record. Approval should not require a meeting either. If an approval needs a conversation, the action is not ready to be queued. Change the request until a busy person can decide it in under a minute.

Batching is where a lot of the time savings come from, and a lot of the risk. Twenty drafts reviewed on one screen beats twenty separate interruptions. But a batch should not flatten exceptions. Sort items where the source data conflicts with the draft, and make those the first thing a reviewer sees.

Then log it. Who approved, what they saw, when, and which version went out. We do not log this to catch people. We log it because a client will eventually ask, an auditor may ask, and because the log is the most honest record you have when deciding what to loosen later.

policy:
  internal_meeting_summary:
    requires_approval: false
    log: true
  client_email_send:
    requires_approval: true
    approver: engagement_partner
    show: [draft, source_data, recipients]
  payment_reminder:
    requires_approval: true
    approver: billing_lead
    block_if: invoice_status == paid
  filing_deadline_change:
    requires_approval: true
    approvers_all_of: [engagement_partner, compliance]
  payment_instruction:
    requires_approval: true
    approvers_all_of: [partner, second_partner]  # two different people
Enter fullscreen mode Exit fullscreen mode

Loosen only with evidence

Two numbers tell you whether an approval step is earning its place: how often items are approved, and how often they are edited first. Look at them per action type. An overall average hides the one step that is failing.

If an action is approved almost every time and rarely edited, the step may be doing nothing except slowing the queue. That is a candidate for an undo window or a sampling review. If an action is edited often, the step is doing real work, and the better question is why the draft keeps arriving wrong. Track both numbers next to the log, so the review is a short read and not a project.

Be careful with the first case. A low edit rate can mean the work is good or the reviewers have stopped reading. Check for fast approvals on items that should take thought. We would rather keep a step too long than remove one people were quietly relying on.

And do not create approvals that only transfer blame. If a reviewer cannot evaluate the thing they are signing, their approval is not a control. It is a signature on a risk nobody looked at. Review the policy when the work changes, not on a calendar. A new template, a new filing rule, a new billing process, each one moves actions across the line. A step that was safe to auto-send last quarter may not be safe now.

Precision AI OS builds an Agent OS for professional service firms where agents prepare the work and anything that goes to a client waits for a person's approval by default.

Top comments (0)