Test the handoffs before trusting an always-on AI coworker

An AI coworker demo usually shows a successful conversation. An operational evaluation should show what happens when that conversation moves between a web application, a team channel and a background task. The difficult boundary is often the handoff: who is speaking, which permissions apply, and what happens to work that is already running?

Consider an internal assistant that investigates a failed deployment and drafts a status update. Before giving it production access, run the following exercise with synthetic incidents and a test workspace. The goal is to expose state and permission failures without depending on a particular model's eloquence.

Start with an ownership map

Write down who operates the conversation store, scheduler, model connection, tool execution and messaging integration. For every component, name the person or team responsible for recovery. A hosted interface does not answer all of these questions, and deploying source code does not establish an operating process.

Our comparison of OpenAI Dots and CopilotKit OpenDots is useful background for this ownership discussion. Use it to frame procurement questions, then verify the behavior of the actual deployment you will use. The tests below are proposed acceptance criteria, not reported test results for either product.

Test identity across two surfaces

Give one test user access to an incident record and deny another user access. Ask both to request the same summary from the web interface and the team channel. The denied request must stay denied, including when the second user quotes a previous answer or supplies the incident identifier directly.

CopilotKit's Channels documentation describes application-controlled mapping between channel users and application users. Treat that mapping as a security boundary. Log the resolved application identity and permission decision, rather than trusting a display name carried in message text.

Also test group conversations. A person may be authorized to read an incident privately while the channel audience is not. Retrieval permission and permission to disclose the result to everyone in a room should be separate checks.

Interrupt a task at an inconvenient moment

Start the investigation, pause it after evidence retrieval, and restart the worker. Require a visible task identifier and a clear record of the last completed step. The assistant should resume from recorded state or explain that it cannot safely continue.

Deliver the same event twice. The system should recognize the duplicate or safely repeat a read; it should not generate two external status updates. Then revoke the user's incident access while the task is paused. Resumption should reevaluate authorization before retrieving more evidence or disclosing stored material.

Set a deadline and a maximum number of tool calls for the exercise. Confirm that cancellation prevents new work from being scheduled, and explicitly document any in-flight operation that cannot be interrupted.

Bind approval to a concrete action

Show the proposed status message, destination and evidence timestamp together. If the draft or destination changes after approval, require a fresh decision. A generic acknowledgment such as "looks good" should not authorize an unspecified future message.

Finish by producing a compact receipt: initiating identity, task identifier, evidence references, approval decision and external result. Keep sensitive document contents out of routine operational logs.

Score the exercise by permission consistency, duplicate handling, recovery behavior and the completeness of that receipt. A fluent answer is useful, but a coworker becomes dependable when a human can reconstruct what it did across every handoff.