DEV Community: Yaniv Shenhav

Working With a Personal AI Assistant Without Losing Control

Yaniv Shenhav — Mon, 02 Nov 2026 07:49:32 +0000

Adapted from the Hebrew guide "How to work with a personal AI assistant without giving up control" on AI NEWS IL (12 September 2026): https://ainewsil.co.il/articles/working-with-personal-ai-assistant

The labels chatbot, assistant and agent do not mark a uniform boundary between products. Some chat services keep context or memory between conversations; others do not, or let you turn the capability off. Assistants and agents may also act (coordinate, draft, track), depending on the product and the permissions you grant. The capabilities that make these tools useful are also the risk: accumulated memory is a store of personal data, and the ability to act is a permission that has to be managed.

The category is young and products change fast, so the framework below is built to stay correct when tools are swapped: defined tasks, minimal permissions, approval before irreversible action, and routine human review.

Two lists before you connect anything

The first list is not "what is the assistant good at" but "which tasks do I delegate". A task fits delegation if it repeats, its output can be checked in a minute, and a mistake is reversible. Good candidates: a weekly calendar summary, draft replies that always pass through you, tracking an open topic, collecting background before a meeting.

The second list is what never gets delegated: payments and orders, sensitive messages to family or your boss, any decision that affects another person. The line is not the tool's capability but the kind of mistake possible: the harder an action is to undo and the more it touches someone else, the more it stays in your hands.

Each connection is a separate decision

Mail, calendar, files and contacts are not a bundle. Ask three questions about each connection: which delegated task needs it, what happens if this content leaks, and how do you disconnect it. Information you are obligated to protect (a client's, an employer's) does not enter a personal assistant without the owner's explicit approval. Start with one connection, such as calendar only, work for two weeks, then consider the next.

Some actions always need your yes

Define the list up front: sending a message to another person, purchases and bookings, changing or deleting information, sharing a file. The practical test: before letting the assistant act alone, imagine the worst plausible mistake. If it embarrasses you in front of someone or costs money, it stays behind approval.

Check outputs without spending the day on it

An assistant that knows your context produces more convincing output, so errors are easier to miss. Keep fixed checkpoints per output type. Meeting summary: decisions and owners. Email draft: recipient, tone, numbers. Week plan: dates and conflicts. The warning sign is when you stopped checking because "it is usually right".

Manage memory actively

Once a quarter, review what the assistant stored about you: delete what is stale or sensitive, update preferences. Stale memory is a source of errors, like recommendations based on a preference you changed. Ask before you start, not after: "If I want to leave in a year, how do I take what is mine and delete the rest?"

A week of caution saves a month of cleanup.

This English adaptation was prepared with AI assistance.

Yaniv Shenhav.

A New Model Just Launched. Five Questions Before You Change Anything

Yaniv Shenhav — Mon, 26 Oct 2026 07:54:53 +0000

Adapted from the Hebrew guide "How to Read a New AI Model Announcement Without Falling for the Hype" on AI NEWS IL (published 3 August 2026).

An impressive benchmark is a starting point, not a conclusion. Every major launch arrives with a headline number, and a headline number is not an instruction to replace a tool. Five questions separate a shiny announcement from a useful improvement.

1. Who chose the benchmark?

A comparative figure becomes meaningful only when you know the task it measured, the control group, the evaluation setup and whether the result can be reproduced. Until then, treat each comparative number in a launch as a claim that still has to meet a real task.

2. What exactly was measured, and under which conditions?

Read the small print before the headline. Which model version ran? Were external tools allowed? How many attempts were made, and does the table show the best run or the typical one? An improvement on a coding benchmark does not guarantee an improvement on a writing task, and the reverse is also true - a high score in one domain says little about another.

3. What will the user actually feel?

Next to quality, check availability, speed, message limits, privacy and price. A theoretical gain that is not accessible in the product you use - or that makes the process significantly more expensive - may change nothing in daily work. The question is not whether the model is better, but whether the improvement is available to you in practice.

4. Does it improve a metric you defined in advance?

The short path to a decision is a small experiment. Take one real task from your pipeline, define the metric before you run anything, and compare old against new under identical conditions. If the predefined metric does not move, there is no need to rush and switch.

5. What decision does this announcement actually demand?

Usually: none yet. Product announcements shape real purchasing and adoption decisions, and critical reading prevents unnecessary switching between tools. An announcement is an invitation to test, not an instruction to replace a tool.

One step to take now

Keep a fixed set of ten representative tasks from your real work. Every significant launch, run the same set under the same conditions and record quality, time and cost. After a few launches you have something no launch table gives you: a trend line measured on your own work.

Sources used in the original: OpenAI News, Google AI, Anthropic News.

This English adaptation was prepared with AI assistance.

Yaniv Shenhav.

AI Agent Memory Needs a Correction Path, Not Just Storage

Yaniv Shenhav — Mon, 19 Oct 2026 10:29:11 +0000

Adapted from the Hebrew guide "When the Agent Remembers Something That Has Changed: Managing AI Memory in an Organization" on AI NEWS IL (published 8 September 2026).

Memory helps an agent keep context. It can also return an old price, a changed preference or a procedure that was cancelled. Storing something well does not make it correct today.

This guide turns that risk into decisions and tests for an agent pilot. The examples are hypothetical, not case studies or measured results.

The direct answer: every memory needs a correction path

Do not let an agent keep every detail forever just because storage is available. Decide what is worth keeping, where it came from, when it was last checked and what happens when it changes. If validity cannot be checked, the agent should go back to the approved source or ask for clarification.

The goal of memory is to cut repeated work, not to create a second store that nobody owns.

What the AWS post adds

On 4 September 2026 AWS published a guide to designing lifecycle policies for AgentCore Memory. It describes the risk of stale information piling up and suggests patterns for evaluating, consolidating and reducing memories, with an AWS example implementation.

Do not read it as saying every memory is deleted automatically, that every component ships preconfigured, or that one policy fits every organization. The useful point for any stack is the difference between storing information and maintaining knowledge. The process below is an applied proposal, not an AWS benchmark.

Step 1: Map what the agent actually stores

List the types of information that enter memory in the process you chose: a language preference, a meeting summary, a service request status, an explanation of a procedure. Each changes at a different rate and means something different. Do not mix a detail a user shared about themselves with an organizational rule approved in a formal document.

Trace a real path. An agent may write a summary to a database, keep chat history and add a snippet to a separate search index. Fixing one place does not fix the others. Pick a non-sensitive example and follow it: which message created it, where it was stored, who can read it and in which answer it comes back.

Step 2: Separate memory from the source of authority

In a hypothetical case, a sales rep asks the agent about promotion terms early in the month. The agent stores a summary. The promotion then ends and the approved page is updated. If the old summary wins over the current page, wrong terms can reach a customer.

Decide in advance which source may answer questions about price, eligibility or process stage, and when it must be rechecked. Conversation memory can explain what the customer asked. It should not become the terms document.

Attach to each item a source ID, a creation date, a last-checked date and an owner. If two sources conflict, do not ask the agent to pick the more convincing one. Set a professional tiebreak rule or route the question to a person.

Step 3: Set validity by use, not by an arbitrary number

No single number of days fits all memories. A ticket status, a language preference and an operating guide change at different speeds. Define recheck triggers: a source document changes, a request closes, the user gives an update, or a date set by the process arrives.

If you also use periodic checks, write down why the interval fits the use case and what harm could happen between two checks. Do not copy a technical default as a business policy.

Stopping the use of an item in answers is not the same as deleting it from the system. An item may need to leave retrieval results immediately, while other copies follow a separate data policy. This guide does not say how long personal data may or must be kept. For sensitive or regulated data, involve the right owners before changing policy.

Step 4: Build a way to update, retire and delete

Define the action for when an employee reports that the agent remembers something wrong. Who receives the report, how is the item found, and who may fix it? Adding a new message that says "ignore the previous one" is not enough. If both versions stay available with no rule to decide between them, another question can bring the old information back.

After a fix, test retrieval in several phrasings: a direct question, an indirect one and one that includes the old wording. Also try a new conversation, so a fix in one session is not mistaken for a fix in the store. For deletion requests, define a check for each storage layer you manage. Seeing no result in one interface does not prove every copy is gone.

Step 5: Watch for summaries that lose the condition

Merging several memories into one summary can make retrieval easier. It also changes how the information is represented. In a hypothetical case, the sentence "an item can be exchanged within a week if the packaging is sealed" becomes "exchange within a week". The answer sounds clear, but the condition is gone.

Before approving a summarization mechanism, test it on examples with exceptions, numbers and conditions. Do not remove professional distinctions just to save space. Two customers with similar names are not the same person, and two versions of a procedure are not always duplicates. Keep markers that tell items apart: process ID, scope, version and access permission. When you cannot merge with confidence, keep items separate or send them for review.

Step 6: Test in more than one language without mixing permissions

A team may use the same concept in Hebrew, English and an acronym. Prepare test questions in the common phrasings, including product names and internal terms. The aim is not to show that the model knows a language. It is to check that the agent reaches the same approved source whatever the language of the question. If translation created a separate memory, make sure a source update reaches that retrieval path too.

A language test does not replace a permission test. Sign in as an ordinary employee and as someone with a different role, and compare what each can retrieve. A memory created in a manager's conversation should not reach another team just because the topic is similar.

Step 7: Measure memory quality and decide whether to expand

Prepare a small test set with valid information, changed information, a contradiction and information the agent must not see. For each question, write the expected source and the success condition. Count how many answers relied on an old version, how many corrections reappeared and how many cases needed human review.

Response time alone is not enough. A fast answer based on wrong information is not an improvement.

End the pilot with a defined decision. If the agent uses the right sources and the correction path works, add one more information type and test it separately. If not, narrow what it may remember or go back to direct retrieval from the approved source.

What should be ready at the end of the pilot

A list of information types the agent may keep.
A map of authoritative sources.
An owner for each process.
A test set that can be rerun.
A short record of one correction from report to verified answer. Use test data rather than real personal details.

Also set a stop condition. If old information returns after a fix, pause memory for that information type and point the agent to the approved source until it is resolved. This is a pilot design suggestion, not a feature every product guarantees. Make sure the team can actually do it and that someone has the authority to decide.

One step to take now

Pick one process and prepare five questions about information that may change. Update the approved source, rerun the questions and check that the agent uses the new version and shows the right source.

This English adaptation was prepared with AI assistance.

Yaniv Shenhav.

Your First AI Agent Task Needs a Contract, Not More Permissions

Yaniv Shenhav — Mon, 12 Oct 2026 08:06:05 +0000

Adapted from the Hebrew guide "AI Agent: How to Choose a Task, Set Boundaries and Check the Result" on AI NEWS IL (published 17 September 2026, updated 24 September 2026).

An agent's first production task should be small enough to test and safe enough to stop. "Manage our support inbox" is neither. "Classify one type of support request, find the relevant policy and prepare a draft for a representative" is a better starting point.

This guide turns that narrower task into a testable contract. The support example is hypothetical, not a case study or a claim of measured results.

1. Decide whether you need an agent

Use an agent when the goal is clear but the next step depends on what it finds. A support request may require a policy lookup, another document or a handoff to a person.

If every step is fixed, ordinary automation is usually easier to predict and test. Copying a field, sending a scheduled reminder or generating a standard report does not automatically need an agent.

Before connecting anything, write one sentence:

The agent receives X, returns Y, and Z decides what happens next.

If you cannot name the output or the person responsible, the task is not ready. Start with a reversible task. Do not make payments, deletion, publication or permission changes part of the first pilot.

2. Use a narrow support-triage example

A customer writes: "My monthly charge has increased. Why?"

The agent can:

Classify the request as billing.
Retrieve the approved policy section about price changes.
Draft an explanation with a link to that section.
Flag that it has not checked the customer's invoice and cannot confirm the amount.

A representative opens the invoice, checks the policy, corrects the draft and decides whether to send it. The agent does not send the reply or update the customer record.

The point is to reduce search work without inventing service terms or exposing information. Adapt the example to your systems, privacy rules and business risks before using it.

3. Define an input and output contract

Allow only the information needed: the request text, a minimal customer reference, the response language and links to approved policies. Keep passwords, payment details and unrelated files out of the model input.

Use a fixed output shape:

Field	What the reviewer checks
Category	Is it one of the allowed categories?
Confidence or uncertainty	What remains unresolved?
Source section	Does the cited policy support the answer?
Draft reply	Has the agent added a promise or changed the policy's meaning?
Items to verify	What must the representative check before sending?

Define invalid outputs too. A missing source link, an unknown category or an unsupported refund promise should produce "review required" with a reason, rather than a ready-to-send draft.

A missing or conflicting document is a reason to stop and ask for review. It is not a blank the model should fill.

4. Separate reading, drafting and acting

For the first test, grant read access only to the approved policy folder and test requests. Do not give the agent the entire inbox, full CRM, invoices or contact directory because they might be useful later.

Keep credentials outside prompts and user-visible logs. Limit access to the environment and operations required, and use short-lived credentials where supported.

Treat these as separate permission stages:

Read approved material.
Produce a draft.
Take an external action.

Passing the first ten cases does not authorize sending messages. That is a new capability with a new risk and needs its own decision and tests.

Ask the organization's access owner to approve the permission list. Also ask what an attacker could reach if the credential leaked. Narrow the scope before the pilot if the answer is unacceptable.

5. Test failure cases before real use

Write the expected outcome for each case before running it: draft, clarification or stop.

Include routine requests alongside:

Missing information or conflicting policies.
An outdated document or the wrong customer reference.
An unavailable external service.
Sensitive information or a request requiring a financial decision.
Instructions embedded in a customer message or file that conflict with the workflow.

For example, a customer message might contain a demand to ignore the policy and approve a full refund. The agent should apply the approved policy and flag the request for review. The message is task data, not permission to change the workflow.

Measure category accuracy, source fit, changes in meaning, reviewer corrections, time to approval and cost. There is no universal number of test cases that proves readiness. Expand the test set according to task variety, error severity and how easily harm can be detected.

6. Make stopping and rollback concrete

Stop when a required source is missing, information is sensitive, uncertainty is unresolved, a financial decision is needed or a tool attempts an operation outside the allowed scope.

Name one process owner who can stop the pilot. Record the limited inputs, sources, tool calls, output and human decision under your privacy policy. Changes to the model, prompt, data or permissions should trigger regression tests.

Before the pilot, rehearse rollback: revoke the credential, return requests to representatives and verify that the manual process works. A rollback plan that has never been tried is still an assumption.

7. Measure accepted work, not generated drafts

Run a short, defined draft-only pilot after the test set passes. Two weeks is an example, not a universal requirement.

For each draft, record whether it was:

Used with little change.
Used after corrections.
Rejected.

Track review time and whether the source was correct. Compare similar request types with and without the agent. A fast draft is not a saving if checking it takes longer than writing the reply manually.

Set the continuation rule before the pilot. For example, continue only if corrections are small, no draft invents service terms and handling time falls. Stop for a data leak, skipped human review or costs that exceed the benefit.

The first milestone is a useful draft under controlled permissions. More access is a separate decision, not a reward for a convincing demo.

AI Knowledge Base Without Stale Answers: A Practical Guide to Freshness, Permissions and Sources

Yaniv Shenhav — Mon, 05 Oct 2026 08:12:01 +0000

Adapted from the Hebrew guide on AI NEWS IL.

A knowledge base for an AI assistant does not start by uploading every file into one folder. It starts with a source register: an owner, a validity window and an access rule for each source. When retrieval, permissions and the update path work together, you connect it to real users.

The goal is not a model that never errs. The goal is an error you can detect: which source was returned, when it was approved, who may see it, what was missing and who fixes it. A knowledge base that cannot retire an old document or honor a permission change will give fluent, wrong answers even if the model never changed.

1. Build a source register before you index

Create one row per source, not per folder. Each row answers: what is the document, who owns it, who is it for, when is it valid, what replaces it, and how is it removed from the index.

Do not assume the newest file wins. Upload date can be later than content date, and a copy moved between folders can look new while being stale. Define a precedence order per information type. A signed policy may beat a training deck, and a numbered correction notice may beat the policy until a new version ships. If there is no rule, the right state is conflict. Do not ask the model to pick the document that sounds more convincing.

2. Separate active, historical and draft content

A historical document can matter for audit, but it must not silently compete with the active one. Keep it in an archive behind access control and remove it from the normal answer path with a status field and an expiry date. A draft stays out of the active store until the owner approves it. A filename that says "final" is not approval.

Useful metadata at minimum:

source_id: expenses-policy-2026
owner: ops-team
audience: all-employees
status: active        # draft | active | superseded | expired
effective_from: 2026-01-01
reviewed_at: 2026-09-01
version: 3
supersedes: expenses-policy-2025

Some search products can filter on file attributes. That capability does not set your policy. Define what each field means and who may change it first, then implement filters in the product you chose.

3. Keep permissions end to end

Access to the knowledge base is not permission to see everything in it. At ingest, store the audience of the source. At query time, filter results by the user's identity. In the answer, never show, quote or link a document the user may not open. A broad service account is not a reason to widen what the user sees.

Check the source permissions too. A product that respects user permissions can still surface content that was shared too widely in the past. Microsoft's documentation, for example, explains that SharePoint and OneDrive controls affect discovery without changing user permissions, and that lifecycle and sharing policies help reduce oversharing. That is a statement about Microsoft 365 Copilot, not a guarantee for every RAG tool.

Start read-only. The knowledge base admin can import, disable and rebuild. A normal user can ask. The model gets no write access to source documents. Deleting, changing an audience or replacing a version are logged admin actions, not the result of a chat.

4. Make ingestion verifiable

A typical pipeline is ingestion, text extraction, chunking, embeddings, indexing and retrieval. Google's RAG Engine documentation shows a similar chain. It describes the product. It does not prove your documents were chunked well or that the output is faithful to the source.

Log every run: run ID, source list, versions, successes and failures. A file is not searchable just because you sent it to an API. OpenAI's File Search documentation, for example, says to check that a file reached the completed state, and it describes file citations and metadata filtering. If ingestion is partial, keep the previous version active or block the topic. Do not mix half an update without a marker.

The tooling can be simple: a sheet or database for the register, an exporter or crawler, a parser, a search engine or vector store, a link checker, a set of reference questions, and a log that ties each question to the retrieved results and the answer.

5. Require an openable, versioned citation

The answer should show a clear source name and a link that opens with the user's permissions. For testing, also keep source_id, version and chunk ID. A filename alone can change or repeat across folders. If the source cannot be opened, the citation does not let the user verify the claim.

A citation shows the system linked a claim to a file. It does not prove the claim follows from it. When testing, compare each material claim to the source sentence, check that the context was not cut off, and check that the source was valid at answer time. Also sample answers where a similar paragraph was found but belongs to another audience, product or period.

6. Build an acceptance test for retrieval

Do not only check whether the wording sounds good. For each question, store the expected result at three levels: which sources should be retrieved, what may be concluded from them, and what the correct decision state is. That lets you tell a retrieval failure from a wording failure. If the right document never reached the context, a better prompt is not the fix.

Include:

a normal question and a rephrased one
a technical term in another language
a superseded document
two conflicting versions
a user without permission
a file that failed ingestion
a question with no answer
a deleted link

There is no universal number of questions that guarantees quality. Pick coverage by content variety and the harm a wrong answer can cause. Useful metrics: right-source retrieval rate, share of answers with a valid source, conflicts detected, permission leaks, unanswered questions that correctly stopped, and time from approval to active. An average cannot make up for a permission leak or use of a revoked document.

7. Update and remove without a gray zone

Define a change path: the owner approves a version, the admin ingests it, regression tests run, and only then does an alias switch to the new active index. Keep a way back to the previous index, but never restore a document that was revoked for safety or privacy reasons. A technical rollback is subject to the business status of the source.

When a permission is removed or a document is revoked, set a response time and check every copy: source, cache, index, saved results and the test environment. Do not promise instant deletion if the product describes another retention period. Record what was deleted, what remains under policy, and when you checked.

Example (hypothetical): expense policy

This is a proposed design that was not tested in production. An ops team wants the assistant to answer travel expense questions. There is an active policy, an old training deck and a draft for next year.

Input: the active policy and an approved correction notice. The deck is marked superseded. The draft stays out of the active store.
Output: a short answer, an amount or rule only if it appears in a valid source, a link to the section, the version, and a state of answered or needs-clarification.
Permissions: all employees can read the policy. Only ops manages versions. The model cannot write or approve expenses.
Tests: a regular employee, a unit with an exception, Hebrew and English phrasing, a question about next year, a broken link, and a conflict between the notice and the policy.
Stop conditions: no active source found, two equal-rank sources conflict, the user is not authorized, a file did not finish ingestion, or the answer adds a condition the source does not contain.

"Contact ops" is an acceptable result when there is no valid source. It is expected behavior that keeps the model from inventing policy.

Failure cases

Two active versions: stop with conflict, show both sources to the owner, do not pick automatically.
Partial ingestion: do not turn on the new index. Log the file and the error, retry without lowering the threshold.
Source too widely shared: fix the sharing or remove the source until a decision. An app-level filter does not replace source access control.
Valid document not retrieved: mark a retrieval failure. Check extraction, chunking, metadata and the query before you rewrite the answer.
Citation does not support the claim: block the claim, fix the context check, add the case to regression.
No answer in the base: return no-valid-source with an escalation path. Do not fill in general knowledge as if it were company policy.
Broken link: show that the source cannot be verified and route it to the owner.
Permission changed after indexing: run removal or rebuild, invalidate caches, and test with the user's identity that access is gone.

Limits

RAG or file search does not make a source correct, current or permitted. Chunking can separate a rule from its exception, metadata can be wrong, retrieval can miss a term, and a model can state a conclusion the source does not support even with the right context. Permissions in one product do not prove permissions in another.

This guide does not replace data classification, a privacy assessment, legal advice, records management or a vendor contract review. Retention, deletion, processing region and sub-processors depend on the product, the plan and the real settings. Before using personal, confidential or regulated material, get approval from the responsible people in your organization.

Sources

NIST AI 600-1, Generative AI Profile (published July 26, 2024). A voluntary cross-cutting framework for AI risk management. The source register and worksheet here are a proposed implementation, not a NIST requirement.
OpenAI API, File Search documentation. Describes ingestion status, file citations, search result inclusion and metadata filtering for that product.
Microsoft Learn, Microsoft 365 Copilot data protection architecture. Explains how permissions, sensitivity labels, sharing and lifecycle policies affect Copilot. Not a deployment guide for every RAG system.
Google Cloud, Vertex AI RAG Engine overview. Lists the ingestion, transformation, embeddings, indexing, retrieval and generation chain.

Originally published in Hebrew on AI NEWS IL (ainewsil.co.il), with a printable worksheet. This is an English adaptation.

The full Hebrew guide, with the printable worksheet, is on AI NEWS IL: read the original article. The Israeli Institute for AI runs AI training and implementation programs for organizations worldwide.

Seven Controls to Put in Place Before an AI Agent Gets Production Access

Yaniv Shenhav — Sun, 20 Sep 2026 19:26:15 +0000

AI agents cross a line that ordinary chat systems do not: they connect probabilistic decisions to real actions.

An agent that can read mail, update a record, run a tool, or spend money is part of your production control plane. Prompt quality matters, but prompts are not a security boundary. The controls below should exist before an agent receives production access.

1. Separate read, recommend, and execute

Do not treat "access" as a single permission.

Use three distinct stages:

Read - retrieve the minimum data needed for the task.
Recommend - prepare a proposed action without changing external state.
Execute - make the approved change through a narrowly scoped tool.

This split gives you useful checkpoints. A support agent may read a ticket and draft an answer automatically, while sending remains gated. A finance agent may analyze invoices but require an approval before changing payment details.

2. Give every tool its own narrow permission

A single broad credential turns every mistake into a high-impact mistake. Prefer separate credentials and scopes for separate tools.

Ask four questions for each tool:

Which resources can it access?
Which operations can it perform?
How long does the permission last?
Which environment does it apply to?

Short-lived credentials and environment-specific roles reduce the damage from a bad decision, a leaked token, or a malicious instruction hidden in external content.

3. Add an action budget

Agents need more than an API rate limit. They need a business-action budget.

Useful limits include:

maximum records changed per run;
maximum messages sent per hour;
maximum cost per transaction and per day;
maximum number of retries;
maximum number of external systems touched in one workflow.

The budget should fail closed. Crossing it should stop the run and request review, not trigger an improvised workaround.

4. Put approval at the point of commitment

Approval works only when the reviewer can see the exact action that will occur.

For a message, show the final recipient and wording together. For a deployment, show the target environment and revision. For a purchase, show the item, delivery address, and full total.

Avoid vague approvals such as "continue" when several actions are still possible. The approval object should be concrete enough to log and later audit.

5. Treat external content as untrusted input

Email, webpages, documents, issue comments, and retrieved knowledge can all contain instructions that conflict with the agent's actual task.

Do not let retrieved content redefine the agent's goals, choose new tools, expand permissions, or select a new disclosure destination. Parse it as data. Keep authority in a separate, trusted control path.

Test whole workflows rather than individual prompts. A sequence of individually allowed actions can still produce a harmful result.

6. Log decisions and effects separately

A useful audit trail answers two different questions:

Why did the agent decide to act?
What changed in the external system?

Record the task identifier, tool, target, approval reference, before/after state, and provider response. Keep logs outside the agent's own write permissions when possible.

Observability should also include alerts for unusual tool combinations, repeated failures, sudden increases in volume, and actions outside normal hours.

7. Build a real stop mechanism

A stop button is not a prompt that says "please stop." It is an independent control that can revoke credentials, disable tool execution, or block the workflow runner.

Define who can activate it, what it disables, and how recovery works. Practice using it before launch.

A small pre-production failure drill

Before giving an agent live access, run at least these tests:

A document contains an instruction that conflicts with the assigned task.
The agent is given the wrong recipient or resource identifier.
A required external service fails halfway through the workflow.
The same event is delivered twice.
An approval expires before execution.
The agent tries to exceed its action budget.
The stop mechanism is activated during a run.

The goal is not to prove that the agent never fails. It is to prove that failures are bounded, visible, and recoverable.

The minimum launch checklist

Before production access, confirm that you have:

a tool allowlist;
least-privilege, short-lived credentials;
separate read and write paths;
explicit approval for high-impact actions;
action and cost limits;
idempotency protection;
tamper-resistant logs;
monitoring and alerts;
a credential-level stop mechanism;
tested failure and recovery paths.

An AI agent should earn more authority only after its behavior is observable under real conditions. Start with read-only access, measure what happens, and expand one permission at a time.

This article is an English technical adaptation of the original Hebrew guide published by AI NEWS Israel: לפני שנותנים לסוכן הרשאה: שבע בקרות שחייבות להגיע לפני הייצור.

DEV Community: Yaniv Shenhav

Working With a Personal AI Assistant Without Losing Control

Two lists before you connect anything

Each connection is a separate decision

Some actions always need your yes

Check outputs without spending the day on it

Manage memory actively

A New Model Just Launched. Five Questions Before You Change Anything

1. Who chose the benchmark?

2. What exactly was measured, and under which conditions?

3. What will the user actually feel?

4. Does it improve a metric you defined in advance?

5. What decision does this announcement actually demand?

One step to take now

AI Agent Memory Needs a Correction Path, Not Just Storage

The direct answer: every memory needs a correction path

What the AWS post adds

Step 1: Map what the agent actually stores

Step 2: Separate memory from the source of authority

Step 3: Set validity by use, not by an arbitrary number

Step 4: Build a way to update, retire and delete

Step 5: Watch for summaries that lose the condition

Step 6: Test in more than one language without mixing permissions

Step 7: Measure memory quality and decide whether to expand

What should be ready at the end of the pilot

One step to take now

Your First AI Agent Task Needs a Contract, Not More Permissions

1. Decide whether you need an agent

2. Use a narrow support-triage example

3. Define an input and output contract

4. Separate reading, drafting and acting

5. Test failure cases before real use

6. Make stopping and rollback concrete

7. Measure accepted work, not generated drafts

Further reading

AI Knowledge Base Without Stale Answers: A Practical Guide to Freshness, Permissions and Sources

1. Build a source register before you index

2. Separate active, historical and draft content

3. Keep permissions end to end

4. Make ingestion verifiable

5. Require an openable, versioned citation

6. Build an acceptance test for retrieval

7. Update and remove without a gray zone

Example (hypothetical): expense policy

Failure cases

Limits

Sources

Seven Controls to Put in Place Before an AI Agent Gets Production Access

1. Separate read, recommend, and execute

2. Give every tool its own narrow permission

3. Add an action budget

4. Put approval at the point of commitment

5. Treat external content as untrusted input

6. Log decisions and effects separately

7. Build a real stop mechanism

A small pre-production failure drill

The minimum launch checklist