Some of the hardest workflows to automate are not technically complicated. They are simply trapped inside software that was never designed to integrate with anything else.
A team may depend on an internal portal that is fifteen years old. Employees sign in, open several screens, search for a record, update fields, submit a form, and repeat the same sequence dozens of times a day. The application may be critical to the business while exposing no useful API at all.
Traditionally, the options have been uncomfortable. Rebuild the system, create brittle UI automation around it, or keep paying people to perform repetitive browser work manually.
Browser-capable AI agents create another option. They can navigate the interface itself, interpret the page, interact with forms, and pause for a person before a consequential action.
AWS published a reference architecture on August 13 showing exactly this pattern with Amazon Bedrock AgentCore Browser Tool and Strands Agents. The useful lesson is broader than AWS: a missing API no longer means the workflow is impossible to automate, but browser automation needs stronger boundaries than a normal API integration.
This guide walks through those boundaries.
1. Define the exact browser task first
Do not begin with a requirement such as:
Automate our legacy portal.
That is too broad.
Start with one workflow a person can describe clearly. For example:
Find a customer policy, update the mailing address, verify the new value, and stop before final submission if anything is unclear.
That gives the agent a bounded job.
A simple definition might look like:
type BrowserTask = {
goal: string;
startingUrl: string;
expectedSteps: string[];
criticalActions: string[];
requiredEvidence: string[];
};
For example:
const task: BrowserTask = {
goal: "Update the customer mailing address",
startingUrl: "https://legacy.example.com/policies",
expectedSteps: [
"Search customer",
"Open policy",
"Edit mailing address",
"Review changes"
],
criticalActions: [
"Submit final change"
],
requiredEvidence: [
"Customer identity",
"Original address",
"Updated address"
]
};
This is much easier to control than giving an agent a browser and asking it to “handle policy changes.”
2. Run the browser inside an isolated session
An AI browser agent is interacting with a real application. It may see customer data, authentication state, internal records, or sensitive forms.
That browser should not simply run inside the same environment as the rest of your application.
The safer pattern is:
Task starts
↓
Fresh browser session
↓
Task-specific authentication
↓
Agent operates the legacy app
↓
Evidence and result stored
↓
Session ends
AWS AgentCore Browser Tool uses isolated managed browser sessions. AWS documentation also allows session recording to an S3 bucket for later review and uses IAM execution roles to control which AWS resources the browser environment can access.
The implementation will differ on other platforms, but the principle is the same: treat browser execution as a contained runtime, not an invisible extension of your main application.
3. Separate observation from action
A useful browser agent repeatedly performs three jobs:
- understand the current page
- decide what should happen next
- execute the browser action
That loop can be represented simply:
Observe page
↓
Interpret state
↓
Choose next action
↓
Execute
↓
Observe again
For a traditional automation script, the observation step may rely almost entirely on selectors.
For an AI-assisted workflow, page structure, screenshots, visible text, and application state can all help the system understand what is currently happening.
AWS's reference implementation uses a vision-capable foundation model to inspect browser screenshots, determine the next action, execute it through Playwright, and repeat the cycle until the task is complete or human confirmation is required.
That is particularly useful for older applications whose interfaces are difficult to model through a clean API contract.
4. Keep browser actions explicit
Even when the model decides what should happen next, the actual browser operations should remain narrow and inspectable.
A tool surface might expose actions such as:
type BrowserAction =
| { type: "navigate"; url: string }
| { type: "click"; target: string }
| { type: "type"; target: string; value: string }
| { type: "select"; target: string; value: string }
| { type: "read"; target: string }
| { type: "screenshot" };
The model chooses from known actions rather than receiving unrestricted control over the environment.
That gives the workflow a clearer audit trail and makes failed runs easier to understand.
5. Put human confirmation before irreversible actions
This boundary matters more than how clever the browsing agent is.
Imagine the agent successfully finds a customer record, opens the correct form, enters the new address, and reaches the final Submit button.
At that moment, there is a large difference between:
Agent clicks Submit
and:
Agent prepares the change
↓
Operator sees the current screen
↓
Operator confirms
↓
Agent submits
AWS's August 13 reference implementation follows the second pattern. When the model reaches a critical step, such as submitting a form or confirming a record selection, it can pause and present the operator with the current screenshot before continuing.
That gives automation speed on the repetitive steps while keeping human judgment at the boundary where the consequence becomes permanent.
A simple application-level guard could look like:
async function executeAction(
action: BrowserAction,
context: TaskContext
) {
if (context.criticalActions.includes(action.type)) {
const approved = await requestHumanApproval({
action,
screenshot: await browser.captureScreenshot()
});
if (!approved) {
return {
status: "paused",
reason: "Human approval declined"
};
}
}
return browser.execute(action);
}
The exact mechanism will vary. The architectural boundary should not.
6. Record enough evidence to reconstruct the run
Browser automation becomes difficult to trust when the only thing you know is:
The agent says it completed the task.
A production workflow should leave enough evidence to explain what happened.
That may include:
- task ID
- session ID
- page visited
- actions executed
- screenshots at important checkpoints
- human approvals
- final status
- failure reason
A simple event could look like:
type BrowserAuditEvent = {
taskId: string;
sessionId: string;
timestamp: string;
action: string;
pageUrl: string;
screenshotRef?: string;
humanApproved?: boolean;
};
AWS's reference implementation stores session transcripts and screenshots in Amazon S3 and uses CloudWatch for audit logging and observability. AgentCore Browser also supports session recording and replay.
This becomes valuable the first time a user asks:
Why did the automation change this record?
The system should have an answer better than “the model decided to.”
7. Treat authentication as workflow infrastructure
Legacy systems are often difficult to automate precisely because authentication is awkward.
There may be:
- SSO
- MFA
- internal network restrictions
- IP allowlists
- corporate proxy requirements
- long-lived browser sessions
Do not solve those problems by putting usernames and passwords into prompts.
Authentication should be handled by infrastructure around the agent.
AWS AgentCore Browser supports browser profiles that can preserve authentication state across sessions, and AWS's reference architecture also discusses corporate proxy configuration and secret handling for internal applications.
The general pattern is:
Human or identity system authenticates
↓
Browser receives scoped session
↓
Agent uses authenticated browser
↓
Credentials remain outside model context
The agent needs access to the logged-in application. It does not need to know the underlying credential.
8. Design for UI change
An API normally gives you a relatively explicit contract.
A web interface does not.
Buttons move. Labels change. A modal appears. A field is renamed. A redesign shifts the page structure.
That makes browser automation inherently more exposed to presentation changes.
A useful implementation should therefore detect uncertainty rather than pretending every page looks exactly as expected.
For example:
type PageCheck = {
expectedState: string;
observedState: string;
confidence: number;
};
If the confidence drops below your safe threshold, stop.
if (pageCheck.confidence < 0.75) {
return requestHumanReview(pageCheck);
}
The objective is not to make the model guess harder. The objective is to make uncertainty visible before the agent changes something important.
9. Keep business rules outside the browser whenever possible
The browser may be the only way to interact with a legacy system.
That does not mean every business decision belongs inside the browser agent.
Suppose a policy change is only allowed when:
- the account is active
- the user has sufficient permissions
- the effective date is valid
- the change does not exceed a threshold
If those rules are known to your own application, evaluate them outside the browser.
const validation = validatePolicyChange(request);
if (!validation.allowed) {
return {
status: "rejected",
reasons: validation.reasons
};
}
return runBrowserTask(request);
Let the browser agent handle the interface.
Let deterministic application logic handle deterministic business rules.
This reduces the number of consequential decisions the model is expected to make.
10. Make retries idempotent where possible
Browser workflows fail.
A page times out. A session drops. The agent loses confidence. A network request does not return.
A retry should not blindly restart the entire job if earlier actions may already have succeeded.
Track state explicitly:
type BrowserTaskState = {
taskId: string;
currentStep: number;
completedSteps: string[];
submitted: boolean;
};
Before repeating a step, confirm whether its intended effect already happened.
This is especially important when the browser flow includes actions such as submitting records, sending messages, changing account state, or triggering downstream processes.
11. Use browser agents where they actually fit
Browser automation is not automatically the right solution simply because an application lacks an API.
It is a strong candidate when:
- the workflow is repetitive
- the interface is reasonably stable
- a person currently performs the same steps manually
- the task can be checked visually
- critical actions can pause for human review
- rebuilding the system immediately is unrealistic
It becomes less attractive when:
- the UI changes constantly
- important business rules are invisible
- the workflow involves highly consequential actions with weak review boundaries
- throughput requirements are extreme
- a stable supported API already exists
If an API is available and appropriate, use it.
Browser agents are most valuable when the browser itself is the integration boundary you actually have.
12. Think of it as a modernization bridge
A legacy application does not need to become modern internally before every surrounding process can improve.
Browser automation can sometimes remove repetitive work while the underlying system stays in place.
That can buy the team time.
It can reduce manual handling now while a longer-term API, migration, or replacement strategy develops separately.
But that distinction matters.
A browser agent should not become an excuse to keep every fragile legacy system forever. It can be a bridge between the workflow the business has today and the architecture it eventually wants.
A practical reference architecture
A production pattern can look like:
Operator request
↓
Authentication
↓
Task validation
↓
Isolated browser session
↓
Observe page
↓
Model selects browser action
↓
Action executed through Playwright
↓
Critical action?
↙ ↘
No Yes
↓ ↓
Continue Human confirmation
↓
Continue
↓
Result recorded
↓
Session audit stored
That architecture gives the agent flexibility where the legacy interface is messy while keeping the consequential boundaries visible.
AWS's August 13 reference implementation
AWS published its legacy web automation reference architecture on August 13, 2026.
The implementation combines Amazon Bedrock AgentCore Browser Tool with Strands Agents and a vision-capable foundation model. The agent drives a managed Chromium session through Playwright/CDP, stores screenshots and session context for review, and can pause for human confirmation before critical actions.
AWS positions the architecture for legacy web applications that depend on browser interaction rather than modern APIs.
The exact stack is AWS-specific.
The engineering pattern is not.
Final thought
A missing API used to make many automation ideas feel blocked before they started.
Browser agents change that constraint.
They let software interact with the same interface a person already uses, while modern application controls can sit around that interaction: isolation, auditability, approval, authentication, and failure handling.
That does not make legacy software modern.
It can make the workflow around it much less manual.
Source
AWS
Automate legacy web applications with Amazon Bedrock AgentCore Browser Tool
Top comments (0)