Web Agents for Insurance Operations: Prior Auth, Portal Monitoring, and Regulatory Tracking

---

If you work in insurance operations, you already know the portal problem. Every payer has one. Each one is different. Some require two-factor login to your own account. Some time out after 15 minutes of inactivity. Some show prior auth status on page three of a workflow that starts on a dashboard that loads differently depending on which browser you're using.

Multiply that across 50 payers, and you have a problem that doesn't scale with headcount. Hiring more people to click through more portals isn't a solution — it's a deferral. The portals don't get simpler. The volume doesn't go down.

Web agents don't replace the portals. They handle the repetitive navigation tasks that humans currently perform manually — logging in, extracting status, and returning structured data that your systems can actually use. This article covers three use cases where that matters most: prior authorization tracking, multi-pharmacy price monitoring, and real-time regulatory document monitoring.

---

Scope note: This article covers insurance operations automation patterns using web agents. The use cases below represent technically valid workflows validated through internal testing, not published case studies from specific TinyFish insurance customers.

When does a web agent apply to insurance operations?

Before the use cases: if you're evaluating whether this fits your team's situation, here's the honest framework.

1. You're checking the same portals repeatedly — prior auth status, claim status, eligibility — on a schedule your staff maintains manually

2. Portal interfaces vary across payers — no two are identical, making traditional RPA brittle

3. You need structured output, not screenshots — downstream systems need data, not PDFs

4. Volume exceeds what staff can cover — 50+ portals, daily or more frequent checks

5. Login walls to your authorized payer portals are blocking automation — most traditional scrapers stop at the login screen. Web agents using your authorized credentials navigate login flows as a normal part of the workflow

If your team checks fewer than 10 portals and the data isn't time-sensitive, manual workflows may be cheaper. The case for automation compounds as portal count and check frequency increase.

---

The portal problem: why insurance operations are stuck in 2005

📸 IMAGE — Architecture diagram showing TinyFish web agents checking 50 payer portals in parallel returning structured JSON

Fifty payer portals sounds like a technology problem. It's actually a coordination problem that technology made worse.

Each payer built their portal independently, on their own timeline, for their own internal reasons. The result is a landscape where every portal has a different login flow, a different session timeout, a different way of presenting the same data. Prior auth status might be on the dashboard for one payer, buried in a sub-menu for another, and only visible after clicking through a three-step workflow for a third.

Traditional RPA — the previous generation's answer to this — was built around CSS selectors and fixed screen coordinates. When a payer updates their portal layout, the RPA script breaks. Someone has to find it, diagnose it, and fix it before the monitoring gap compounds. In a 50-portal environment, that's a near-continuous maintenance cycle.

Web agents approach portals through standard browser sessions: reading the page, understanding what's on it, and navigating based on intent rather than coordinates. When a portal layout changes, the goal — "find the prior auth status for claim ID X" — doesn't.

Benchmark: In TinyFish's internal testing, 50 payer portals were checked in 2 minutes and 14 seconds. A staff member doing the same task manually takes 45 minutes or more. At daily frequency, that's roughly 180 staff-hours per month on portal checks alone.

---

Use case 1: Prior authorization status tracking across 50 portals

Prior auth is time-sensitive in both directions. A pending auth that gets approved needs to trigger the next step in the care workflow. A denial needs to trigger the appeals process. Both require someone — or something — checking portal status on a regular cadence.

The code

import asyncio
import json
from datetime import datetime, timezone
from tinyfish import AsyncTinyFish, BrowserProfile

client = AsyncTinyFish()

PRIOR_AUTH_CHECKS = [
    {
        "payer": "Aetna",
        "portal_url": "https://payer-portal-a.example.com/...",
        "auth_id": "PA-2026-00441",
    },
    {
        "payer": "UnitedHealthcare",
        "portal_url": "https://payer-portal-b.example.com/...",
        "auth_id": "PA-2026-00892",
    },
    # scale to 50+ payers
]

async def check_auth_status(payer: str, portal_url: str, auth_id: str) -> dict:
    goal = f"""
    Find the prior authorization status for auth ID: {auth_id}.

    Log in if prompted using your authorized portal credentials. Navigate to the prior authorization or claims section.
    Locate the record matching auth ID {auth_id}.

    Return JSON with this exact structure:
    {{
        "auth_id": "{auth_id}",
        "status": "approved" or "pending" or "denied" or "not_found",
        "effective_date": "YYYY-MM-DD or null",
        "expiry_date": "YYYY-MM-DD or null",
        "denial_reason": "string or null"
    }}

    If you cannot find the auth ID, return status as "not_found".
    Do not click any Submit or transaction buttons.  # prevents accidental form submissions or transactions during portal navigation
    """

    response = await client.agent.run(
        url=portal_url,
        goal=goal,
        browser_profile=BrowserProfile.MANAGED,
    )

    # For debugging: response.streaming_url contains a live browser replay (valid 24h)
    # Layer 1: infrastructure failure — result is None
    if response.result is None:
        return {
            "payer": payer, "auth_id": auth_id,
            "status": "error", "error": "infrastructure_failure",
            "checked_at": datetime.now(timezone.utc).isoformat(),
        }
    # Layer 2: goal failure — agent ran but task didn't succeed

# (continued)
    result = response.result

    if result.get("status") == "failure":
        return {
            "payer": payer,
            "auth_id": auth_id,
            "status": "error",
            "error": result.get("reason", "goal_failed"),
            "checked_at": datetime.now(timezone.utc).isoformat(),
        }

    return {
        "payer": payer,
        "auth_id": auth_id,
        "status": result.get("status"),
        "effective_date": result.get("effective_date"),
        "expiry_date": result.get("expiry_date"),
        "denial_reason": result.get("denial_reason"),
        "checked_at": datetime.now(timezone.utc).isoformat(),
    }

async def main():
    tasks = [
        check_auth_status(c["payer"], c["portal_url"], c["auth_id"])
        for c in PRIOR_AUTH_CHECKS
    ]
    results = await asyncio.gather(*tasks)
    print(json.dumps(results, indent=2))

asyncio.run(main())

Note on result handling: status: "COMPLETED" means the browser ran — not that the auth ID was found. A portal that returned a "not found" page will also return COMPLETED, but result.status will be "not_found" as instructed in the goal. Always check result, not just the run status.

Note on login walls: This example assumes portal URLs for portals you are authorized to access that accept authenticated sessions or have login steps navigable by the agent. For portals with 2FA or SSO flows, additional goal steps are needed.

Output schema

[
  {
    "payer": "Aetna",
    "auth_id": "PA-2026-00441",
    "status": "approved",
    "effective_date": "2026-03-01",
    "expiry_date": "2026-08-31",
    "denial_reason": null,
    "checked_at": "2026-03-27T09:00:01Z"
  },
  {
    "payer": "UnitedHealthcare",
    "auth_id": "PA-2026-00892",
    "status": "pending",
    "effective_date": null,
    "expiry_date": null,
    "denial_reason": null,
    "checked_at": "2026-03-27T09:00:04Z"
  }
]

This output feeds directly into your care coordination workflow. Approved auths trigger the next step. Denials trigger the appeals queue. Pending items re-enter the next day's check cycle. The agent produces the data; your existing systems handle the routing.

---

Use case 2: Drug price monitoring across state transparency portals

Most states now publish drug pricing data through public transparency portals — mandated by law, updated regularly, and freely accessible without a contract relationship. For pharmacy directors, formulary analysts, and benefit consultants, these portals are a legitimate and underused data source. The problem is coverage: 40+ states with different portal designs, different update schedules, and no unified feed.

Monitoring them manually is the same problem as payer portal monitoring — the data exists, accessing it at scale requires either staff time or automation.

import asyncio
import json
from datetime import datetime, timezone
from tinyfish import AsyncTinyFish, BrowserProfile

client = AsyncTinyFish()

# State drug price transparency portals — publicly accessible, no contract required
STATE_DRUG_PORTALS = [
    {
        "state": "CA",
        "url": "https://www.dhcs.ca.gov/provgovpart/pharmacy/Pages/drug-pricing.aspx",
        "drug_name": "metformin 500mg",
    },
    {
        "state": "NY",
        "url": "https://www.health.ny.gov/health_care/medicaid/program/pharmacy/",
        "drug_name": "metformin 500mg",
    },
    {
        "state": "TX",
        "url": "https://hhs.texas.gov/providers/medicaid-prescription-drug-program",
        "drug_name": "metformin 500mg",
    },
    # add remaining states
]

async def check_drug_price(state: str, url: str, drug_name: str) -> dict:
    goal = f"""
    Find the current published price or reimbursement rate for: {drug_name}.

    Navigate to the drug pricing or formulary section.
    Look for a price table, search tool, or downloadable listing.

    Return JSON:
    {{
        "drug_name": "{drug_name}",
        "published_price": number or null,
        "currency": "USD",
        "price_type": "reimbursement rate / retail price / AWP / MAC — whichever applies",
        "effective_date": "YYYY-MM-DD or null",
        "source_label": "label or table header as shown on page"
    }}

    If no price is found, set published_price to null.
    Do not submit any forms or create any accounts.
    """

    response = await client.agent.run(
        url=url,
        goal=goal,
        browser_profile=BrowserProfile.LITE,  # state gov sites typically don't need stealth
    )

    # For debugging: response.streaming_url has a live browser replay (valid 24h)
    result = response.result or {}

    if result.get("status") == "failure":
        return {
            "state": state,

# (continued)
            "drug_name": drug_name,
            "error": result.get("reason", "goal_failed"),
            "checked_at": datetime.now(timezone.utc).isoformat(),
        }

    return {
        "state": state,
        "drug_name": drug_name,
        "published_price": result.get("published_price"),
        "currency": result.get("currency", "USD"),
        "price_type": result.get("price_type"),
        "effective_date": result.get("effective_date"),
        "source_label": result.get("source_label"),
        "checked_at": datetime.now(timezone.utc).isoformat(),
    }

async def main():
    tasks = [check_drug_price(p["state"], p["url"], p["drug_name"]) for p in STATE_DRUG_PORTALS]
    results = await asyncio.gather(*tasks)
    print(json.dumps(results, indent=2))

asyncio.run(main())

State transparency portals publish reimbursement rates, MAC pricing, and AWP data — the reference prices that feed formulary decisions and benefit design. Aggregating this across 40+ states concurrently takes minutes instead of days, and the output is structured data rather than a stack of PDFs to parse manually.

---

Use case 3: Real-time regulatory document monitoring

State insurance filings change. Rate approvals, form filings, regulatory bulletins — the information is public, posted on state insurance department websites, and directly relevant to compliance and underwriting decisions. The problem is that 50 states publish on different schedules, in different formats, with no unified feed.

Manual monitoring at this scale means either missing changes or dedicating staff to daily checks across 50 state websites. Neither is sustainable.

The agent approach: run a daily scan across state insurance department filing pages, extract new documents since the last check, and return structured metadata.

import asyncio
import json
from datetime import datetime, timezone
from tinyfish import AsyncTinyFish, BrowserProfile

client = AsyncTinyFish()

STATE_FILING_PAGES = [
    {"state": "CA", "url": "https://www.insurance.ca.gov/0400-news/0200-bulletins/"},
    {"state": "NY", "url": "https://www.dfs.ny.gov/industry_guidance/circular_letters"},
    {"state": "TX", "url": "https://www.tdi.texas.gov/rule/"},
    # add all 50 states
]

LAST_CHECK_DATE = "2026-03-26"  # replace with your stored value

async def check_state_filings(state: str, url: str) -> dict:
    goal = f"""
    Extract all regulatory documents or bulletins published after {LAST_CHECK_DATE}.

    For each document found, return:
    {{
        "title": "document title",
        "published_date": "YYYY-MM-DD",
        "document_url": "full URL",
        "category": "bulletin / circular / rate filing / form filing / other"
    }}

    Return as JSON array. If no new documents found, return an empty array [].
    Do not click any login or subscription prompts.
    """

    response = await client.agent.run(
        url=url,
        goal=goal,
        browser_profile=BrowserProfile.LITE,  # state gov sites typically don't need stealth
    )

    result = response.result or {}

    # Check for goal-level failure before checking result type
    # Without this, a failure dict would pass the isinstance check as non-list and silently return []
    if result.get("status") == "failure":
        return {
            "state": state,
            "new_documents": [],
            "error": result.get("reason", "goal_failed"),
            "checked_at": datetime.now(timezone.utc).isoformat(),
        }

    return {
        "state": state,
        "new_documents": result if isinstance(result, list) else [],
        "checked_at": datetime.now(timezone.utc).isoformat(),
    }

async def main():

# (continued)
    tasks = [check_state_filings(s["state"], s["url"]) for s in STATE_FILING_PAGES]
    results = await asyncio.gather(*tasks)

    # Filter to states with new documents
    updates = [r for r in results if r["new_documents"]]
    print(json.dumps(updates, indent=2))

asyncio.run(main())

The output feeds a compliance dashboard, triggers alerts for specific document categories, or routes to the relevant underwriting or legal team. The agent doesn't interpret the documents — it surfaces them. Interpretation stays with your team.

---

The compliance question: what web agents can and can't do in regulated environments

This is the question that comes up in every insurance technology conversation, so let's address it directly.

What web agents do: navigate browser sessions, extract data from pages, return structured output. They interact with portals through standard browser sessions — navigating to a page, reading what's on it, and returning the relevant data.

What web agents don't do: store protected health information, transmit data across systems, or make clinical or coverage decisions. The agent is a data extraction layer. Where that data goes next — and what compliance obligations apply to that storage and transmission — is determined by your system architecture, not the agent.

On HIPAA: A web agent accessing a payer portal doesn't inherently create new HIPAA obligations beyond those that already exist when a staff member accesses the same portal through the same browser. The agent performs authenticated browser access — reading page content through an authenticated session and returning structured data. The compliance obligations arise from what you do with the extracted data: how it's stored, who can access it, and how it's transmitted. That's your data pipeline's responsibility, not the extraction layer's.

This doesn't mean HIPAA review isn't necessary — it means the review should focus on the full data flow, not just the agent step. Any healthcare or insurance deployment should involve a legal review of the complete architecture. The agent layer is usually not where the compliance complexity lives.

---

Build vs. buy: when to use TinyFish vs. building your own portal integration

The alternative to a web agent for payer portal access is a direct API integration — but most payer portals don't have APIs. The ones that do (typically large national payers via HL7 FHIR) are worth pursuing first. Where APIs exist, use them.

Web agents become the right tool when:

• The portal has no API and no roadmap to add one

• The portal interface is stable enough to automate but too variable for traditional RPA

• The volume of checks justifies automation over manual workflows

• Your team can't maintain custom automation scripts as portals evolve

The build-vs-buy question for the agent infrastructure itself: building a raw HTTP automation layer that handles login walls, session management, 2FA, and concurrent execution is months of engineering work. The free tier — 500 credits, no credit card — is enough to run a proof of concept against your real target portals before that decision needs to be made.

---

Get started

The free tier gives you 500 credits — enough to run a prior auth status check across 50 portals and see real output before committing to anything.

For healthcare and insurance teams with compliance requirements or enterprise-scale portal monitoring needs, contact our enterprise team.

---

FAQ

Can web agents handle your own payer portals that require 2FA or SSO login?

Agents can navigate multi-step authentication workflows, including TOTP-based MFA and standard SSO redirects, for portals you're authorized to access. Include the authentication steps for your authorized portal in the goal prompt, or use TinyFish's credential vault for secure credential management. SSO with federated identity is more complex and depends on the specific implementation. Include explicit authentication steps for your authorized portal in your goal prompt and use the credential vault for secure credential storage.

Does this work for all 50 payer portals or just the major ones?

Any publicly accessible portal — including smaller regional payers — is reachable. Success rate depends on the portal's complexity and technical protection level, not its size. The agent navigates based on what it sees on screen, so portal size is not a limiting factor.

What happens when a payer portal updates its layout?

The goal-based approach is more resilient to layout changes than selector-based RPA. When a portal redesigns, the goal — "find prior auth status for ID X" — remains valid. In practice, goal prompts may need minor tuning after major redesigns, but the maintenance burden is substantially lower than maintaining CSS selectors.

Is the extracted data HIPAA-compliant?

HIPAA obligations arise from how extracted data is stored, accessed, and transmitted — not from the extraction mechanism itself. A web agent accessing a payer portal creates the same data handling obligations as a staff member accessing the same portal. The compliance review should cover your full data pipeline, not just the agent step. Any healthcare deployment should involve legal review of the complete architecture before going to production.

How do I handle portals that time out mid-session?

Include session management instructions in the goal: "if a timeout or re-login prompt appears on your authorized portal, log in again with the provided credentials and resume." The agent follows standard re-authentication flows natively.

---

See It in Action

The free tier includes 500 steps — enough to run a complete insurance data extraction workflow against real data before committing to a plan.

Start free, no credit card →

Related Reading

• Pillar: What Can AI Web Agents Actually Do? 10 Real-World Use Cases

• How to Monitor 1,000 Websites in Parallel with the TinyFish API

• When Web Agents Fail: Debugging Goal-Based Automation

• Web Agents for Procurement: Multi-Vendor Portal Automation

---

Want to scrape the web without getting blocked? Try TinyFish — a browser API built for AI agents and developers.