DEV Community

Thirdwatch for Apify

Posted on AI-assisted

I let an Apify Actor read my docs and write one GitHub issue. It found a 404 without cloning the repo.

A broken-link checker usually starts from a deployed website. Mine needed to start one step earlier, in Markdown that lives in GitHub.

That distinction mattered. Some of the files were not deployed yet. I wanted exact path:line evidence, not the URL of a rendered page. I also wanted a scheduled run to leave the result where maintainers already work, without cloning the repository into the Actor or putting a GitHub token in Actor input.

So I built a GitHub Documentation Link Auditor around an Apify MCP connector. The Actor reads explicit .md or .mdx paths, checks at most 100 public HTTP links, and creates or updates one marker-owned GitHub issue. It cannot edit a file, create a branch, or open a pull request.

On August 9, 2026, I ran the published build twice against a public fixture pinned to one commit. Both runs checked four links: two were reachable, one returned 404, and one loopback URL was rejected before a request. The first run created issue #5. The second updated issue #5.

The update is as important as the 404. A daily checker that opens a daily issue is just another kind of broken workflow.

Where the connector fires

MCP connectors let an Actor call a third-party service during its run. That is the opposite direction from the Apify MCP server, which lets an external AI client call Actors as tools.

This workflow uses the connector at two narrow boundaries:

The first call replaces a checkout or manual upload. The last two calls replace copying a report back into GitHub and checking whether yesterday's issue already exists. Link extraction, DNS policy, HTTP checks, classification, and report rendering happen inside the Actor.

I use two connector inputs because dry runs should not require write authority. A user may select the same GitHub connector for both fields, but the Actor's contract does not demand issue tools until writing is enabled.

Put the permission boundary in the input schema

An MCP connector is declared with resourceType: "mcpConnector". The read input exposes one GitHub tool:

{
  "githubReadConnector": {
    "title": "GitHub read connector",
    "type": "string",
    "resourceType": "mcpConnector",
    "mcpServers": [
      {
        "url": "https://api.githubcopilot.com/mcp*",
        "tools": { "required": ["get_file_contents"] }
      }
    ]
  }
}
Enter fullscreen mode Exit fullscreen mode

The optional write input declares only issue search and issue write:

{
  "githubWriteConnector": {
    "title": "GitHub issue connector",
    "type": "string",
    "resourceType": "mcpConnector",
    "mcpServers": [
      {
        "url": "https://api.githubcopilot.com/mcp*",
        "tools": {
          "required": ["search_issues", "issue_write"]
        }
      }
    ]
  }
}
Enter fullscreen mode Exit fullscreen mode

No content-write, branch, pull-request, or merge tool is available. The connector's own GitHub authorization still applies underneath this ceiling.

The Actor receives a connector ID and an Apify proxy URL, not the GitHub credential. It connects with the run token through a standard Streamable HTTP MCP client:

base_url = os.environ["ACTOR_MCP_CONNECTOR_BASE_URL"].rstrip("/")
token = os.environ["APIFY_TOKEN"]

async with httpx.AsyncClient(
    headers={"Authorization": f"Bearer {token}"},
    trust_env=False,
) as http_client:
    async with streamable_http_client(
        f"{base_url}/{connector_id}",
        http_client=http_client,
    ) as (read, write, _):
        async with ClientSession(read, write) as session:
            await session.initialize()
Enter fullscreen mode Exit fullscreen mode

Before making a call, I inspect tools/list. A read session must expose get_file_contents; a write session must expose both issue tools. Failing there is cheaper and clearer than discovering a permission problem after 100 HTTP checks.

Read exact files at an immutable ref

The input names specific Markdown paths rather than asking the Actor to wander through a repository:

{
  "owner": "poojitha-rachuri",
  "repo": "apify-mcp-connector-fixture",
  "ref": "f7e56bcddba23cea5232fbe7735b5065ecc01ed3",
  "markdownPaths": ["docs/link-audit-fixture.md"],
  "maxLinks": 20,
  "dryRun": false,
  "confirmWriteTarget": "poojitha-rachuri/apify-mcp-connector-fixture"
}
Enter fullscreen mode Exit fullscreen mode

A full 40-character commit makes the run reproducible. Branches and tags work, but the output marks them as mutable.

The connector call is small:

response = await session.call_tool(
    "get_file_contents",
    arguments={
        "owner": owner,
        "repo": repo,
        "path": path,
        "ref": ref,
    },
)
Enter fullscreen mode Exit fullscreen mode

GitHub MCP can return file data as structured content, text, or an embedded resource. My decoder checks all three and does not let an empty {} in structuredContent hide a valid resource block. That edge case came from running a different GitHub connector workflow against the real service; I reused the lesson here.

Each file is capped at 1 MB. I skip fenced code, inline code, and HTML comments, then retain every source line for each distinct URL. Extraction also has a per-file match ceiling. If that ceiling or maxLinks is reached, the run says PARTIAL; it never presents a prefix as a complete audit.

Treat repository text as input, not instructions

A Markdown file can contain URLs chosen by anyone with repository write access. Fetching them blindly would turn a link checker into an SSRF tool.

Before each request, including every redirect, the Actor rejects:

  • embedded credentials;
  • ports other than 80 and 443;
  • localhost, .local, and .internal names;
  • loopback, private, link-local, multicast, reserved, and other non-public addresses.

A preflight DNS check is not sufficient. A hostile hostname could return a public address during validation and a private address when the HTTP client resolves it again. I use an aiohttp resolver that connects only to the exact public address set already validated for that hostname. The original hostname remains available for the HTTP Host header, TLS SNI, and certificate verification.

addresses, error = await resolve_public_addresses(hostname, 5)
if error:
    return classify_resolution_error(error)

resolver.pin(hostname, addresses)
async with client.get(
    current_url,
    headers={"Range": "bytes=0-1023"},
    allow_redirects=False,
) as response:
    status = response.status
Enter fullscreen mode Exit fullscreen mode

Environment proxies are disabled. Redirects are followed manually so each new hostname passes the same validation. The Actor uses GET with a small range instead of HEAD because plenty of healthy servers implement HEAD differently from normal navigation.

The deliberate fixture URL http://127.0.0.1:8080/admin never reaches client.get. It becomes unsafe_target with the reason non-standard ports are rejected. A standard-port loopback target would be rejected by the address check.

A 404 is an observation, not a verdict

I use four result states:

State Meaning
reachable A 2xx response was observed
observed_not_found A 404 or 410 was observed and needs human confirmation
needs_review Auth, rate limiting, DNS, timeout, malformed redirect, or server failure made the result inconclusive
unsafe_target The target was rejected before a request

One 404 does not prove that a URL is permanently dead. Geo routing, a CDN rule, or bot protection can make the Actor see something a maintainer does not. The issue therefore says “HTTP 404/410 observed,” includes the timestamp and source line, and asks a human to open the source file before changing it.

A run also reports its own completeness:

  • COMPLETE when every requested file was read and every discovered link fit within the limits;
  • PARTIAL when a file, extraction, or link limit prevented a complete result;
  • FAILED when no requested file could be read.

A failed zero-file audit is pushed without the paid event. It cannot return NO_NOT_FOUND_OBSERVED or charge for a clean result it never established.

Give each audit scope its own issue identity

A fixed issue title is not enough. A README audit and a release-docs audit in the same repository should not overwrite each other.

The Actor hashes the repository, ref, and normalized path set into a 12-character scope key:

canonical = "\n".join(
    (owner.casefold(), repo.casefold(), ref or "default", *sorted(set(paths)))
)
scope_key = hashlib.sha256(canonical.encode()).hexdigest()[:12]
marker = issue_marker(scope_key)
Enter fullscreen mode Exit fullscreen mode

It searches open issues with the base title, then updates only an issue whose body contains the exact scope marker. A human-written issue with a similar title is left alone.

Create responses from issue_write do not always contain an issue number. I do not label such a write verified on faith. The Actor searches again for the exact title and marker, with a bounded retry for GitHub's search-index delay. If it still cannot read the issue back, the dataset preserves the completed link audit but reports issue_action: write_failed.

Dry run is the default. A real write also requires confirmWriteTarget to match the exact owner/repo. That confirmation is not authorization—the connector decides what the user may access—but it catches a surprising number of copy-paste mistakes.

Two failures the real run caught

The first network smoke labeled https://apify.com/ inconclusive even though the page was healthy. The response carried a Content Security Policy header larger than aiohttp's default 8 KB field limit. The client failed while parsing headers, before I saw the 200.

I raised the field allowance to a bounded 32 KB:

async with aiohttp.ClientSession(
    connector=connector,
    timeout=timeout,
    trust_env=False,
    max_field_size=32_768,
) as client:
    ...
Enter fullscreen mode Exit fullscreen mode

The second failure was subtler. The GitHub connector created the issue, but its issue_write response omitted the issue number. My strict write check called that unverified. The next run found and updated the issue, proving the side effect had happened.

That led to the marker-bound read-back described above. It also changed the output contract: link evidence is saved even when issue delivery fails, and issue_error explains the failed stage. A transient write problem should not erase a completed audit.

Both defects passed mocked happy-path tests. The fixture run earned its keep before the article had a headline.

The production evidence

The public fixture contains two normal links, one stable missing path at example.com, and one unsafe loopback target. The final build was 1.0.4.

Observation Create run Repeat run
Actor run NQdWlaw0v0yYU2Kbh nSZwPk7XhSTD29HBH
Dataset cAjWlHWSC2ZGLMfno GzfWfFJJKYS5bbM0f
Immutable ref f7e56b…01ed3 f7e56b…01ed3
Links checked 4 4
Reachable 2 2
404/410 observed 1 1
Unsafe targets rejected 1 1
Audit status COMPLETE COMPLETE
GitHub action Created issue 5 Updated issue 5
Runtime 16.1 seconds 5.5 seconds
Platform usage $0.000734 $0.000276

The create run took longer because it waited for marker-bound read-back after GitHub accepted the write. Both costs were far below one cent, with no browser or proxy.

The output contains the resolved public addresses, final URL, status, redirect count, duration, observation timestamp, and every source location. It does not include connector credentials.

What the connector replaced, and what it did not

Without the connector, I had three unattractive choices: clone the repository with a credential, ask someone to upload Markdown and copy the report back, or maintain another automation service just for GitHub glue.

The connector removed that glue. A schedule can read the same authorized source at a pinned ref, run a bounded audit, and refresh one review surface. The Actor never possesses a general-purpose GitHub token and never gains a code-mutation tool.

It did not remove judgment. A maintainer still decides whether the link is intended, whether the Actor saw the same response a reader sees, and what replacement belongs in the docs. Nor is this a site crawler: if the job is to discover every page on a deployed domain, a normal broken-link crawler is the better tool.

That boundary is the point. MCP connectors are most useful when they give an Actor the smallest missing piece of context or delivery authority. Here, three GitHub calls turned a standalone checker into a repeatable maintenance workflow without turning it into a repository administrator.


Disclosure: The Actors described in this article are built and operated by Thirdwatch.

Top comments (0)