DEV Community: infracore

Audit an agent skill like a pull request before it gets file access

infracore — Tue, 22 Sep 2026 12:11:26 +0000

Installing a skill or MCP server hands someone else's instructions your files, credentials, and shell. Treat it like merging unreviewed code, not adding docs.

A credible DIY baseline covers most one-off installs if you install rarely. Clone or unpack it outside your worktree, read the manifest and tool definitions first, then grep for shell execution, file writes, reads of env files and SSH keys, and outbound network calls. Run the first trial in a throwaway container or separate user with no live secrets.

That leaves recurring gaps:

static hits without context: a flag tells you curl exists, not whether it exfiltrates data or fetches a schema
install-time vs runtime drift: what you read today is not what runs after an auto-update
review fatigue: nobody re-reads every dependency on every update

A practical method that helps: a pre-run check bot for the boring second pass. Point it at the unpacked skill directory, open the code around each risky call, classify read-only versus write versus network behavior, and output a short verdict with file paths and line numbers. Keep that verdict next to the install so the next update gets a diff, not a fresh full review.

What workaround actually sticks for you: blocking installs by default, sandboxing every new skill, or re-auditing on each version bump?

Turn Screen Recordings Into Bug Reports Agents Can Act On

infracore — Mon, 21 Sep 2026 12:56:12 +0000

A 30-second clip of a flickering dropdown tells an agent almost nothing. Without steps, expected behavior, and environment, the model fills gaps with plausible but wrong fixes.

A solid manual baseline works for most teams. Rewatch the recording once and fill a fixed template:

repro steps with timestamps from the clip
expected vs actual behavior in one sentence each
browser, viewport, test account state, and feature flags
console errors and failing request URLs

Paste that Markdown with the video link into the agent prompt. That alone removes most back-and-forth about what broke and where.

Keep clips under a minute and trim to one bug per file. If the repro needs login or seeded data, note the setup commands so the agent can reproduce locally.

The remaining gap is assembly cost. Copying timestamps, selectors, and logs by hand is slow and easy to skip under pressure. Capture-to-Markdown tooling can export the same structure automatically after one recording instead of typing it.

What do you include in UI bug reports for agents today, and what part do you still skip because it takes too long to capture?

MCP Is an Adapter Layer, So Version the API First

infracore — Sat, 19 Sep 2026 22:11:01 +0000

If an MCP server is usually a thin layer over an API, the practical takeaway is simple: treat the API contract as the thing that can actually break you.

A useful method is to review changes in this order:

API surface: endpoints, required params, response fields, enums, auth shape
MCP mapping: which API fields become tool inputs/outputs
Agent behavior: prompts, tool choice, retry logic, and error handling

That order matters because MCP can make an integration look stable while the underlying API has already shifted. A renamed field, a newly required parameter, or an enum value removal may not show up as an obvious MCP problem at first. It often lands later as vague tool failure, bad completions, or agents taking the wrong branch.

The solid DIY baseline is enough for many teams: keep an OpenAPI spec, diff it in CI, and manually classify changes as breaking or non-breaking before updating the MCP wrapper. If your API is small and the tool surface is narrow, that can be perfectly sufficient.

The remaining gap is consistency when lots of small schema edits pile up. The MCP layer may stay thin, but the review burden does not. Safest habit: version and gate the API first, then regenerate or update the MCP adapter second.

What's the most annoying break you've seen in practice: required fields changing, enum drift, auth changes, or something else at the MCP-to-API boundary?

Expose Crypto KAT Runners as MCP Tools Instead of Pasting Hex

infracore — Fri, 18 Sep 2026 21:27:45 +0000

When a Known Answer Test fails on a crypto primitive, the cause is often byte-ordering or padding. Pasting that raw hex plus the ACVP JSON into a chat window works once, but it does not scale to large vector files.

A credible baseline stays manual: run the KAT binary locally, copy only the failing vector ID and byte offset into the prompt, fix, and re-run. For a single algorithm and a small file, that loop is sufficient and easier to audit than adding server code.

The gap appears with massive ACVP payloads and repeated re-runs. One method is to put the runner behind Model Context Protocol over stdio:

tool run_crypto_kat(target, algorithm) returning structured status, failing case ID, and diff window
tool load_acvp_vectors(path) with filtering or paging instead of dumping the whole file
resources for expected schema layout and tail of the latest log

Keep tool outputs small and typed: verdict, file, case, expected vs actual slice, and next command. Let the agent request more context explicitly rather than pushing the whole file by default.

What workaround do you use today to keep large KAT vectors out of the prompt while still giving the agent enough to fix the byte-level bug?

A pre-flight gate for vague MCP store rejections

infracore — Thu, 17 Sep 2026 22:36:17 +0000

The failure class to design for is this: your MCP server works against local clients, then stalls in a plugin store review with feedback too vague to act on.

Freeze what you submitted. Pin the manifest, tool list with JSON Schemas, auth scopes, and example transcripts to one commit. Then run the same pre-flight before every resubmit:

validate all tool input/output schemas and examples with strict validators
diff tools and required fields against your last submitted commit
check scope minimization, stable names/descriptions, and consistent error shapes

A credible DIY baseline is plain CI: Ajv or Zod plus a JSON diff and saved fixtures. For infrequent releases and a small tool surface, that catches schema drift and accidental breaking renames without new tooling.

The remaining gap is reviewability. A green CI run does not show a reviewer exactly what changed and what you checked. Keep the commit pair and check logs together so a rejection can be answered with specifics instead of guesses. That bundle matters most when feedback is one vague line.

What pre-submit check has actually caught a store rejection for you before you hit submit?

Check renewable sessions before forcing a browser re-login

infracore — Wed, 16 Sep 2026 16:46:24 +0000

Provider OAuth sessions can fail in a confusing way: a short-lived access token expires while a usable refresh token is still stored. A background quota probe then gets a 401, and the UI makes a renewable session look like it needs a new browser login.

Fix it with a manual, explicitly quota-consuming session check. On operator click only - never from status polling - run one isolated request through the provider's supported path, using a fixed minimal prompt with no tools, no MCP servers, safe mode, and no conversation persistence. Hold the same exclusive process slot as tasks and login, then compare safe credential metadata before and after without logging or returning token material.

Report a normalized outcome so operators stop guessing:

session renewed
session already current
session valid but quota exhausted
re-authentication required
check failed

The DIY baseline is enough for most teams: a single serialized button, clear disclosure that it may consume quota, and a separate Re-authenticate action. It breaks down when adapters guess across vendors, or when quota exhaustion is mislabeled as auth failure and clears the wrong backoff.

Keep the control-plane operation vendor-neutral, advertise quota cost, and make unsupported adapters fail loudly. Only a rotated credential should clear a stale unauthenticated backoff and allow one immediate quota retry.

How do you keep quota-exhausted separate from truly expired in your session UI today?

Don't Trust Green Karate Runs: Diff Them Against OpenAPI

infracore — Tue, 15 Sep 2026 16:39:31 +0000

A passing Karate suite only proves the scenarios you wrote still pass. It says nothing about OpenAPI operations that were never called. That gap grows when specs add endpoints, deprecate fields, or rename routes faster than feature files are updated.

A direct check is to diff exercised calls against the spec instead of trusting the green run. A small script is often enough for small services.

List every path + method pair from your OpenAPI document as the expected set.
Collect the actual set by parsing Karate feature files for url, path, and method, or by logging requests during a run.
Join the two sets to find untested operations, then prioritize by auth scope and breaking-change risk.

Generate stubs only for the missing operations, with one happy path and one auth or validation failure each. Keep the coverage diff in version control so the next spec change shows which operations lost coverage.

For larger specs, the harder part is keeping that mapping stable across refactors, parameterized paths, and versioned routes without hand-maintaining aliases.

How do you currently detect OpenAPI endpoints your Karate suite never exercises?

Keeping One MCP Server Working Across SDK Renames and Three CLIs

infracore — Mon, 14 Sep 2026 23:47:59 +0000

An MCP server breaks at three seams: the SDK class name, the upstream request shape, and each CLI's server config. Treating those as one upgrade is what makes a small rename expensive.

Put a thin adapter between your tools and the vendor SDK. Keep all imports and request construction in one module, so a FastMCP to MCPServer or genai client bump touches one file. Add a startup check that lists tools, sends one tiny generation call, and prints the resolved model, endpoint, and SDK version on failure.

DIY baseline that covers the rename case: pin exact versions in the lockfile, run that smoke script in CI, and keep one config snippet per CLI pointing at the same entrypoint. For a single-maintainer server, that is usually enough.

The remaining gap is silent divergence: one CLI passes env or cwd differently, or the Interactions API returns 400 for a field shape the old SDK sent. Log the outbound payload bytes on non-2xx and diff them against the last known-good call before changing code.

What workaround have you used when the same MCP server passes in one CLI but fails in another?

Wrapping foreign MCP and skill metadata without silent authority

infracore — Sun, 13 Sep 2026 15:40:22 +0000

When you pull an existing MCP server or an OpenAI/Claude-style skill into a local agent package model, the hard part is not the install. It is turning foreign tool metadata into something reviewable: provenance and version kept intact, network/filesystem/secret needs mapped to explicit permissions, and hooks left off until someone approves them.

A credible baseline is still manual. Read the server or skill manifest, list every tool action, note undeclared filesystem or network reach, and reject or mark unsupported anything you cannot map. Auto-enable is how permanent owner-memory write and silent background behavior sneak in.

Running imports through the same validator and permission model as native packages only helps if unmappable authority fails closed and compatibility status stays visible (native, wrapped, partial, rejected).

How do you currently record provenance and permission gaps when wrapping a third-party MCP server so a later review can tell those four outcomes apart?

Stale entries in agent discovery manifests

infracore — Sat, 12 Sep 2026 16:07:24 +0000

Publishing /.well-known/ai-catalog.json is one way to list MCP servers, A2A agents, and OpenAPI schemas for agent discovery. The shape is simple: specVersion, host, and an entries array.

The failure mode is quieter than a missing file. A catalog can stay up while an entry still points at a moved MCP URL, a renamed skill, or an OpenAPI document that no longer matches the live API.

DIY checks that catch this without new tooling:

curl each entry URL in CI and require a non-error response
diff listed names against what the live MCP or schema endpoint actually returns
fail the job when the catalog and runtime disagree

When you change an MCP mount path or ship a breaking schema edit, what process updates the ARD manifest, and what check do you run so agents do not keep discovering the stale entry?

Hard caps on autonomous agent API spend

infracore — Fri, 11 Sep 2026 23:05:10 +0000

Autonomous agents in long research loops can exhaust an API budget before anyone notices. Provider dashboards and account-level alerts help after the fact, but they often fire too late for a single runaway session.

A practical DIY baseline is a thin client wrapper that:

tracks cumulative token or dollar cost per session
refuses the next call once a hard session ceiling is hit
logs the stop reason so the operator can inspect what burned the budget

Provider key quotas and org spend limits still matter as a backstop when the wrapper fails open.

When you run agents against paid model APIs, where do you enforce the hard stop: in the agent loop, at an API gateway, or only via the provider's billing controls?

Hunting Down a 25-Second Network Ghost

infracore — Thu, 10 Sep 2026 13:34:47 +0000

Some HTTP MCP clients log a basic connectivity pre-check, finish TCP/TLS in well under a second, then sit idle for a tight ~25s band before logging success and sending the real request.

When a capture shows the handshake fully ACKed and zero application bytes until the client closes, the stall is almost certainly in client logic: a fixed timer, a probe that never writes, or negotiation that only continues after a timeout.

Useful isolation without owning the server:

Wall-clock from the "testing connectivity" log to the first client-sent application byte
Whether an HTTP/2 preface, SETTINGS, or any request leaves during the gap
Local config beside the server URL for protocol or feature-flag state that could gate the next step

Which client pre-check designs have you seen that wait a fixed multi-second delay after the socket is already live?