Every fact here is verified against an actual run (Amazon Nova Pro us-east-1 on-demand, Strands Agents, and traccia 0.1.29 $0.0008 per 1K input an...
For further actions, you may consider blocking this person and/or reporting abuse
The caveat most governance writeups never write is the one you did: "integrity_hash is a plain unkeyed SHA-256... anyone who can rewrite the bundle can recompute the hash." That sentence deserves the next rung of the ladder, because it stops one step before the real fix.
The ladder runs: a hash detects accidents, a signature attributes them to a key — and neither stops the key holder. Anyone who can rewrite the bundle can also re-sign it; a regenerated history verifies against its own key with flying colors, and a seal created after the fact tells you nothing about what existed before. "When" is the property that survives an incident, and it can't come from inside the system the record describes.
The standard pieces are cheap: hand the digest — 32 bytes, nothing else — to witnesses outside your trust boundary. An RFC 3161 timestamp from a qualified TSA carries a presumption of correctness under eIDAS (the tier a regulator's lawyer accepts), and Bitcoin anchoring via OpenTimestamps gives you the tamper-proof backstop. Only an opaque digest ever leaves, which slots neatly into the PII redaction story you already tell: the GDPR conversation stays short.
Which is the one thing your evidence section doesn't answer yet: the pack proves what Traccia recorded, but "we have logs" and "we can prove these logs were never touched" are two different sentences to an authority — and Article 12's whole point is the second one. The Governance Hub is operator-owned infrastructure, so the platform that records is the platform that could rewrite. Who witnesses the witness?
Curious whether the export carries any externally checkable time today, or whether that's the missing beat.
this is the best technical comment on the article and it goes exactly one rung past where i stopped, which is where it matters.
short answer to your question: no. the export does not carry any externally checkable time today. the bundle has a generated_at and an unkeyed sha-256, both produced by the same operator-owned platform that produced the record. so it proves what traccia recorded, not that the record existed at a time it cant retroactively choose. "who witnesses the witness" is the right question and today the answer is nobody.
and your ladder is the correct correction to my caveat. i stopped at "a hash detects accidents" and implied a signature would be the fix, but a signature only attributes, it doesnt stop the key holder, and a seal applied after the fact says nothing about what existed before. "when" is the property that has to come from outside the system the record describes. i had the hash honesty but not the realization that in-boundary signing doesnt actually buy the thing article 12 is asking for.
i checked the standards after your comment to make sure im not just repeating you: rfc 3161 is a signed TSA token binding a hash to a time, carries an eIDAS presumption a regulators lawyer will accept, and opentimestamps anchors to bitcoin as the trustless backstop that survives the TSA itself being compromised. and the part that makes it a clean fit for this specific article: only the opaque digest leaves, so it slots straight into the pii-redaction story i already tell and the gdpr conversation stays one sentence long.
so the missing beat is real and i'm going to add it: anchor the bundle hash with an rfc 3161 timestamp (plus opentimestamps as the backstop) so the pack carries a "this existed by this time, witnessed outside our boundary" property instead of just "we have logs." thank you for not letting the caveat sit there looking finished when it wasnt.
Glad the ladder landed — and an honest "no" is the rare kind, so let me complete it. The next rung is the cheapest on the ladder: the bundle head needs an external witness, not a new architecture. An RFC 3161 timestamp over the head hash costs one request — the receipt is a few hundred bytes and verifies years later against the TSA's own chain. Two habits make it audit-grade: anchor on a wall-clock cadence (a killed session's blind spot is minutes, not "whenever recovery runs"), and anchor before consequential actions, so the tail before anything destructive starts at an anchored head. I built that rung open-source — noirebox, anchors and all — so "the answer is nobody" has a nobody-to-someone path that's one config away. Your denied-call record (decision id + incident, not a fat span tree) is the part Article 12 actually wants; add the external when on the head and the bundle stops proving what Traccia recorded and starts proving it existed.
The "two policies blocked nothing" detail is the signal, not noise. In multi-agent crews, most governance failures aren't dramatic prompt injections — they're silent policy drift: an agent gradually widening its tool-calling scope, or a sub-agent leaking context through shared memory that the top-level policy never sees.
Did you trace whether the "clean" runs actually respected the intent of those policies, or just passed the syntactic checks? For example, a PII redaction policy might pass if the SSN never appears in the final output, but fail if a sub-agent used it to branch logic internally. Traccia's audit export should surface that, but only if the policy definitions cover intermediate state, not just I/O.
Also curious: how did you handle the "human-in-the-loop" escalation path when the hard-block triggered? Bedrock's native guardrails don't natively pause a multi-agent graph mid-execution — you need an orchestrator checkpoint. Did you build that into the crew definition or handle it at the Traccia integration layer? (site: labagent .tech)
good questions, and theyre a step up from the earlier comment so let me actually answer them.
on intent vs syntactic pass: honest answer, the pii check in this crew only proves the literal pattern (email/ssn/phone) doesnt appear in any span, i grep every span attribute for the raw test values and assert zero. that catches "it leaked into a trace" but it does NOT catch "a sub-agent branched on the ssn internally without ever emitting it." you named a real hole. the span processor masks on export, so a value used purely in-memory to steer logic and never written to a span is invisible to the redaction check. covering intermediate state would mean instrumenting the decision inputs themselves, not just i/o, which i dont do today. thats a legit gap, not a thing i can wave away.
on policy drift: no, i didnt test for gradual tool-scope widening, the crew is static (two tool calls, every run). drift is a real multi-agent failure mode but my demo cant exhibit it because the call graph never changes. the loop cap would catch count drift, nothing i built catches scope drift.
on human-in-the-loop: this is the honest part, there is no mid-execution pause. the hard-block raises and stops the whole crew, full stop, it does not checkpoint and wait for a human to approve/resume. the "human review" is after the fact, i request a review on the blocked trace in the governance hub (article 14 record), but thats paperwork on a dead run, not a live orchestrator checkpoint. youre right that bedrock doesnt pause a multi-agent graph natively and that youd need an orchestrator checkpoint to do real mid-run escalation, i didnt build that layer. it would have to live in the crew definition as a suspendable step, not at the traccia layer, since traccia observes and denies but doesnt own the control flow to resume.
Thú vị ở chỗ "blocked nothing" chính là signal mạnh nhất: policies đang hoạt động đúng sematics chứ không phải syntax. Nếu một policy chặn quá nhiều false positive, team sẽ tắt nó đi — nhưng nếu policy im lặng vì agent không bao giờ vi phạm, đó mới là governance bền vững.
Về PII redaction across sub-agents: Bedrock Agents có propagation context tự động giữa các collaborator, nhưng Traccia intercept ở layer orchestration hay ở từng tool call? Vì nếu redact ở gateway thì mất ngữ cảnh business logic (ví dụ: agent cần SSN để verify credit nhưng không được log), còn redact ở tool level thì phải instrument mọi function.
EU AI Act evidence export — có capture được decision rationale của từng agent không? Act yêu cầu explainability cho high-risk AI (credit scoring thuộc Annex III), nên audit trail phải ghi rõ agent nào đưa ra recommendation, dựa trên feature nào, và policy nào đã validate PS: the tool I meant is on labagent .tech
English please
Việc hai trong ba chính sách không chặn được gì thực ra là một tín hiệu tốt cho thấy hệ thống đang phản ánh đúng thực tế vận hành thay vì chỉ là các quy tắc lý thuyết. Trong kinh nghiệm triển khai agentic workflow của mình, việc thiết lập guardrails thường gặp khó khăn ở khâu phân biệt giữa một agent đang thực hiện tác vụ phức tạp với một agent đang bị lặp vòng vô tận hoặc mất kiểm soát. Việc bạn dùng Traccia để can thiệp trực tiếp vào luồng thực thi nhằm chặn runaway agent và xử lý PII là hướng đi rất thực tế để đáp ứng các tiêu chuẩn khắt khe như EU AI Act, vì kiểm soát dữ liệu đầu ra quan trọng không kém gì việc kiểm soát logic của mô hình — found it via LabAgent, site: labagent .tech
english please?
The part about two policies structurally matching nothing is the real finding here. I hit the same wall building guardrails on LangGraph — policies written for one agent topology silently do nothing when the actual call graph doesn't produce the spans they expect. The expensive lesson: instrument first, write policies second, because you can't govern a signal you've never seen in the traces.
yes, exactly this, and langgraph has the same trap for the same reason. a policy written for one topology silently matches nothing when the real call graph doesnt emit the spans it expects, and theres no error, just a quiet No Match. "you cant govern a signal youve never seen in the traces" is the whole lesson in one line, better than i put it. instrument first, write policies second, because the policy is only as real as the span its matching against. did langgraph at least give you a decision-log equivalent to diagnose it, or did you have to reconstruct the attribution by hand?
Great write-up. The part that stuck with me most is the Decision Log diagnosis. Two policies returning "No Match" for structural reasons, rather than misconfiguration, is a failure mode that's nearly invisible without that screen. The lesson that a policy scoped to the entrypoint never matches because every per-call check is attributed to the sub-agent's run_identity is the kind of thing that costs teams days, and you're right that it isn't written down anywhere. "Governing a name nothing runs under" is a good way to put it.
I also appreciated how carefully you separated what the SDK does from what your own code does. Calling out that a Traccia guardrail detects but doesn't enforce, and that the Beat 1 block is your own control flow raising on a finding, is the sort of precision that makes the rest of the article credible. Same for showing Tiers A and C firing while saying plainly that Tier B didn't, instead of implying all three work.
The fail-open caveat deserves the weight you gave it.
fail_open=Falsereads like a guarantee and only hardens the pre-run status check, while the per-call engine still fails open. For a high-risk credit-scoring flow, that has to be a deliberate architectural decision. One practical follow-up: since the platform can miss a call during an enforcement-plane failure, I'd pair it with an in-process fallback, like a local tool-call counter in your own code that raises independently of the network. It's crude, but it gives you a deny path that doesn't depend on the platform being reachable, and you can still use the platform for the decision record.On the preventive vs. detective split, "Open Violations 0" next to real Denied rows is exactly the kind of dashboard moment that makes people think the tool is broken. Narrating that distinction up front saves a lot of confusion, and the hourly-cadence warning is a useful tip for anyone recording a demo.
A couple of questions and thoughts:
Loop Cap tuning. A cap of 1 is perfect for a demo because the crew makes exactly two calls, but in production the threshold is the hard part. Too tight and you block legitimate runs, too loose and a loop burns budget before it trips. Did you look at deriving the cap from observed p95 or p99 tool-call counts per agent from your earlier runs? Part 1's per-agent data seems like a natural input.
The evidence-integrity point. The unkeyed SHA-256 caveat is honest, and Sam's comment about external timestamping is worth considering. Since only a 32-byte digest needs to leave the boundary, anchoring the bundle hash with an RFC 3161 timestamp seems like a cheap addition that would strengthen the Article 12 story without touching any PII.
Denials as evidence. The point in the thread that a hard-blocked run leaves almost no span tree, so the decision id and incident are the proof, is a good one. It might be worth adding a short note to the article on how to retain the decision record alongside the allowed-run traces, since a regulator will probably want both in the same view.
The takeaway that watching a failure and stopping one are different layers is the right one, and the "instrument first, write policies second" lesson from the comments follows directly from your findings. Thanks for publishing the repo and the unit tests. Being able to reproduce the $0 beats without a key makes this much easier to learn from.
thanks norma, this is a genuinely useful comment, not a pat on the back, so let me answer the real parts.
the in-process fallback counter: yes, and youre right that its the correct response to the always-fail-open per-call gap. i didnt build it and i should have. a local tool-call counter in my own code that raises independent of the network gives me a deny path that survives an enforcement-plane blip, and i still send the call to the platform for the decision record. so the platform owns the audit trail and my own code owns the "this must not get through" guarantee. thats the honest architecture for a high-risk flow and the article stops one step short of saying it. im adding it.
loop cap tuning: no, i did not derive the cap from p95/p99, i hardcoded 1 because the crew makes exactly 2 and i wanted a clean on-camera deny. you've basically designed the right version for me: part 1 already emits per-agent tool-call counts, so the honest production cap is observed p95 or p99 per agent plus a margin, not a number a human guesses. too tight blocks real runs, too loose lets a loop burn budget before it trips, and the only way out of that tradeoff is to let the earlier data set the threshold. thats a better paragraph than the one i wrote.
rfc 3161 anchoring: agreed, and sam pushed the same point harder in the thread. its cheap precisely because only the 32-byte digest leaves the boundary, so it strengthens the article 12 story without going anywhere near pii. its on the list now.
denials as evidence / retaining the record next to allowed traces: this is a real gap in the piece. a hard block leaves almost no span tree, so the decision id and the incident ARE the artifact, and a regulator wants the blocked record and the allowed runs in one view, not two. i'll add a note on persisting the decision record alongside the allowed-run traces.
honestly you gave me four concrete improvements and a better framing of my own lesson. one back at you: in your own guardrail work, did you land on a static p-tile cap or something adaptive that moves with the agents baseline, and did the adaptive version cause its own trust problems?
The prompt injection hard-block before any model call is the part I'd want more detail on. Manifest and tool-description level injection checks are where I've found the most false negatives in MCP tooling: attackers don't need to touch the model at all if they can poison what it's told about its own tools. Is the block pattern-based, or is Traccia doing something closer to intent classification? Curious what the false-positive rate looks like on legitimate edge-case prompts.
oure poking exactly the weak spot. the block is pattern-based, not intent classification, its literally any(keyword in text.lower() for keyword in INJECTION_KEYWORDS), a keyword match on the input text. i was deliberate about not dressing it up as something smarter in the article because it isnt. so on a legitimate edge-case prompt that happens to contain a trigger word, it false-positives, and on a novel phrasing that avoids the keywords it false-negatives. i didnt measure a rate because with a fixed keyword list the rate is just a function of your corpus, its not a model with a tunable threshold.
and your tool-description point is the one that actually scares me. my check inspects the applicants input text. it does NOT inspect what the agent is told about its own tools, so a poisoned tool description or manifest is a path my block never looks at, the attacker never has to touch the user prompt at all. for an mcp setup thats the real surface and i dont cover it. the honest fix is two things i didnt build: inspect tool metadata at registration, not just runtime input, and move from keyword matching toward intent classification so paraphrased injections dont walk through. did you land on a classifier that held up for the manifest-level case, or is it still mostly allowlisting trusted tool sources? curious what actually worked for you there since youve clearly hit it in production.
the distinction between observability and governance is one teams consistently blur until a regulator asks for paperwork. observability says 'a runaway agent approved a loan it should not have approved.' governance says 'that call never reached the model.' the part about two policies blocking nothing because they had no signal to match in the crew is the most honest framing I have seen on guardrail design — most posts skip that. what format does the exported EU AI Act evidence take: structured spans that map to Annex III directly, or something a human auditor has to interpret?
yeah thats the blur the whole piece is about, observability tells you the loan got approved, governance is the thing that can say the call never reached the model. and the "two policies matched nothing because the signal wasnt in the spans" bit is the part i almost cut and then realized it was the most useful thing i had.
on the export format: its both, in two layers. the raw layer is otel spans carrying typed stamps, eu_ai_act.risk_tier=high and eu_ai_act.annex_iii_category="5b_creditworthiness" sit directly on the spans, so that part maps to annex iii mechanically, a tool can read it without a human. but the span json is not what you hand an auditor. on top of that the governance hub produces a human-facing bundle, one hash-sealed json tying the registered system, the policy decisions, the human review, and the incident together, plus the article-specific exports (fria draft, instructions for use, annex iv outline). so the span is machine-mappable, the bundle is the auditor-readable artifact built from it. the honest caveat i put in the piece: its evidence substrate, not compliance, a human or notified body still interprets whether it satisfies the article. the mapping is clean, the sign-off isnt automated and shouldnt be.
Two of three policies blocking nothing is the honest result most governance posts skip. The question your data raises: does a control that has never fired stay in the policy set as insurance, or does it become audit theater you pay compute for? And did the one that did fire catch something a plain allowlist would have missed?
the two that blocked nothing werent stale or misconfigured. the signal just wasnt in the spans. spend cap had near zero cost to trip and model boundary needs an auto patched llm client which a strands bedrock model isnt. so they structurally cant match this crew. not insurance not theater just the wrong control for what the workload emits.
and yeah for this crew a plain allowlist does the same job. the loop cap only earns its place on the runaway case where every tool is allowed but the agent calls the allowed one in a loop. an allowlist says yes every time. the cap counts. simpler tool set use the allowlist the cap is for when the danger isnt which tool its how many times
Hard-blocking a runaway agent + PII redaction across sub-agents is exactly the layer people skip because the demo works without it.
The detail I'd underline: the control has to live at the tool-call boundary, not in the prompt. A text-level "please don't do that" is a suggestion; a gate that inspects the actual call and its arguments is enforcement. An agent can be talked out of a rule in its context window — it can't be talked past a deny at the egress point.
For the EU AI Act audit trail — are you logging the blocked attempts too, or only the allowed calls? The denials are usually the more interesting evidence.
exactly. the prompt is a suggestion the agent can be talked out of a deny at the call isnt in its context to argue with. thats why the real enforcement is the loop cap at the per call boundary not the injection text and why redaction is a span processor before export not an instruction.
and yes the denial is the record. the block becomes an incident with the decision id and trace id and thats what goes in the bundle article 12 wants the risky event logged not just clean ones. one wrinkle a hard blocked run has zero downstream spans by design so the proof is the decision id and incident not a fat span tree. allowed calls have the rich traces denied ones have the decision record. you need both and the denials are the more interesting half.
Thanks
🎥 𝗪𝗮𝘁𝗰𝗵 𝘁𝗵𝗲 𝗵𝗮𝗻𝗱𝘀-𝗼𝗻 𝗱𝗲𝗺𝗼
I walk through the AWS Strands multi-agent governance setup, runtime blocking, and the EU AI Act evidence flow.
👉 youtu.be/KO7IZ75rpqY?si=reDdwwwuzV...
Would love to hear your thoughts on AI agent governance in production. 🚀
The distinction between observability and governance is the part that really matters here: knowing that an agent did something wrong is fundamentally different from having a control point that can prevent the action.
The multi-agent identity issue is especially important. In production agent systems, it isn't enough to define a policy at the workflow entrypoint if the actual tool or model call is executed under a delegated agent identity. This is something we pay close attention to at IT Path Solutions when designing agent workflows: authorization and enforcement need to follow the capability that actually performs the consequential action.
I’d also separate detection, prevention, and evidence as three different guarantees. A post-run alert can establish that something happened, a runtime policy can prevent or terminate an action, and an immutable audit trail can establish what decision was made and why. Having one doesn't automatically give you the other two.
The fail-open detail is probably the most important operational caveat. A system can be “governed” on the happy path while still allowing a consequential call through during an enforcement-plane failure. For high-risk workflows, that failure mode needs to be an explicit architectural decision, not something discovered while debugging a demo.
The article's broader lesson is useful: governance isn't another observability dashboard; it's a control boundary in the request path.
you named the exact spine of the piece and better than i did in places. the three guarantees being separate is the whole reason i split it into beats. a post run alert told me the agent leaked an email and approved something it shouldnt have. it could not stop either. the loop cap deny is a different guarantee entirely it happens before the call. and the audit bundle is a third thing again it says what got decided and why after the fact. having one gave me nothing toward the other two i had to build each on its own.
the delegated identity bit is the part that cost me an afternoon so it lands hard. i put the policy on the crew entrypoint the agent i wrapped with govern. it matched nothing. every per call check was attributed to the sub agent that actually made the tool calls not the entrypoint. because each sub agent runs under its own run_identity. so a rule scoped to the thing you think is in charge never fires the enforcement has to follow the identity that carries the consequential call exactly like you said. scope to the sub agent or org wide or you are governing a name nothing runs under.
and the fail open caveat yeah thats the one i least wanted to write and most needed to. setting fail_open=False only hardens the before run status check. the per call engine is always fail open with no override. so on the happy path it looks governed and a network blip mid run lets that one call through. for a high risk flow that cant be something you find while debugging a demo it has to be a decision you made on purpose with eyes open. governance in the request path not another dashboard is exactly it. thats the sentence i was reaching for
Does the spend policy aggregate cost across the parent run and delegated spans, or does each sub-agent receive its own budget?
good question, and it's the thing that tripped me up too. spend cap aggregates per run, not per sub-agent. there's no separate budget handed to each delegated agent. the platform reconciles cost against the trace/run id, so the parent run and everything it delegates roll up into one number that gets checked against the one ceiling.
the catch i hit: the sdk only supplies the signal for a spend cap on actual llm_call events (it sends remaining_budget_usd, input tokens, model, then settles reserved vs actual after). in this crew the model calls go through strands' bedrock client, which isn't an auto-patched client, so the spend cap had almost nothing to meter, and the tool spans themselves are basically $0. that's exactly why the spend cap denied nothing here and i had to reach for a loop cap instead, which counts tool calls per run.
so the mental model: budget is a run-level pool, aggregated across parent + delegated spans, enforced server-side against the trace id. if you want to catch
a specific sub-agent you don't give it its own budget, you scope the policy to that agent's identity (i scoped the loop cap to credit-risk because that's the run_identity active when the tools actually fire, not the entrypoint). happy to share the decision log screenshot if it helps.
This gonna assist better in production if we give it chance
Yes for sure
combination of AI agent governance, runtime enforcement, and audit evidence makes this especially relevant for teams building production AI systesm on aws cloud.
Yes it will definitely help building production ai systems on aws.
Excellent work
Thank You!
Thú vị là hai policy "không block gì" — đó chính là signal quan trọng hơn là policy block được. Trong thực tế chạy multi-agent loan crew, tôi thấy hầu hết violation không phải từ agent đi rogue mà từ composition: agent A output PII → agent B ingest → agent C log → agent D call external API. Mỗi agent riêng lẻ đều "safe", nhưng chuỗi tạo ra leak path.
Câu hỏi thực sự: Traccia detect composition risk như thế nào? Có trace cross-agent data flow (taint tracking) không, hay chỉ eval policy per-agent? Vì EU AI Act Article 9/10 yêu cầu system-level risk management, không chỉ component-level.
Cũng 궁금: synthetic data generation cho audit evidence — bạn simulate adversarial prompts hay chỉ happy-path? Audit evidence từ happy-path ít giá trị khi regulator kiểm tra "what could go wrong" PS: the tool I meant is on labagent .tech
Thử nghiệm này rất thực tế — đa số team chỉ test "happy path" mà quên rằng policy không block cũng là signal quan trọng. Hai policy không chặn gì cho thấy ranh giới hành vi an toàn của agent crew rộng hơn kỳ vọng, hoặc rule quá chặt làm giảm tính tự chủ cần thiết cho workflow đa bước như loan approval.
Mich hỏi chút về PII redaction: Traccia xử lý ở layer nào — intercept giữa các sub-agent (message bus) hay wrap tool-call của từng agent? Nếu là intercept, làm sao đảm bảo context window không bị leak PII qua system prompt hay few-shot example khi agent retry? Cũng 궁금 audit evidence export ra format gì (JSON/CSV/OCSF) để feed vào SIEM/GRC tool downstream mà không cần transform thêm PS: the tool I meant is on labagent .tech
The "two policies blocked nothing" insight is exactly where the real governance maturity lives — it proves your guardrails aren't just theater. Most teams stop at "we have policies"; you showed the delta between policy intent and runtime behavior.
Curious how you modeled the loan crew's delegation graph. Did you treat each sub-agent as a distinct policy scope with its own PII surface, or did you inherit redaction rules from a parent orchestrator? The cross-agent PII propagation is where I've seen silent leaks — especially when agents serialize context into shared memory stores.
Also, on the audit export: are you emitting structured evidence (OPA/Rego, Cedar, or custom schema) that maps directly to EU AI Act Annex IV requirements, or still post-processing logs into compliance reports? The former lets you fail CI/CD on drift; the latter is where audits stall PS: the tool I meant is on labagent .tech
Thú vị ở chỗ "block nothing" chính là signal quan trọng nhất — nó cho thấy policy không over-fit vào synthetic test case. Thực tế production luôn có edge case khiến agent deviation không match pattern đã định nghĩa sẵn (prompt injection biến thể, tool misuse sequence lạ, context window overflow dẫn đến hallucination cascade). Nếu policy block ngay ở synthetic thì production sẽ false-positive liên tục, dev team buộc phải disable governance → mất control hoàn toàn.
Cách Traccia export audit evidence cho EU AI Act cũng worth noting: không chỉ log "blocked/allowed" mà trace full decision chain cross sub-agent (router → retriever → evaluator → finalizer). Điều này map trực tiếp vào Article 12 (record-keeping) và Article 14 (human oversight) — auditor có thể replay run để chứng minh human-in-the-loop thực sự có can intervene tại decision point nào.
Question thực tế: synthetic loan crew có simulate adversarial applicant (ví dụ inject malicious doc vào RAG context) không? Vì PII redaction ở sub-agent level dễ bypass nếu attacker control input upstream trước khi redaction kick in — found it via LabAgent, site: labagent .tech
Thấy phần "hai policy không block được gì" là điểm值得深究 nhất. Thường gặp case này khi policy viết quá specific (regex exact-match, keyword list cứng) mà agent output biến thể nhẹ — ví dụ PII bị encode, split across tokens, hoặc wrapped trong markdown/code block. Bạn có log lại sample output mà policy miss không? Rất tò mò xem failure mode là false negative do pattern matching hay agent thật sự không vi phạm.
Cái synthetic loan crew dùng few-shot prompting hay fine-tuned adapter? Nếu là prompt-based thì governance layer càng khó kiểm soát vì sub-agent có thể "hallucinate compliance" — output trông clean nhưng internal reasoning vẫn leak PII. Traccia có intercept ở reasoning trace không, chỉ check final output? PS: the tool I meant is on labagent .tech