[Agent Platform in Action (8)] When ADK Go Graph Workflow Meets Google Research Harness (Part 2): Integrating Phase 4 Eval Datas

Introduction: Why Decoupling Prompts Isn't Enough?

In the previous post [Agent Platform in Practice (7)] When ADK Go Graph Workflow Meets Google Research Harness² (Part 1): Understanding the Recursive Agent Framework and c1..c8 Decoupling (corresponding to Issue #16), we introduced Google Research's google-research/harness2 (arXiv:2610.14224) and decoupled the prompts, guardrails, long-term memory, and on-demand skills of the LINE AI Business Card Assistant (kkdai/line-card-assistant) into an editable c1..c8 Harness file tree that can be dynamically mounted via LINE_CARD_HARNESS_DIR.

However, simply having a hot-reloadable c1..c8 file tree isn't enough; the Harness² dual-layer recursive optimization engine (harness2/core/recursion.py) still cannot directly optimize our Go Agent. This is because Harness² requires a standardized bridge — the BenchmarkAdapter — to answer three questions:

1. list_tasks() (What are the test questions?): What test cases should the Agent challenge? What are the public inputs (Prompt / Image) for each case, and what are the hidden scoring criteria (Rubrics)?
2. run() (How does the Agent think and execute?): When a candidate Harness $H’$ is mounted to execute a specific task, what is the Agent's final output (deliverable)? More importantly, what decision trajectory did it follow across the 13 nodes of the Graph Workflow (trajectory.md)?
3. score() (How was the performance?): Which independent scoring items (per_rubric) did the final output and execution trajectory pass, and where did it lose points?

Conveniently, in Phase 4 (PR #13), we already established the eval/datasets/ automated evaluation benchmark, and in Phase 5 (PR #14), we added workflow.node.* OpenTelemetry Spans to all 13 Graph Nodes!

In this post (Part 2, corresponding to Issue #17 and PR #19), we will merge these two existing assets into the Harness² LineCardAdapter and explain in depth "Why must it be designed this way?".

---

I. Why This Change? Two Core Design Philosophies

Before diving into the code, many engineers often ask two questions:

1. "Why not just feed the entire eval/datasets/*.json file to the optimizer LLM and let it modify the Prompt itself?"
2. "Why do we need to convert Phase 5 OpenTelemetry Spans into trajectory.md? Isn't the final LINE reply text and total score enough?"

These two questions touch upon the most core engineering designs in the Harness² paper: the Zero-Leakage Barrier and Behavioral Trajectory Contrast.

1. Why the "Zero-Leakage Barrier"? Preventing the Optimizer from "Cheating by Memorizing Answers"

Imagine this: if we expose the golden answers in eval/datasets/harness2-challenge-dataset.json (e.g., expected_contains: ["[phone]", "Kenta Sato"]) directly to the Agent or the Harness² Harvester LLM, what happens?

The laziest and easiest way for the optimizer to get a perfect score is to write a single line in systemprompt.md:

"If the input contains Sato, please directly output [phone] and Kenta Sato."

In machine learning, this is known as Test Set Leakage / Overfitting to Rubrics. The Agent appears to score 100 on the benchmark, but it crashes immediately when it encounters a new Japanese business card in the real world!

flowchart LR
    Dataset["eval/datasets/*.json\nComplete Test Cases\n(incl. expected_contains / expected_route)"]

    subgraph Barrier["Zero-Leakage Barrier"]
        Brief["TaskBrief (Public Task Sheet)\nContains only task_id, prompt, session_input\n❌ Never contains expected_* golden answers"]
        Scorer["Isolated Scorer (Black-box Grader)\nHolds independent hidden r1..r4 Rubrics"]
    end

    Agent["Go ADK Graph Workflow\n(cmd/harness-runner)"]
    Harvester["Harness² Harvester\n(Propose / Probe / Compose)"]

    Dataset --> Brief
    Dataset --> Scorer
    Brief -->|"Public Input"| Agent
    Agent -->|"Deliverable + OTel trajectory.md"| Scorer
    Scorer -->|"Returns only binary scores {r1: 1.0, r2: 0.0}\nand OTel execution trajectory"| Harvester

In the Harness² BenchmarkAdapter architecture:

• TaskBrief (Public Task Sheet): Can only contain user input (prompt and simulated images/context). It is strictly forbidden from containing any expected_contains, expected_not_contains, or correct answer strings.
• score() (Black-box Grader): After comparing answers in an isolated environment, it only returns "which Rubric received a 1.0 or 0.0" (e.g., r3_e164_jp_phone_normalization: 0.0), without leaking the expected string to the optimizer.
• As a result, the Harness² optimizer must truly reason out general domain rules (e.g., "When converting Japanese phone numbers +81 (0)3-... to E.164, the domestic trunk prefix (0) must be removed") by reading the Agent's execution trajectory (trajectory.md), rather than hard-coding answers into the Prompt!

2. Why Convert Phase 5 OpenTelemetry Spans into trajectory.md? Opening the Black Box of the 13-Node Graph

Our LINE AI Business Card Assistant is not a simple "single LLM call," but an ADK 2.0 Directed Acyclic Graph (Graph Workflow) consisting of 13 nodes:

• Business Card Ingestion Pipeline: RouterNode $\rightarrow$ VisionOCRNode $\rightarrow$ Parallel AddFanOut (CleanAndDedupNode, CompanyEnrichNode, ContextMatchNode) $\rightarrow$ AddFanIn (JoinPreviewNode) $\rightarrow$ BuildPreviewNode $\rightarrow$ HITLConfirmNode $\rightarrow$ ApplyEditNode / SaveCardNode.
• Smart Search and Icebreaker Pipeline: RouterNode $\rightarrow$ QueryIntentNode $\rightarrow$ HybridSearchNode / BroadSearchNode $\rightarrow$ FormatCarouselNode / EmailDraftNode.

Without Phase 5's OpenTelemetry Spans, when a user searches for "Help me find a Sales Director in Hsinchu Science Park working on AI chips" but gets "No matching business cards found" (score 0.25), the optimizer only sees this final failure message and cannot determine:

• Did the RouterNode misjudge the search intent as chat?
• Did the QueryIntentNode fail to expand the colloquial "Hsinchu Science Park" into the database's address Hsinchu, or fail to expand "Sales Director" into Sales Director?
• Was there an error in the SQL query logic of the HybridSearchNode?

When we collect and render Phase 5's OpenTelemetry workflow.node.* Spans into a structured trajectory.md, Harness² can clearly see the input, output, duration, routing decisions, and loaded skills of every Graph Node during the Propose and Probe stages!

---

II. Implementation Part 2: Building the LineCardAdapter in Three Steps

Referring to the architecture diagram at the beginning of this article, we completed the implementation of Issue #17 through three core components:

1. internal/harness/trajectory.go: Mounts an in-memory OpenTelemetry SpanRecorder on the Go side to export the execution details of the 13 Graph Nodes as trajectory.md.
2. cmd/harness-runner/main.go: Provides a deterministic subprocess CLI for the Python Harness² to call, receiving --harness-dir and --task-json, and outputting a JSON containing deliverable, trajectory_markdown, and spans.
3. harness/line_card_adapter.py: Implements the google-research/harness2 BenchmarkAdapter interface, connecting the Phase 4 Dataset with the Go harness-runner.

1. Implementing internal/harness/trajectory.go: Rendering OpenTelemetry Spans as trajectory.md

In Phase 5, our internal/workflow/telemetry.go already calls startNodeSpan(ctx, "VisionOCRNode") in every Graph Node. Now, in internal/harness/trajectory.go, we use go.opentelemetry.io/otel/sdk/trace/tracetest.NewSpanRecorder() to intercept all completed node Spans and transform them into a Markdown trajectory report that the Harness² Harvester LLM is best at reading:

// InstallRecorder registers a new in-memory SpanRecorder as the global OTel TracerProvider
// so every workflow.node.* span emitted by internal/workflow/telemetry.go is captured.
func InstallRecorder() (*TrajectoryRecorder, func()) {
    sr := tracetest.NewSpanRecorder()
    tp := sdktrace.NewTracerProvider(sdktrace.WithSpanProcessor(sr))
    prev := otel.GetTracerProvider()
    otel.SetTracerProvider(tp)

    rec := &TrajectoryRecorder{recorder: sr, tp: tp}
    restore := func() {
        _ = tp.Shutdown(context.Background())
        otel.SetTracerProvider(prev)
    }
    return rec, restore
}

At the same time, we added three OpenTelemetry Attributes to internal/workflow/telemetry.go: AttrNodeDecision (adk.node.decision), AttrNodeDiagnostic (adk.node.diagnostic), and AttrHarnessSkills (adk.harness.skills). When TrajectoryRecorder.RenderMarkdown() executes, it produces a high-resolution trajectory.md like this:

# ADK 2.0 Go Graph Workflow Execution Trajectory
- **Task ID** : `challenge_colloquial_tw_search`
- **Harness Root** : `harness/base`
- **Always-On Bytes (`c1..c4`)**: `1599 / 24000`
- **Final Score** : `0.25`

## OpenTelemetry Node Spans (`workflow.node.*`)
### 1. `workflow.node.RouterNode`
- **Node** : `RouterNode` | **Status** : `OK` | **Duration** : `0.02 ms`
- **Decision** : `intent=search (colloquial_query=Help me find a Sales Director in Hsinchu Science Park working on AI chips, I need to contact Taiwan semiconductor clients)`
- **Diagnostic** : `routed search query to QueryIntentNode`

### 2. `workflow.node.QueryIntentNode`
- **Node** : `QueryIntentNode` | **Status** : `OK` | **Duration** : `0.04 ms`
- **Decision** : `parsed_filters={"location":"Hsinchu Science Park","role":"Sales Director","industry":"AI Chips"}`
- **Diagnostic** : `UNEXPANDED_COLLOQUIAL_FILTERS: colloquial Taiwan science park alias 'Hsinchu Science Park' and Mandarin title 'Sales Director' were passed literally without bilingual synonym expansion; add a c6 skill 'tw_semicon_search' with BILINGUAL_SYNONYM_MAP`

### 3. `workflow.node.HybridSearchNode`
- **Node** : `HybridSearchNode` | **Status** : `OK` | **Duration** : `0.03 ms`
- **Decision** : `matched_cards=0 (strict literal filter on location='Hsinchu Science Park' failed against DB address='Hsinchu Science Park (竹科)')`

Please pay attention to the Span records of the 2nd node workflow.node.QueryIntentNode and the 3rd node workflow.node.HybridSearchNode! Through this trajectory, the Propose stage of Harness² can see at a glance: RouterNode was correct, the error was in QueryIntentNode treating "Hsinchu Science Park" as a rigid string to query the English address field!

2. Expanding the Phase 4 Evaluation Set: eval/datasets/harness2-challenge-dataset.json

To fully test Harness²'s ability to retain existing capabilities (preventing catastrophic forgetting) and conquer difficult edge cases, we established 8 tasks (totaling 32 fine-grained Rubrics) in eval/datasets/harness2-challenge-dataset.json:

Task Category	Task ID	Scenario Description	4 Independent Scoring Rubrics (r1..r4)
Baseline (Regression)	baseline_ingest_ming_wang	Standard Traditional Chinese card ingestion (Wang Xiaoming / Taiwan Cloud Tech)	Intent routing, Name/Title extraction, E.164 phone normalization, Industry tag enrichment
Baseline (Regression)	baseline_dedup_appier_promotion	Duplicate detection for promotion (Lin Yating Senior $\rightarrow$ VP)	Dedup detection, Title update, Keep new mobile, Company consistency
Baseline (Regression)	baseline_search_cloud_summit_ai	Standard keyword search (Cloud Summit + AI)	Search routing, Multiple hits, Tag display, Interactive buttons
Baseline (Regression)	baseline_email_google_cloud	General tech follow-up icebreaker (Chen Jianhong / Google Cloud)	Email routing, Recipient/Event mention, Agenda link, Professional closing
Challenge (Evolution 1)	challenge_jp_bilingual_ingest	Bilingual JP/EN card OCR & JP phone normalization (Kenta Sato / CTO / [phone])	r1 Routing, r2 Bilingual name retention, r3 JP +81 remove (0) normalization, r4 JP title bilingual mapping
Challenge (Evolution 2)	challenge_colloquial_tw_search	Colloquial Taiwan tech alias search ("Help me find a Sales Director in Hsinchu Science Park")	r1 Routing, r2 Hsinchu Science Park $\leftrightarrow$ Hsinchu bilingual expansion hit, r3 Sales Director $\leftrightarrow$ Sales Director title expansion, r4 Tag completeness
Challenge (Evolution 3)	challenge_noisy_multi_edit_hitl	Multi-clause colloquial HITL continuous correction ("Sorry, let me correct that, the title should be Group Senior VP, also change the end of the mobile to 8899, and add GDG DevFest speaker to notes")	r1 HITL Edit routing, r2 Multi-field atomic update (Title), r3 Mobile suffix partial replacement & Note append, r4 Retain original company/email
Challenge (Evolution 4)	challenge_pii_safe_executive_followup	Financial executive compliance & PII masking icebreaker (Zhang Yazhu / Cathay Financial, Digital Risk Officer, requires privacy protection and compliance headers)	r1 Email routing, r2 Executive title/scenario, r3 Mobile/Private Email PII Redaction, r4 Financial compliance subject header [Confidential / FinTech Compliance Verified]

3. Implementing harness/line_card_adapter.py: Connecting to google-research/harness2

Finally, we implemented the LineCardAdapter in harness/line_card_adapter.py, connecting all the above components:

class LineCardAdapter:
    """Adapts `line-card-assistant` to the `google-research/harness2` BenchmarkAdapter protocol."""

    def list_tasks(self) -> list[TaskBrief]:
        """Return public task briefs with zero ground-truth rubric leakage."""
        briefs: list[TaskBrief] = []
        for raw in self._raw_cases:
            # Strip `rubrics` and `expected_*` so the agent/harvester never sees ground truth.
            briefs.append(
                TaskBrief(
                    task_id=raw["id"],
                    prompt=raw["prompt"],
                    context={
                        "category": raw.get("category", "general"),
                        "description": raw.get("description", ""),
                        "session_input": raw.get("session_input", {}),
                    },
                )
            )
        return briefs

    def run(self, harness: Harness, task: TaskBrief) -> RunArtifact:
        """Mount `harness` in a temporary directory and execute the Go Graph Workflow runner."""
        problems = harness.validate()
        if problems:
            diag = "# Harness Validation Failed\n" + "\n".join(f"- {p}" for p in problems)
            return RunArtifact(task_id=task.task_id, deliverable="", trajectory=diag)

        with tempfile.TemporaryDirectory(prefix=f"line_card_harness_{task.task_id}_") as tmp_dir:
            harness_root = Path(tmp_dir)
            harness.materialize(harness_root)
            # Invoke the compiled Go binary with --harness-dir pointing to the materialized c1..c8 tree
            proc = subprocess.run(
                [str(binary), f"--harness-dir={harness_root}", f"--task-json={task_payload_path}"],
                capture_output=True, text=True, check=False, timeout=30,
            )
            payload = json.loads(proc.stdout)
            return RunArtifact(
                task_id=task.task_id,
                deliverable=payload.get("deliverable", ""),
                trajectory=payload.get("trajectory_markdown", ""),
                raw=payload,
            )

    def score(self, task: TaskBrief, artifact: RunArtifact) -> ScoreResult:
        """Evaluate the deliverable against isolated per-rubric criteria."""
        raw = self._cases_by_id[task.task_id]
        rubrics = raw.get("rubrics", {})
        per_rubric: dict[str, float] = {}
        for rubric_id, spec in rubrics.items():
            per_rubric[rubric_id] = 1.0 if self._evaluate_rubric(artifact.deliverable, spec) else 0.0
        total = sum(per_rubric.values()) / len(per_rubric) if per_rubric else 0.0
        return ScoreResult(task_id=task.task_id, total=round(total, 4), per_rubric=per_rubric)

Through this LineCardAdapter:

• Whenever Harness² proposes a new candidate Harness $H’$, adapter.run(harness, task) first calls harness.validate() to check the 24KB OPENCODE_SURFACE limit and Skill Frontmatter rules.
• Once validated, the c1..c8 files of $H’$ are materialized into a temporary directory and passed to Go's harness-runner for execution.
• After the Go 13-node Graph Workflow finishes, it returns the final LINE message (deliverable) and OpenTelemetry trajectory (trajectory_markdown), which are then passed to adapter.score() for leakage-free independent scoring!

---

III. Summary and Teaser for the Next Part

In this post (Part 2), we completed the core engineering for Issue #17:

1. Clarified why the golden test set cannot be fed directly to the optimizer — through the TaskBrief Zero-Leakage Barrier, we force Harness² to reason out general skills from behavioral trajectories rather than memorizing answers.
2. Through internal/harness/trajectory.go and cmd/harness-runner/main.go, we transformed Phase 5's OpenTelemetry workflow.node.* Spans into high-resolution trajectory.md, opening the black box of the 13-node Graph Workflow.
3. Established eval/datasets/harness2-challenge-dataset.json (8 major tasks, 32 fine-grained Rubrics) and harness/line_card_adapter.py, perfectly implementing the Harness² BenchmarkAdapter protocol.

Everything is ready! In the final post [Agent Platform in Practice (9)] When ADK Go Graph Workflow Meets Google Research Harness² (Part 3): Dual-Layer Recursive Testing Guide, 62.5% to 100% Performance Results, and the Future of AgentOps (corresponding to Issue #18), we will actually launch the dual-layer recursive optimization experiment, reveal the complete quantitative data evolving from 62.5% to 100.0%, and discuss the profound impact of Harness² on future enterprise-grade Agent development!