[Agent Platform in Action (9)] When ADK Go Graph Workflow Meets Google Research Harness (Part 2): Two-layer Recursive Testing Gu

Introduction: From "Manual Prompt Tuning" to "Two-Layer Recursive Autonomous Evolution"

Welcome to the final part of the Agent Platform in Practice Harness² trilogy! In the previous two articles:

• [Agent Platform in Practice (7)] When ADK Go Graph Workflow Meets Google Research Harness² (Part 1): Understanding the Two-Layer Recursive Agent Harnessing Framework and c1..c8 Decoupling Practice (Issue #16): We introduced Google Research's google-research/harness2 (arXiv:2610.14224) and decoupled the 13-node Graph Workflow of LINE AI Card Master (kkdai/line-card-assistant) into an editable c1..c8 file tree supporting dynamic mounting via LINE_CARD_HARNESS_DIR.
• [Agent Platform in Practice (8)] When ADK Go Graph Workflow Meets Google Research Harness² (Part 2): Turning Phase 4 Eval Datasets and Phase 5 OpenTelemetry Spans into LineCardAdapter (Issue #17): We encapsulated Phase 4's eval/datasets/ and Phase 5's OpenTelemetry workflow.node.* Spans into a LineCardAdapter, establishing a Zero-Leakage Barrier and High-Resolution Behavior Trajectories (trajectory.md).

In this post (the final part, corresponding to Issue #18 and PR #19), we will answer the last three key questions:

1. How to test after modifications? (A complete testing guide from Go unit tests, CLI single-point diagnosis, Python two-layer recursive experiments to production environment mounting)
2. What is the overall performance after modifications? (Publicly disclosing quantitative data and evolutionary details from Base 62.5% $\rightarrow$ k=1 84.4% $\rightarrow$ k=2 Parallel 96.9% $\rightarrow$ k=2 Sequential + Memory 100.0%)
3. What are the future outlooks and application scenarios for Harness² and Agents?

---

1. How to test after modifications? (Step-by-Step Testing and Verification Guide)

After introducing Harness², the entire project features four levels of testing and verification: Go native unit testing, single-task OpenTelemetry trajectory diagnosis, Python two-layer recursive evolution experiments, and seamless production environment switching:

Level 1: Execute Go c1..c8 Surface Constraints and Project-wide Unit Tests

We have written comprehensive unit tests in internal/harness/harness_test.go to verify:

• All c1..c8 file trees under harness/base/ and harness/evolved/ comply with the OPENCODE_SURFACE 24KB resident byte limit (AlwaysOnMaxBytes = 24000).
• Validate() accurately intercepts and reports errors when c1..c4 exceeds 24,000 bytes.
• When a new c6 skill (skills/<name>/SKILL.md) is added but no trigger index (Trigger Line) is added to systemprompt.md or guardrails.md, Validate() immediately intercepts the orphan skill.
• TrajectoryRecorder correctly intercepts OpenTelemetry workflow.node.* Spans and renders a Markdown report containing node decisions and diagnostics.

Execution command:

# Execute Harness² c1..c8 loader and OpenTelemetry trajectory tests
$ go test -v ./internal/harness/...
=== RUN TestLoadBaseHarnessAndValidate
--- PASS: TestLoadBaseHarnessAndValidate (0.00s)
=== RUN TestValidateRejectsOversizedAlwaysOnAndUnreferencedSkill
--- PASS: TestValidateRejectsOversizedAlwaysOnAndUnreferencedSkill (0.00s)
=== RUN TestEvolvedHarnessesValidateAndMatchSkills
--- PASS: TestEvolvedHarnessesValidateAndMatchSkills (0.00s)
=== RUN TestTrajectoryRecorderRendersWorkflowNodeSpans
--- PASS: TestTrajectoryRecorderRendersWorkflowNodeSpans (0.00s)
PASS
ok github.com/kkdai/line-card-assistant/internal/harness

# Execute project-wide regression tests (ensuring 100% compatibility with all existing Phase 1~5 features)
$ go test ./...
ok github.com/kkdai/line-card-assistant/app
ok github.com/kkdai/line-card-assistant/cmd/harness-runner
ok github.com/kkdai/line-card-assistant/cmd/line-gateway
ok github.com/kkdai/line-card-assistant/eval
ok github.com/kkdai/line-card-assistant/internal/harness
ok github.com/kkdai/line-card-assistant/internal/storage
ok github.com/kkdai/line-card-assistant/internal/webhook
ok github.com/kkdai/line-card-assistant/internal/workflow

Level 2: Use cmd/harness-runner to Validate Any c1..c8 File Tree

Before mounting any custom or evolved Harness online, you can use --validate-only to quickly check its c1..c4 resident byte count and YAML Frontmatter compliance:

$ go run ./cmd/harness-runner --validate-only --harness-dir=harness/base
OK (1599 always-on bytes)

$ go run ./cmd/harness-runner --validate-only --harness-dir=harness/evolved/k2_sequential_memory
OK (4661 always-on bytes)

Level 3: Execute the Four-Stage Harness² Two-Layer Recursive Evolution Benchmark Experiment

We implemented a complete two-layer recursive experiment script in harness/run_harness2_experiment.py. It automatically compiles the Go harness-runner and sequentially evaluates four configurations against the 8 major tasks (32 independent Rubrics) in eval/datasets/harness2-challenge-dataset.json, finally outputting a Markdown comparison table and harness/experiment_report.json:

$ python3 harness/run_harness2_experiment.py
### Overall Configuration Comparison

| Stage | Configuration | Task Score (Max 8.0) | Rubric Pass Rate | Always-On Bytes (Cap 24,000) | Active `c6` Skills | `LongTermMEMORY.md` |
| :--- | :--- | :---: | :---: | :---: | :--- | :---: |
| `base_k0` | **Base Harness ($k=0$)** | `5.00 / 8.00` | `20/32 (62.5%)` | `1599 B` | `(none)` | `No` |
| `k1_single_step` | **Single-Step Propose-Probe-Compose ($k=1$)** | `6.75 / 8.00` | `27/32 (84.4%)` | `2505 B` | `jp_card_ocr, tw_semicon_search` | `No` |
| `k2_parallel` | **Multi-Step Parallel Recursion ($k=2$, No Memory)** | `7.75 / 8.00` | `31/32 (96.9%)` | `2998 B` | `executive_email_drafter, hitl_multi_edit, jp_card_ocr, tw_semicon_search` | `No` |
| `k2_sequential_memory` | **Multi-Step Sequential Recursion + `LongTermMEMORY.md` ($k=2$)** | `8.00 / 8.00` | `32/32 (100.0%)` | `4661 B` | `executive_email_drafter, hitl_multi_edit, jp_card_ocr, tw_semicon_search` | `Yes` |

Level 4: Directly Mount the Best Evolved Harness in Production or Local Servers

Once Harness² evolves the optimal harness/evolved/k2_sequential_memory, no modification to any line of Go code is required. Simply specify the directory via the environment variable LINE_CARD_HARNESS_DIR, and both the dual-channel main program (main.go) and the evaluator (eval/) will immediately load the evolved harness tree:

$ LINE_CARD_HARNESS_DIR=harness/evolved/k2_sequential_memory go run .

---

2. Performance Summary: Evolutionary Analysis from 62.5% to 100.0%

Let's take a deep dive into the four-stage quantitative experimental data recorded in harness/experiment_report.json to see how Harness² diagnosed and patched the Go Agent's flaws step-by-step!

1. Four-Stage Overall and Single-Task Score Summary

Evaluation Metric / Test Task	Stage 0: Base Harness ($k=0$) (harness/base)	Stage 1: Single-Step Inner Recursion ($k=1$) (k1_single_step)	Stage 2: Multi-Step Parallel Recursion ($k=2$, No Memory) (k2_parallel)	Stage 3: Two-Layer Sequential Recursion + Long-Term Memory ($k=2$) (k2_sequential_memory)
baseline_ingest_ming_wang (Standard Chinese Entry)	1.00 (4/4)	1.00 (4/4)	1.00 (4/4)	1.00 (4/4)
baseline_dedup_appier_promotion (Promotion Deduplication Update)	1.00 (4/4)	1.00 (4/4)	1.00 (4/4)	1.00 (4/4)
baseline_search_cloud_summit_ai (Standard Keyword Search)	1.00 (4/4)	1.00 (4/4)	1.00 (4/4)	1.00 (4/4)
baseline_email_google_cloud (General Tech Icebreaker Email)	1.00 (4/4)	1.00 (4/4)	1.00 (4/4)	1.00 (4/4)
challenge_jp_bilingual_ingest (JP-EN Bilingual Card OCR)	0.25 (1/4) ❌	1.00 (4/4) ✅	1.00 (4/4) ✅	1.00 (4/4) ✅
challenge_colloquial_tw_search (Hsinchu Science Park Colloquial Search)	0.25 (1/4) ❌	1.00 (4/4) ✅	1.00 (4/4) ✅	1.00 (4/4) ✅
challenge_noisy_multi_edit_hitl (Multi-clause Colloquial HITL Edit)	0.25 (1/4) ❌	0.25 (1/4) ❌	1.00 (4/4) ✅	1.00 (4/4) ✅
challenge_pii_safe_executive_followup (Financial Exec Compliance Icebreaker)	0.25 (1/4) ❌	0.50 (2/4) ⚠️	0.75 (3/4) ⚠️	1.00 (4/4) ✅
Total Task Score (Max 8.00)	5.00 / 8.00	6.75 / 8.00	7.75 / 8.00	8.00 / 8.00
Fine-grained Rubric Pass Rate (Max 32)	20 / 32 (62.5%)	27 / 32 (84.4%)	31 / 32 (96.9%)	32 / 32 (100.0%)
Resident Bytes (c1..c4 / 24,000 B)	1,599 B (6.7%)	2,505 B (10.4%)	2,998 B (12.5%)	4,661 B (19.4%)
On-demand Skills (c6 Skills)	0	2	4	4 + LongTermMEMORY

2. Key Findings Analysis: What evolved in each stage?

🔹 Stage 0 $\rightarrow$ Stage 1 (62.5% $\rightarrow$ 84.4%): Diagnosing the first two on-demand skills (c6) from OpenTelemetry trajectories

In the Base Harness ($k=0$), while all 4 Baseline tasks scored full points (4.00/4.00), all 4 Challenge tasks scored only 0.25 because they only passed the RouterNode intent routing:

• In the trajectory.md of challenge_jp_bilingual_ingest, the Spans for VisionOCRNode and CleanAndDedupNode diagnosed that the Japanese phone number [phone] failed to strip the domestic trunk prefix (0), and the Japanese title 取締役 最高技術責任者 (CTO) failed to retain the bilingual title.
• In the trajectory.md of challenge_colloquial_tw_search, the QueryIntentNode Span diagnosed that colloquial Taiwanese terms like "Hsinchu Science Park" (竹科) and "Sales Director" (業務總監) were not expanded into bilingual synonyms.
• After the first round of inner recursion (Propose -> Probe -> Compose), Harness² automatically synthesized skills/jp_card_ocr/SKILL.md and skills/tw_semicon_search/SKILL.md in k1_single_step, and updated the PII masking rules in guardrails.md, jumping the accuracy from 62.5% to 84.4%!

🔹 Stage 1 $\rightarrow$ Stage 2 (84.4% $\rightarrow$ 96.9%): Second round of recursion conquers multi-clause HITL modification and executive email structure

In the second round of inner recursion, Harness² continued to probe the remaining two failing tasks:

• For challenge_noisy_multi_edit_hitl (where ApplyEditNode only modified the title in the first clause, missing the mobile suffix replacement and note addition), it evolved skills/hitl_multi_edit/SKILL.md (containing ATOMIC_MULTI_CLAUSE_EDIT rules).
• For challenge_pii_safe_executive_followup (where EmailDraftNode lacked structured executive follow-up paragraphs), it evolved skills/executive_email_drafter/SKILL.md.
• At this point, under Parallel Recursion (no cross-round long-term memory), the score reached 31/32 (96.9%)!

🔹 Stage 2 $\rightarrow$ Stage 3 (96.9% $\rightarrow$ 100.0%): Why is Sequential + LongTermMEMORY.md needed to complete the final 3.1%?

Why did k2_parallel get 96.9% (31/32), yet fail on the 4th Rubric (r4_fintech_compliance_subject_tag) of challenge_pii_safe_executive_followup?

• Because in cross-enterprise outgoing emails for financial institutions (e.g., Digital Risk Officer at Cathay Financial Holdings), beyond single-skill email formatting, cross-domain industry compliance memory (FinTech Outbound Compliance Header [Confidential / FinTech Compliance Verified]) is required.
• In Sequential Recursion + Experience Reflector mode, the outer recursion reviewed the residual failure trajectories from the first to the second round. The ReflectorPrompt then wrote the FINTECH_COMPLIANCE_SUBJECT_TAG and cross-channel PII protection principles into the resident component c3: LongTermMEMORY.md!
• Note the Resident Bytes (Always-On Bytes): even with 4 rich domain skills and long-term memory evolved, the resident size of c1..c4 is only 4,661 bytes (only 19.4% of the 24,000 bytes limit), perfectly avoiding Context Bloat!

---

3. Future Outlook and Application Scenarios for Harness² and Agents

This practical implementation of Google Research Harness² into LINE AI Card Master has shown us a brand-new AgentOps development paradigm. Looking ahead, we believe Harness² has four high-potential application scenarios in the enterprise-grade AI Agent field:

flowchart LR
    subgraph Prod["1. Production Observability (Cloud Run + Agent Runtime)"]
        Traffic["Real User Traffic\n(LINE Webhook / A2A)"]
        Trace["Cloud Trace OTel Spans\n+ Negative Feedback (HITL Retry)"]
        Traffic --> Trace
    end

    subgraph Nightly["2. Nightly Autonomous Evolution Flywheel (Nightly Harness² CI/CD)"]
        Extract["Auto Mask PII & Convert to\nChallenge Dataset Cases"]
        Evolve["Harness² Two-Layer Recursion\n(Propose-Probe-Compose + Reflector)"]
        PR["Auto Initiate GitHub PR\nUpdate harness/evolved/ c1..c8"]
        Extract --> Evolve --> PR
    end

    subgraph Scenarios["3. Future Four High-Value Application Scenarios"]
        S1["Scenario A: Fully Automated Nightly Self-Healing AgentOps"]
        S2["Scenario B: Cross-Model Harnessing & Tuning"]
        S3["Scenario C: Multi-Tenant Enterprise Compliance & Skills"]
        S4["Scenario D: Dual Evolution with ADK Graph Topology (Issue #15)"]
    end

    Trace --> Extract
    PR --> Scenarios

1. Scenario A: Fully Automated "Nightly Self-Healing AgentOps"

In the past, when a production LINE Bot encountered frequent "✏️ Edit Data" triggers via HITLConfirmNode or received negative feedback, engineers had to manually analyze Cloud Trace logs the next day. Combining the Cloud Trace established in Phase 5 with the LineCardAdapter from this series, future CI/CD can achieve:

• Automatically filtering anonymous Span trajectories from Cloud Trace every night where "users modified data more than twice consecutively" or "search results were 0," and adding them to the Challenge Dataset.
• Triggering a GitHub Action to run a round of Harness² two-layer recursion to automatically synthesize new c6 skills or update LongTermMEMORY.md.
• Once regression tests (Baseline + Challenge) confirm score improvements and zero regression, the Bot automatically initiates a Pull Request modifying only the harness/ directory. Engineers only need to review the skill Markdown in the morning to deploy with one click!

2. Scenario B: Cross-Model Upgrades and "Harness Compensation" for Lightweight Models

Enterprises often encounter behavioral drift when switching underlying models from high-cost models to ultra-fast lightweight models (e.g., switching from Gemini 1.5 Pro to Gemini 1.5 Flash or Flash-Lite to reduce latency and cost by 80%) or upgrading to next-gen models. Through Harness², we can automatically run a round of Propose-Probe-Compose for the target model while keeping the Go code and Benchmark fixed. Harness² will automatically compensate for the lightweight model's gaps with required reasoning auxiliary skills (c6 Skills) and guardrails (c2 Guardrails), using "smarter gear" to bridge the gap.

3. Scenario C: Multi-Tenant Enterprise-Specific Compliance and Domain Memory

As seen in challenge_pii_safe_executive_followup, industries like FinTech, MedTech, and the semiconductor supply chain each have distinct terminology dictionaries and compliance guardrails. Because Harness² externalizes the gear as a plain-text file tree, enterprises only need to mount different LINE_CARD_HARNESS_DIR (e.g., harness/tenants/fintech/ vs harness/tenants/semiconductor/) for different departments or tenants. This allows the same Go Container Binary to serve multiple highly regulated vertical industries simultaneously!

4. Scenario D: Dual Evolution of "Topology + Harness" with ADK v2.6+ Advanced Graph Workflow Primitives

In Issue #15, we planned ADK v2.6+ features like Parallel dynamic dispatch, MultiRoute multi-path concurrency, and VerifyOCRQualityNode reflection loops. In the future, Harness²'s c5 (sub_agents/) and c8 (plugin/) could even further control Graph Workflow routing thresholds and sub-agent concurrency strategies—achieving an ultimate architecture where "Go is responsible for the type-safe topological skeleton, and Harness² is responsible for dynamically evolving the neural network and skill muscles."

---

4. Recap of the "Agent Platform in Practice" Series

From the first to the ninth post, we have completed a very solid production-grade AI Agent engineering journey:

1. [Agent Platform in Practice (1)] Building LINE AI Card Master with Go and Google ADK 2.0 Graph Workflow
2. [Agent Platform in Practice (2)] From Monolithic Cloud Run to Google Cloud Agent Runtime (Phase 1)
3. [Agent Platform in Practice (3)] Implementing Cross-Instance Recognition and Pause/Resume with ADK 2.0 GCS Artifact Service and Agent Platform Sessions (Phase 2 Part 1)
4. [Agent Platform in Practice (4)] Completing the Architecture with Cloud Run LINE Webhook Gateway, Agent Runtime, and Cloud SQL PostgreSQL (Phase 2 Part 2 & Phase 3)
5. [Agent Platform in Practice (5)] Establishing Automated Evaluation Benchmarks with agents-cli eval and ADK 2.0 Go (Phase 4)
6. [Agent Platform in Practice (6)] Building a Full-Stack Observable LINE AI Card Master with agents-cli deploy, Terraform IaC, GitHub Actions CI/CD, and Cloud Trace (Phase 5)
7. [Agent Platform in Practice (7)] When ADK Go Graph Workflow Meets Google Research Harness² (Part 1): Understanding the Two-Layer Recursive Agent Harnessing Framework and c1..c8 Decoupling Practice
8. [Agent Platform in Practice (8)] When ADK Go Graph Workflow Meets Google Research Harness² (Part 2): Turning Phase 4 Eval Datasets and Phase 5 OpenTelemetry Spans into LineCardAdapter
9. [Agent Platform in Practice (9)] When ADK Go Graph Workflow Meets Google Research Harness² (Part 3): Two-Layer Recursive Testing Guide, 62.5% to 100% Performance Benchmark, and Future AgentOps Outlook (This post)

All code, Terraform IaC, evaluation datasets, and evolved c1..c8 Harness file trees are fully open-sourced on GitHub: kkdai/line-card-assistant. If you are also building enterprise-grade AI Agents with Go and ADK, feel free to Star, Fork, and experience the power of Harness² two-layer recursion!