DEV Community

Fenju Fu
Fenju Fu

Posted on

HER Hack-Astron #2: a bounty to bring Langfuse observability to astron-agent

#ai #agents #opensource #observability

When a multi-agent workflow returns a wrong answer, can you quickly answer these?

  • Which LLM call started drifting from what you expected?
  • Which tool execution or handoff node slowed the whole chain?
  • How many tokens, how much cost, and how much time did one workflow actually consume?
  • After a model or prompt change, did quality improve — or quietly regress?

As agents move from demo to production, the hard part is rarely "make it run." It's seeing how it runs, why it fails, and what each change did. HER Hack-Astron #2 opens exactly that engineering problem to the community: bring Langfuse observability to the enterprise-grade agent workflow platform astron-agent (already listed in the CNCF Landscape).

HER Hack-Astron #2 — Langfuse observability challenge

What the challenge asks for

Integrate Langfuse across astron-agent's workflow execution path so a full run becomes traceable:

  • Hierarchical traces — nest LLM calls, tool executions, retrieval, agent steps and handoffs into one traceable chain.
  • Cost & latency monitoring — token usage, model cost, per-node and end-to-end latency.
  • Evaluation — LLM-as-a-judge, custom scores and user feedback, so quality regressions have evidence.
  • Enterprise deployability — Langfuse self-hosts, which fits data, audit and compliance requirements.

This isn't a "add a doc page" or "stub an interface" task — it's a contribution that can land in the main tree and serve real production debugging and quality governance.

Three routes (pick one, or combine)

1. Native instrumentation. Wrap workflow execution nodes with the Langfuse SDK, writing key LLM / tool / retrieval / agent calls into traces, plus config for toggle, keys and host. Most direct path to a complete, usable chain.

2. Callback / hook. Give the workflow runner an extensible callback interface, then implement a Langfuse callback on top. Solves this integration and leaves a clean boundary for future observability backends. More about interface abstraction and compatibility.

3. OpenTelemetry bridge. Reuse the repo's existing OpenTelemetry infrastructure — export via OTLP with semantic conventions into Langfuse. Can stay compatible with Jaeger, Grafana and the wider OTel ecosystem at the same time.

Whichever route, a submission should include: substantive code changes; config + usage docs; a reproducible run example; and real Langfuse traces (or equivalent verification evidence).

What counts as "done"

The bar is real, runnable, reproducible — not lines of code. A PR must provide at least one of:

  • A screenshot or short video of real traces in the Langfuse dashboard, showing nested LLM / tool / retrieval / agent-step info.
  • Local run logs with full repro steps — config, launch, and the command that triggers a trace.
  • Concrete Langfuse output: trace IDs, cost / latency data, or evaluation results.

Stub code, empty docs or placeholder PRs don't count. If you used AI assistance, disclose the agent / model, main prompts, and what you personally designed and verified at the top of the PR description — tooling can speed you up, but you own the design, verification and final code.

Rewards

  • Champion (1): 500 RMB cash / equivalent gift card, plus the "HER Hack-Astron #2 Champion" title.
  • Outstanding contributions (several): an Astron swag pack, 5000 Loomy points, or a coffee voucher.

Who can join

HER Hack-Astron is an open-source challenge built to support women developers' participation and growth — individuals or teams welcome. The one hard requirement: women contributors must account for at least 50% of the code changes in the PR. You don't have to be a Langfuse or astron-agent veteran; familiarity with any of Python, agent workflows, OpenTelemetry, distributed tracing or evaluation systems is enough to start from the issue discussion.

How to submit

  1. Read Issue #1575 for the goal and acceptance criteria.
  2. Fork iflytek/astron-agent, do the code + docs + example on a branch.
  3. Open a PR titled [HER Hack-Astron #2] Langfuse integration + <your highlight>.
  4. Add Closes #1575 to the PR description.
  5. Attach reproducible steps and real trace evidence, then wait for maintainer review.

Commenting your intended route and rough design on the issue before coding usually saves everyone a duplicated implementation.

Get started

Observability decides whether an agent system can be debugged, evaluated, and ultimately trusted in production. If you want one clean, end-to-end infrastructure contribution that lands in a real open-source project, HER Hack-Astron #2 is waiting.

Her Code, Her Impact.

Top comments (0)