DEV Community

Anubhav Kumar
Anubhav Kumar

Posted on Edited on

We gave our coding agent a graph of what the team had decided

One of our test requests for Actron was to make docFromTextNotation, a helper in Calva, accept Windows line endings.

Before it started building, Actron asked us this:

docFromTextNotation is called from 5 functions and 964 tests. Should its current behaviour stay the same for them?

The 964 came from the dependencies in the graph. We were asking about one helper, but the answer had to account for the code using it.

We built Actron for FalkorDB Graph Hacks because we wanted coding agents to do more of this checking before they edit a shared function. The implementation is in the repository. The reasons behind it are often scattered across Slack, review comments and tickets.

The context we wanted the agent to have

Suppose someone asks an agent to make an API more permissive. The agent can read the code and make the change. Meanwhile, a Slack thread explains why the team deliberately kept it strict. A previous review names the caller that depends on that behavior.

Getting all of that into a prompt by hand would help, but someone has to remember where to look.

Our idea was to connect those sources and let the agent investigate them. A request would arrive through Slack or the web app. Actron would find the relevant code, check its dependencies and earlier decisions, then ask whatever still needed a person's answer. Those answers would become a ticket that the person could approve before a coding agent started work.

We used Calva's public code and git history for the demo, along with our own demo conversations. That gave us a substantial codebase to investigate and a controlled way to test conflicting requests.

What went into FalkorDB

Each workspace gets its own FalkorDB graph. Code parsing gives us functions, callers and tests. Git history adds commits and ownership signals. GitHub brings in PRs and reviews, while Slack and Linear supply conversations and tickets.

We defined the schema in files: 96 node types and 275 expanded relationship types. A small part of it explains most of the demo:

Caller ──CALLS──> Function <──CONSTRAINS── Decision
                   ↑                         │
                 TESTS                 EXTRACTED_FROM
                   │                         ↓
                  Test                  Slack message
Enter fullscreen mode Exit fullscreen mode

A function can now lead us to a decision and then to the message it came from. We keep the supporting quote on the relationship.

We also store how a relationship was made and its confidence. Some come from explicit references, some from parsing, and some from model judgments. When a connection looks wrong, we need to know which of those produced it.

The Calva ingestion we measured contained 19,952 nodes and 76,532 relationships. That included 5,533 functions, 11,754 CALLS relationships, 4,150 TESTS relationships and 2,000 commits. These were the counts at that point in the build; later requests and syncs add to them.

FalkorDB handles the relationships and the queries our agents use to investigate them. PostgreSQL keeps application state, approvals and the event journal. Redis handles things such as live-event delivery and short-lived coordination.

Text search is still useful for finding a candidate function. After that, we need to follow its callers, tests and decisions. Those traversals are where the graph earns its place in the product.

We extracted a decision that was no use to the request

Our first demo message said:

We decided to keep the old test helper API.

After fixing a model-provider error, we got Actron to extract it as a decision. But it didn't connect to any code. The message was too vague, and none of the candidate links scored above our threshold.

That was a useful failure. We'd successfully stored a decision, but the investigator couldn't use it.

For a later run, we wrote:

docFromTextNotation must stay LF-only.

This was a decision we deliberately introduced in our demo conversation. It wasn't something we found in Calva's actual team history.

This time, Actron linked the decision to the named function. When we submitted the request to accept CRLF, it quoted the earlier restriction and asked whether it should still apply. We answered that this request should allow CRLF.

That was the interaction we were trying to build. The graph found something relevant, and the agent asked us about it before proceeding.

It also left us with a harder problem to work on: how much can we recover from vague language without attaching decisions to the wrong code?

The agent gets tools for investigating the graph

The investigator has 20 read-only tools over MCP. They include find_symbol, callers, impact, decisions_for, owners, tests_for and shortest_path.

The model chooses a tool and supplies arguments. Our application runs the query, validates inputs and limits the results. The web app shows those calls on the request timeline.

For example, this is a shortened version of the query connecting a function to its decisions and their sources. The full version also checks deletion flags and decision status:

MATCH (f:CodeFunction {id: $function_id})
MATCH (f)<-[:CONSTRAINS]-(d:Decision)
OPTIONAL MATCH (d)-[e:EXTRACTED_FROM]->(source)
RETURN d.statement, e.quote, source.id
LIMIT 20
Enter fullscreen mode Exit fullscreen mode

We use PageRank over CALLS, together with transitive caller counts, to estimate how structurally important a function is. That helps Actron decide when to ask about preserving behavior for existing callers. The shortest-path tool uses algo.SPpaths to find connections we can show on the canvas.

Findings carry their paths and quotes into the UI. If Actron asks about a decision, you can inspect the message behind it and how it connects to the function. An inferred link can still be wrong, so keeping that evidence visible matters.

The query that sent us to GRAPH.EXPLAIN

While testing on Calva, this query took about 2.3 seconds for 82 IDs:

UNWIND $ids AS fid
MATCH (x:CodeFunction {id: fid})<-[:CALLS]-(c:CodeFunction)
RETURN x.id, c.id
Enter fullscreen mode Exit fullscreen mode

We expected it to look up each function by its indexed ID and then walk to its callers. GRAPH.EXPLAIN showed that the planner was starting with a label scan at the caller end instead.

We split the lookup and traversal:

UNWIND $ids AS fid
MATCH (x:CodeFunction {id: fid})
WITH *
MATCH (x)<-[:CALLS]-(c:CodeFunction)
RETURN x.id, c.id
Enter fullscreen mode Exit fullscreen mode

The same lookup then took roughly 4 milliseconds in our measurements.

We had another issue with heavily shared functions. A variable-length traversal could enumerate many paths that reached the same callers. We only needed each function at its minimum depth, so we changed the application to walk one hop at a time and deduplicate the frontier.

On the hub functions we checked, the impact query went from timing out to about 0.12 seconds. Both changes were specific to the queries and graph we were testing, but they made the investigation usable.

We kept the errors and workarounds in our FalkorDB notes, including traversal-direction surprises and a persistence failure we reproduced on our pinned version.

Then our test environment got in the way

Once the questions are answered, Actron writes a versioned ticket. Approval is tied to the SHA-256 hash of that revision. If the ticket changes, it needs approval again.

Pi, the coding agent, receives the relevant code, callers, tests and decisions from the graph. It also gets the graph tools. Temporal manages the workflow and the waits for answers and approvals.

Our first live build got stuck before any tests ran. We had configured Node in a way that bypassed the expected TypeScript loader, and the test image had JDK 17 where the ClojureScript build needed JDK 21. Pi spent time trying to repair the environment inside its container.

We fixed the runner and tightened the instructions: write a plan first, leave build infrastructure alone, and stop repeating a setup failure. With that corrected, Calva's unmodified base passed 1,698 tests through Actron's runner.

Our first completed PR was a tests-only change

The later CRLF run made it through investigation, clarification, a Linear ticket, approval, a real Pi build and testing. It opened PR #1 on our fork and wrote the result back into the graph. Actron's runner reported 1,701 passing tests.

The PR only added tests because Pi found that the helper already preserved CRLF. We had exercised the delivery workflow and added regression coverage, rather than implemented a new line-ending behavior. Approving it in Actron left the PR open for a person to merge.

A second request, adding getTextBeforeFirstOccurrenceOfSubstring, produced a source change and four tests. That build passed 1,702 tests.

During review, we found a branch-tracking bug. After an edited ticket, a rebuild pushed to a new branch while the existing PR stayed on the old one. The stale-head check caught the mismatch and refused approval. We fixed branch reuse and added regression tests, although we hadn't yet verified the fix through another live rebuild.

These runs used signed requests to exercise the Slack entry point. Typing the actual slash command and completing Slack's browser consent flow were still separate checks in our handoff.

What still needs work

We ran a 60-second load test with eight reader processes, two writer processes and the 50 most-called functions as seeds:

Measurement Result
Shared-query p95 1,747 ms
Impact-query p95 725 ms
Node writes 4,609 per second
Errors 0

The script is in the repository. Those results tell us about that workload on our graph. We still need broader testing before making claims about production capacity.

The conflict judgments need more work too. At our configured threshold of 0.8, eight of 21 positive predictions were false positives. The labeled examples were small in number and developed alongside the prompts. We'd like to calibrate against examples written independently of them; otherwise, we risk building an agent that asks people to resolve too many conflicts that aren't there.

The live runs also left us with some concrete fixes: overriding a decision didn't supersede the old one, Pi's reported spend wasn't reaching the build-cost field, and the build container still had unrestricted internet access. We also had 1,575 unresolved TypeScript calls in the ingestion, largely needing more type information.

For the next pass, the decision override is a good place to start. The answer a person gives should carry through to what the next investigation remembers.

Try it

Actron is our entry for Agents That Act on Connected Data, Agent Memory and Coordination, and Company Brain. You can read the source and setup instructions, inspect the graph model and queries, or follow the failures in the build log.

The local setup uses make up after the README's configuration steps. Live integrations need their credentials; without them, the corresponding features use labeled stubs.

Our team defined the problem, architecture, ontology and specs, then reviewed the implementation and verified it through tests and live runs.

Top comments (0)