DEV Community

Cover image for BUILDING KYNTHERA: TURNING AI AGENT FAILURES INTO RECOVERY INTELLIGENCE WITH HYDRADB
Jayking♟️
Jayking♟️

Posted on

BUILDING KYNTHERA: TURNING AI AGENT FAILURES INTO RECOVERY INTELLIGENCE WITH HYDRADB

Building Kynthera: Turning AI Agent Failures into Recovery Intelligence with HydraDB

How I built a graph-powered recovery workspace, worked through GraphBLAS and Rust deployment challenges, and connected an open-source HydraDB instance to a live application.

By Jayeoba Tunmise Paul (Jayking)

Built for the HydraDB Hackathon | Track 1: The Self-Improving Graph


1. The Problem That Made Me Build Kynthera

One thing I've noticed while building software and working with AI-powered systems is how often we solve the same problems more than once.

A dependency upgrade breaks a build. An API request starts failing. A service crashes unexpectedly. An authentication configuration stops working.

We investigate the issue, read through logs, experiment with possible fixes, and eventually find something that works.

Then we move on.

The problem is that the knowledge gained during that investigation often stays in a terminal session, a temporary conversation, or someone's memory.

When a similar failure happens again, another developer or AI agent may have to repeat much of the investigation.

That got me thinking:

What if software systems could remember their failures, understand which recovery strategies worked, and use that knowledge to make better recommendations next time?

That question became Kynthera.

Kynthera is a recovery intelligence workspace designed to turn incidents, recovery strategies, and recorded outcomes into reusable operational knowledge.

The idea is simple:

Failure is data. Recovery is intelligence.

Instead of treating an incident as something to fix and forget, Kynthera treats it as an opportunity to improve future recovery decisions.


2. Introducing Kynthera

Kynthera is a web application built around a recovery knowledge graph.

It gives developers and teams a place to record failures, create recovery strategies, connect strategies to incidents, and record whether an attempted recovery succeeded.

Over time, those records can be used to recommend relevant recovery strategies when similar incidents occur.

The application includes:

  • Incident capture: Record the failure title, signature, environment, affected service, severity, and description.
  • Recovery strategy library: Save possible recovery approaches and the failure patterns they address.
  • Recovery recommendations: Rank existing strategies against an incident using similarity, environment, and historical outcomes.
  • Feedback recording: Track whether a recovery strategy succeeded or failed.
  • Knowledge graph: Represent connections between incidents and the strategies used to recover from them.
  • Replay lab: Re-evaluate recorded incidents against the current strategy-ranking logic.
  • Recovery dashboard: Explore incidents, strategies, and accumulated recovery evidence.

The goal isn't to automate every possible fix immediately.

It's to build a reliable foundation where recovery knowledge can be captured, connected, evaluated, and reused.

Live application: https://kynthera.vercel.app

GitHub: https://github.com/paulos-ui/Kynthera

Demo walkthrough: https://www.loom.com/share/78cb0770200c4e6f808d8d8764f489b2


3. Why I Chose HydraDB

When I started designing Kynthera, I considered how recovery information should be stored.

A traditional database can store incidents and strategies in separate tables, with relationships represented through identifiers.

That approach works, but the central idea behind Kynthera is about connections.

An incident may resemble several previous incidents.

Multiple incidents may point toward the same recovery strategy.

A strategy may succeed in one environment but fail in another.

As more outcomes are recorded, the useful part of the data becomes the relationships between these records.

That's where HydraDB became interesting to me.

I wanted to explore how an open-source graph database could support a product that learns from recorded recovery history.

In Kynthera, HydraDB provides the graph storage layer for incidents, strategies, and events.

The application creates graph records and connects incidents to recovery strategies through relationships such as:

(i)-[:RECOVERED_WITH]->(r)
Enter fullscreen mode Exit fullscreen mode

Here:

  • i represents an incident.
  • r represents a recovery strategy.
  • RECOVERED_WITH records the relationship between them.

This is useful because recovery knowledge is not just a collection of independent documents.

It's a network of experiences and outcomes.

For the initial implementation, I use HydraDB primarily as the graph-backed persistence and relationship layer. The recommendation scoring itself currently runs in the application.

That separation is important: HydraDB stores the connected knowledge, while Kynthera's ranking engine determines how existing recovery evidence should influence suggestions.


4. How Kynthera Works

The recovery workflow follows a straightforward cycle.

Step 1: Capture an incident

A developer opens the Kynthera console and records a failure.

For example:

Incident: Build fails after dependency upgrade

Failure signature: npm ERESOLVE react peer dependency

Environment: Node.js

Service: Frontend application

Severity: Medium

Description: An application fails to install dependencies after upgrading a package because of incompatible peer dependency requirements.

The incident becomes a record Kynthera can reference later.

Step 2: Create recovery strategies

A developer can create strategies for resolving similar failures.

For example, a strategy might involve reviewing conflicting dependency versions, checking compatibility requirements, and selecting a compatible package combination.

Rather than keeping that knowledge in a temporary troubleshooting conversation, Kynthera stores the strategy as a reusable record.

Step 3: Recommend relevant strategies

When an incident is reviewed, Kynthera compares it against known recovery strategies.

The current ranking logic considers three factors:

  1. Similarity between failure signatures.
  2. Whether the runtime environments match.
  3. Historical success and failure feedback.

The engine combines these signals into a recommendation score.

Step 4: Record the outcome

After trying a strategy, the user records whether it worked.

That feedback updates the strategy's success or failure counts.

The next time a related incident is evaluated, the updated history contributes to its score.

Step 5: Replay previous incidents

The replay lab evaluates previously recorded successful incidents using the current ranking logic.

This helps developers inspect whether the strategies ranked highest are consistent with recorded recovery choices.

It provides a way to examine the system's behavior as the knowledge base grows.

Importantly, this is currently a descriptive replay over recorded data, not a controlled evaluation against an independent test dataset.


5. The Technical Architecture

Kynthera uses a fairly focused stack.

Layer Technology
Frontend Next.js, React, TypeScript
Deployment Vercel
Authentication Google OAuth through NextAuth
Graph database Open-source HydraDB
Local development Ubuntu on WSL
Cloud graph hosting Render
Graph access HydraDB HTTP interface and Bolt adapter
Recovery logic Application-level ranking and feedback engine

The application is divided into three main parts.

Frontend workspace

The frontend gives users a place to capture incidents, manage recovery strategies, explore graph relationships, and review recommendations.

I wanted it to feel like a focused engineering tool rather than a generic analytics dashboard.

The interface follows a dark visual direction with bright green accents, matching the concept of an intelligence workspace.

Application backend

Next.js server routes handle the application's operations.

They read and update incident records, evaluate recovery strategies, record feedback, and communicate with the selected storage backend.

Authentication protects the workspace, and records include an owner identifier so data can be separated by account.

This is a starting isolation model, not a claim that the system has completed a production security audit.

HydraDB graph layer

HydraDB stores the connected records.

The adapter supports creating and retrieving graph entities, along with relationships linking incidents and recovery strategies.

It also preserves Kynthera's application-generated identifiers alongside the graph's native identifiers.

This keeps the application's internal record references consistent while using HydraDB as the underlying graph engine.


6. Building the Recommendation Engine

One of the most important design decisions was figuring out how the system should recommend recovery strategies.

I didn't want to start with an opaque model that generated suggestions without showing why they were selected.

Instead, I implemented a transparent ranking mechanism.

The current engine calculates a weighted score from three signals.

Failure signature similarity: 54%

The system tokenizes the incident and strategy signatures, then measures the overlap between their meaningful terms.

For example, an incident containing ERESOLVE, react, and dependency would have a stronger signature match with a strategy addressing a React dependency conflict than one addressing an unrelated database timeout.

Environment match: 20%

Recovery advice can depend heavily on the runtime.

A Node.js dependency issue and a Python package-resolution problem might look similar at a high level, but their solutions can differ.

The engine gives additional weight when the incident and strategy share the same environment.

Historical outcome quality: 26%

The engine also considers recorded successes and failures.

Its current outcome-quality calculation uses:

(successes + 1) / (successes + failures + 2)
Enter fullscreen mode Exit fullscreen mode

This avoids giving an untested strategy an extreme score simply because it has no recorded failures.

The overall score is calculated as:

Score = 100 × (
  0.54 × signature_similarity
  + 0.20 × environment_match
  + 0.26 × outcome_quality
)
Enter fullscreen mode Exit fullscreen mode

The resulting strategies are ordered by score.

Kynthera also displays reasons behind the ranking, including signature similarity, runtime match, and historical feedback.

I chose this approach because it is understandable and testable.

As the product evolves, the ranking formula can be compared against other approaches using a stronger evaluation dataset.

The current weights are design choices, not empirically optimized coefficients.


7. The Hardest Part Was Getting HydraDB Running

Building the interface was only one part of this project.

The more interesting challenge was getting the open-source HydraDB engine working in my development environment and connecting it to the application.

I was developing on Windows using Ubuntu through WSL.

At first, I expected to install the required dependencies, build the database engine, start a node, and connect the application.

In practice, several problems appeared along the way.

Challenge 1: Missing LAGraph dependencies

One of the early problems involved LAGraph, a graph algorithms library associated with the GraphBLAS ecosystem.

The build process needed compatible dependencies to be available in the expected locations.

When they weren't, the build couldn't complete cleanly.

I had to investigate the dependency requirements and how the relevant native libraries were being discovered during compilation.

The lesson here was that a Rust-based project can still depend on a substantial native C/C++ ecosystem.

Getting the Rust toolchain installed is not always enough.

Challenge 2: GraphBLAS configuration

GraphBLAS was another important part of the setup.

I encountered configuration and library-discovery issues while preparing the build environment.

This required checking dependency paths and making sure the native libraries and headers were available to the build process.

The underlying problem wasn't related to Kynthera's UI or business logic.

It came from the database engine's native build dependencies.

This was a useful reminder that infrastructure work often involves understanding multiple layers of a software stack.

Challenge 3: Rust linker failures

Once I got further into the build process, I ran into Rust linking problems.

The compiler could progress through parts of the project, but linking the final executable depended on native libraries being properly configured.

I worked through the errors, adjusted the local environment, and continued rebuilding until I could start the HydraDB graph node successfully.

That moment was an important milestone.

Instead of relying on a hypothetical database integration, I had a running open-source graph engine on my machine.

Challenge 4: Connecting the graph node to Kynthera

After starting HydraDB locally, the next problem was connectivity.

Kynthera's frontend was already deployed on Vercel, but the graph database was running inside my local WSL environment.

A cloud-hosted application cannot simply connect to a database listening on localhost inside my laptop.

I initially used a tunnel to expose the local service for testing.

I also had to work through authentication and connection configuration so the application's backend could send graph queries successfully.

This stage taught me an important distinction:

A running database is not the same thing as an application that can reliably reach and authenticate with that database.

The endpoint, credentials, request format, and deployment environment all have to agree.


8. Moving HydraDB From My Laptop to Render

After getting the local integration working, another question came up.

What happens when I turn off my laptop?

With HydraDB running inside WSL, the graph service depends on my computer being powered on and the relevant processes continuing to run.

That wasn't ideal for a publicly accessible hackathon demo.

I explored cloud hosting options, including Google Cloud and Oracle Cloud, but ultimately chose Render for a simpler initial deployment.

I deployed the HydraDB graph node using a container image and configured the service to initialize its graph storage directory.

The first deployment failed because the configured local storage directory didn't exist.

After addressing that, I encountered another failure caused by an incorrect executable path in the custom startup command.

Once those issues were corrected, the service started successfully.

The logs showed the graph-node listeners starting, and the deployed service became reachable through its Render URL.

I then tested its health endpoint:

https://kynthera-hydradb.onrender.com/healthz
Enter fullscreen mode Exit fullscreen mode

The response was:

{
  "status": "ok"
}
Enter fullscreen mode Exit fullscreen mode

That confirmed the service was responding to health checks.

It did not, by itself, prove that every graph operation was working, so authenticated query testing was a separate step.

Connecting Render to Vercel

I added a separate set of environment variables for the Render connection while preserving the local configuration.

That allowed Kynthera to select between the local HydraDB node and the Render-hosted instance.

The application now supports an explicit connection target rather than requiring the local configuration to be deleted whenever I want to use the hosted database.

This also makes development easier because I can keep working against a local instance when needed.

An important limitation of the free deployment

The Render deployment is useful for demonstration and development, but the free service has limitations.

It can become idle, and the temporary filesystem isn't durable storage for a graph database.

That means locally stored graph data may be lost when the service restarts or is replaced.

For production use, Kynthera still needs a persistent storage setup and a more complete operational plan.

I consider the current deployment a working infrastructure milestone, not the final production architecture.


9. What Makes the Graph Useful?

The biggest value of HydraDB in Kynthera is the ability to treat recovery knowledge as connected information.

Consider three incidents:

  • A frontend build fails after a dependency upgrade.
  • Another project encounters a similar package conflict.
  • A third service breaks because of incompatible library versions.

These incidents may share a recovery approach even if they occurred in different projects.

A graph can preserve that shared relationship.

As more incidents and strategies are recorded, Kynthera has a foundation for exploring questions such as:

  • Which recovery strategies have been associated with similar failures?
  • Which strategies have a history of successful outcomes?
  • Are certain failures recurring across services?
  • Do some strategies perform differently depending on the environment?
  • Which recorded recoveries should be investigated further?

The current implementation already connects incidents and strategies and uses stored outcome data during recommendation ranking.

More advanced graph traversal, pattern discovery, and automated agent feedback remain opportunities for future development.

This is what attracted me to the hackathon's self-improving graph track.

The objective isn't simply to store more logs.

It's to organize operational experience so that future decisions can make better use of what the system has already encountered.


10. Testing and Measuring Improvement

A self-improving system needs a way to evaluate whether it's actually improving.

I implemented a replay mechanism in Kynthera to examine recorded recovery decisions.

For incidents with successful recorded outcomes, the replay engine ranks available strategies and checks whether its top recommendation matches the strategy associated with the recorded incident.

The application reports:

  • Number of recorded examples evaluated.
  • Top-one strategy match rate.
  • Expected strategy.
  • Predicted strategy.
  • Recommendation score.

This provides an initial way to inspect ranking behavior.

However, I want to be careful about what these numbers mean.

The replay currently uses previously recorded examples, and the ranking engine can use feedback derived from those records.

Therefore, the replay result should not be described as independent predictive accuracy.

It is a descriptive evaluation of the current strategy-ranking behavior.

What's still needed?

The next step is to create a proper benchmark.

I want to compare:

Baseline: Recommendations based primarily on failure-signature similarity.

Improved approach: Recommendations that incorporate environment context and historical recovery outcomes.

To evaluate them fairly, I would need a separate test set, consistent incident examples, and a clear definition of successful recommendation.

Useful metrics would include:

  • Top-one recommendation accuracy.
  • Top-three recommendation recall.
  • Recommendation latency.
  • Recovery success rate.
  • Performance across different failure categories.

I haven't established a verified improvement percentage yet, so I don't want to invent one.

The real achievement at this stage is building the system and the evaluation foundation needed to test that hypothesis.


11. What I Learned From Building Kynthera

This project taught me several things.

Infrastructure can be harder than the interface

A polished frontend is important, but a working database connection depends on much more than UI development.

Native dependencies, runtime configuration, authentication, networking, and deployment all have to work together.

Explicit reasoning is valuable

I deliberately started with a recommendation engine that can explain why it ranks a strategy highly.

That makes debugging easier and gives developers a way to challenge or improve its decisions.

Feedback needs context

A successful recovery strategy isn't automatically the best strategy for every future incident.

Environment differences, failure signatures, and the amount of recorded evidence matter.

A green health check isn't enough

Seeing a service respond to /healthz is useful, but it isn't a substitute for testing authenticated queries, reading and writing graph records, and confirming persistence.

Self-improvement must be demonstrated

It's tempting to describe any feedback-based system as self-improving.

But genuine improvement should be backed by repeatable measurements.

Kynthera provides a starting point for that process. The next challenge is proving how much the accumulated evidence improves recovery recommendations.


12. What's Next for Kynthera?

There are several directions I want to explore.

1. Persistent graph storage

Move beyond temporary free-tier storage so recovery knowledge survives restarts and deployments.

2. Better evaluation

Build a larger set of realistic incidents and compare recommendation approaches against a controlled baseline.

3. Richer graph relationships

Explore connections between environments, services, failure categories, incidents, and recovery outcomes.

4. Agent integrations

Allow AI agents to submit structured incident reports and retrieve relevant recovery knowledge through an API.

5. Automated evidence collection

Move beyond user-reported outcomes toward recoveries validated through tests, health checks, or execution evidence.

6. Safer recovery workflows

Eventually support carefully controlled recovery execution with approvals, clear permissions, and audit records.

The long-term idea is to move from a workspace that remembers failures toward infrastructure that helps automated systems recover more intelligently.


13. Final Thoughts

I started Kynthera with a simple observation.

Developers and AI agents spend a lot of time solving problems that have already been encountered somewhere before.

The missing piece is often not the solution itself.

It's the ability to preserve the experience, connect it to related failures, and retrieve it at the right moment.

Building Kynthera with HydraDB gave me an opportunity to explore that idea through a real application.

It also pushed me outside the comfort of normal frontend development.

I had to work through native graph dependencies, Rust linking errors, service authentication, deployment configuration, and the challenges of making a locally hosted database accessible to a cloud application.

Those problems were frustrating at times, but they were also the most valuable part of the experience.

Kynthera is still an early implementation.

The recommendation engine is intentionally simple, the replay system needs stronger independent evaluation, and the hosted database requires durable storage before production use.

But the core concept is working as a foundation: capture incidents, connect them to recovery strategies, record outcomes, and use that history to inform future recommendations.

That's what I wanted to build for the HydraDB Hackathon.

Failure is data. Recovery is intelligence.

And I believe AI systems should get better at recovering from failures they've already encountered.


Explore Kynthera

Live application: https://kynthera.vercel.app

GitHub repository: https://github.com/paulos-ui/Kynthera

Demo walkthrough: https://www.loom.com/share/78cb0770200c4e6f808d8d8764f489b2

Original X submission: https://x.com/__Official__1/status/2122293924488208853

Built by Jayeoba Tunmise Paul (Jayking) for the HydraDB Hackathon, using the open-source HydraDB graph engine.

Thanks for reading.

Top comments (1)

Collapse
 
gautamtalksdev profile image
Gautam K •

Solving the same failure twice is a real cost, and a graph of failure, cause and fix is a natural shape for it. The part I'd watch is stale fixes: a recovery that worked for an old version of a dependency can be wrong after an upgrade. Do you store the versions involved alongside each fix, so the workspace can warn when a suggested recovery comes from an older setup? Congrats on shipping it for the hackathon.