Building a Grafana Connector for Cognee: What I Learned About Syncing Safely

I’ve been building a Grafana connector for Cognee as part of Issue #4779. The implementation is in PR #458, which is open for maintainer review. This post describes the implementation and lessons so far; it is not a claim that the PR is merged or that every Grafana setup is supported.

The idea is simple: make selected dashboard context and human annotations searchable alongside other knowledge in Cognee. The engineering turned out to be less about making HTTP requests and more about deciding what the connector can safely conclude from them.

What the connector reads

The initial connector renders selected dashboards, panel titles and descriptions, and human annotations into documents for Cognee. It does not run dashboard queries or ingest metric values. Identity and selection matter: a dashboard UID is used as the dashboard identity, annotations use their IDs, and each connector resource maps to one separately managed corpus.

Before implementing it, I studied existing Cognee connector patterns and the Postman connector PR #366. That review helped me think about the full lifecycle: extraction, staging, retries, replacement, and cleanup—not only the API call itself.

The central challenge: when is a snapshot complete?

Consider a sync that sees fewer dashboards than last time. A dashboard might really have been deleted. But access could also have changed, pagination could be incomplete, or a request could have failed. If the connector treats every missing item as deleted, a temporary visibility problem can erase good data.

The connector therefore validates the read and its authority before allowing a new snapshot to replace the prior staged view. If it cannot establish a usable snapshot, it fails the extraction so that uncertainty does not become deletion.

flowchart TD
    A[Start sync] --> B[Check account, role, and required permissions]
    B --> C[Read selected dashboard inventory]
    C --> D[Read annotation history]
    D --> E[Recheck authority and dashboard inventory]
    E --> F{All reads and checks complete?}
    F -- No --> G[Fail extraction; do not publish replacement snapshot]
    F -- Yes --> H[Render selected content and stage snapshot]
    H --> I[Cognee processes documents and cleanup]

That leads to the principle I’ll carry into future sync work: absence is meaningful only when the read was complete enough to trust.

Permissions are part of the design

On the tested Grafana release, the organization-wide reads needed for this reconciliation were verified using an organization Admin service account. The connector sends GET requests, but the account role also permits writes. Those are separate facts: read-only request behavior does not mean the credential itself is read-only.

The PR documents this as a limitation and asks maintainers to decide whether to accept the initial requirement or pursue a narrower authority strategy. The connector refuses a Viewer account because a filtered view could be mistaken for a complete inventory and cause false deletion.

This was a useful security lesson: explain both the operations your code performs and the privileges held by the credential that performs them.

Deliberately narrow support

The implementation targets Grafana OSS 12.3.1, organization 1, the default namespace, the v1beta1 dashboard API with classic schema 42, and SQL-backed annotations. Library panels are refused instead of silently omitted. Other Grafana versions, Cloud, organizations, schemas, and annotation stores need separate validation.

The narrow contract makes the current behavior easier to reason about, but it is not a substitute for broader compatibility work. The PR asks for maintainer input on the initial boundary and calls out validation still needed.

Incremental work is not an incremental feed

An unchanged dashboard can reuse cached rendered content, reducing repeated rendering and downstream work. However, this connector still reads complete dashboard inventories and scans the configured annotation history. “Incremental” here describes reuse and processing, not a guarantee that Grafana sends only changed records.

flowchart LR
    A[Full provider inventory] --> B{Previously seen revision?}
    B -- Same --> C[Reuse cached rendered row]
    B -- New or changed --> D[Render selected fields]
    C --> E[Build verified replacement snapshot]
    D --> E
    E --> F[Cognee ingestion and reconciliation]

Making this distinction explicit matters for both performance expectations and correctness: the connector must still verify enough of the source to reconcile removals safely.

Redaction and selected context

Dashboards can contain more than useful prose. The connector uses an allowlist of selected fields and avoids ingesting query targets, datasource settings, variable values, raw options, and secure configuration. It also applies best-effort filtering for credentials and accepts explicit redaction values for known secrets.

Redaction is not a guarantee that every secret in arbitrary prose will be detected. That limitation is documented, and users should review selected content and supply known secret strings explicitly. This reinforced another lesson for me: data minimization is stronger than relying on filters after collecting everything.

How I tested it—and what that does not prove

The work includes tests for identity, annotations, permission changes, incomplete reads, selection changes, redaction, retries, and deletion behavior. I also ran installed-wheel tests with Python 3.11, 3.12, and 3.13, plus lifecycle checks against a disposable Grafana instance and local Cognee stores.

Those results provide evidence for the exercised paths; they do not prove universal correctness. Broader Grafana versions, other storage backends, arbitrary cross-store failure recovery, and every permission or concurrency interleaving remain outside what these checks establish. The PR is still open for review, and maintainer feedback may change the design or scope.

What I’m taking forward

This project changed how I think about connectors. A connector is not just an API wrapper. It is a trust boundary and a data lifecycle: it decides what to collect, how to identify it, when a snapshot is complete, how to handle uncertainty, and when removal is safe.

The hardest part was not fetching dashboards. It was making sure the system does not confuse “I can’t see it right now” with “it no longer exists.” That distinction—and the need to document limits honestly—is the learning I’ll take into the next integration.

The next step is maintainer review of the permission model, support boundary, and remaining validation work in PR #458.