Zero Code Execution: Building a Safe Jupyter Notebook Connector for Cognee GraphRAG

In modern data science organizations, Jupyter Notebooks (.ipynb) serve as the primary medium for exploratory data analysis (EDA), model training, hypothesis validation, and architectural experiments. However, standard enterprise knowledge graphs and LLM retrieval pipelines have historically struggled to ingest notebooks safely and meaningfully:

1. Security Vulnerabilities (OWASP LLM02): Ingesting external notebooks that contain untrusted Python or shell magics (!curl, os.system) risks accidental code execution if runtime environments (ipykernel, exec(), eval()) are triggered during parsing.
2. Noise & Bloat: A single uncleaned notebook can contain tens of megabytes of raw Base64 PNG plots, ANSI color codes from failed stack traces, and verbose progress bar outputs.
3. Stale Knowledge & Orphaned Entities: When a data scientist deletes or refactors a notebook, stale entity nodes and outdated embeddings often remain indefinitely in vector and graph storage.

As part of the WeMakeDevs × Cognee Mergetober Hackathon, I engineered and submitted the official Jupyter Notebook data-source connector (cognee-community-connector-jupyter) for Cognee.

In this article, I will break down how we solved these challenges through zero code execution passive parsing, pre-ingestion secret redaction, and Cognee’s native forget-on-delete graph lifecycle.

---

Architecture: From Raw .ipynb to Knowledge Graph

Here is the high-level ingestion pipeline from disk to Cognee's GraphRAG engine:

flowchart TD
    A[".ipynb File on Disk"] --> B["Passive JSON Parser (nbformat v3/v4)"]
    B --> C["Cell Content Extractor"]

    subgraph Sanitization ["Pre-Ingestion Sanitization"]
        C --> D["Markdown Cells (Structured Prose)"]
        C --> E["Code Cells (Fenced Code Blocks)"]
        C --> F["Outputs (stdout / text/plain)"]
        F --> G["Noise Filter (Strip ANSI, Drop Base64 Images)"]
        D & E & G --> H["Regex Redactor (Bearer, API Keys, PATs)"]
    end

    H --> I["dlt Document Generator (cognee_document_source='jupyter')"]
    I --> J["Cognee Ingestion Engine"]

    subgraph Cognify ["Cognify Graph Processing"]
        J --> K["Entity & Relationship Extraction (LLM)"]
        K --> L["Knowledge Graph (NetworkX / FalkorDB)"]
        K --> M["Vector Store (Qdrant / LanceDB)"]
    end

    J --> N["write_disposition='replace' (Forget-on-Delete Orphan Cleanup)"]

---

1. Zero Code Execution: Passive JSON AST Parsing

A critical security requirement when building data pipelines for AI memory is never executing untrusted code. Many naive notebook parsers inadvertently import kernel execution utilities or evaluate expressions.

Under the hood, every .ipynb file is simply a structured JSON document conforming to the Jupyter nbformat schema. Our connector treats notebooks strictly as passive data using Python's standard library:

import json
from pathlib import Path

def parse_notebook_statically(file_path: Path) -> dict:
    """Statically parse .ipynb as raw JSON with zero kernel interaction."""
    with open(file_path, "r", encoding="utf-8") as f:
        nb_data = json.load(f)

    # Validate nbformat version
    nbformat_ver = nb_data.get("nbformat", 4)
    cells = nb_data.get("cells", [])
    return {"version": nbformat_ver, "cells": cells}

By decoupling extraction from ipykernel, the pipeline can ingest thousands of notebooks across legacy versions (nbformat v3 and v4) with zero CPU execution risk and zero dependency overhead.

---

2. Noise Suppression & Output Filtering

Raw notebook outputs are notoriously noisy. To keep LLM context windows efficient and prevent embedding pollution:

1. Base64 Image Suppression: Base64-encoded PNG/JPEG plots (image/png, image/jpeg) are stripped entirely from outputs.
2. ANSI Color Stripping: Terminal escape sequences generated by Python tracebacks (\u001b[0;31m) are removed with compiled regex patterns.
3. Character Caps: Verbose stdout logs are capped to configurable lengths (max_output_chars=2000) with explicit truncation markers.

import re

ANSI_REGEX = re.compile(r"\x1B(?:[@-Z\\-_]|\[[0-?]*[ -/]*[@-~])")

def clean_traceback_output(text: str) -> str:
    """Remove ANSI terminal color escape sequences from traceback strings."""
    return ANSI_REGEX.sub("", text)

---

3. Pre-Ingestion Credential & Secret Redaction

Data scientists frequently forget hardcoded test credentials in scratch cells before saving notebooks. Emitting unredacted API tokens into shared vector databases or external LLM inference endpoints violates enterprise privacy boundaries.

The connector runs a compiled regex redaction pipeline over all Markdown prose, code blocks, and output buffers before records leave the local connector sandbox:

• OpenAI Keys: sk-[A-Za-z0-9]{20,}
• GitHub PATs: ghp_[A-Za-z0-9]{36}
• AWS Access Keys: AKIA[0-9A-Z]{16}
• Bearer Tokens: Bearer [A-Za-z0-9\-\._~\+\/]+=*

REDACTION_RULES = [
    (re.compile(r"Bearer\s+[A-Za-z0-9\-._~+/]+=*", re.IGNORECASE), "Bearer [REDACTED_TOKEN]"),
    (re.compile(r"ghp_[A-Za-z0-9]{36}"), "[REDACTED_GITHUB_PAT]"),
    (re.compile(r"sk-[A-Za-z0-9]{20,}"), "[REDACTED_API_KEY]"),
    (re.compile(r"AKIA[0-9A-Z]{16}"), "[REDACTED_AWS_KEY]"),
]

def redact_secrets(content: str) -> str:
    for pattern, replacement in REDACTION_RULES:
        content = pattern.sub(replacement, content)
    return content

---

4. The Cognee Graph Invariant: "Forget-on-Delete"

In GraphRAG architectures, handling deletions and moves is just as important as handling additions. If a team removes an obsolete experimentation notebook, its graph nodes shouldn't mislead the LLM.

To solve this, the connector integrates directly with Cognee's internal DOCUMENT_SOURCE_ATTR mechanism:

from cognee.tasks.ingestion.dlt_utils import DOCUMENT_SOURCE_ATTR

# 1. Flag the source for document routing
setattr(source, DOCUMENT_SOURCE_ATTR, "jupyter")

# 2. Use full snapshot replacement disposition
@dlt.resource(name="notebooks", write_disposition="replace")
def notebooks_resource():
    for notebook in discover_and_parse_notebooks():
        yield notebook

Why this matters:

1. Bypasses Relational Flattening: DOCUMENT_SOURCE_ATTR = "jupyter" signals to Cognee that incoming items are rich documents rather than relational database tables, automatically routing them to the Cognify entity-extraction pipeline.
2. Automated Graph Pruning: Setting write_disposition="replace" allows Cognee's background orphan cleanup job to compute differences between sync snapshots and automatically prune removed notebooks from both vector collections and the FalkorDB/NetworkX knowledge graph.
3. Content Deduplication: Unchanged notebooks retain identical deterministic data_id content hashes, skipping redundant LLM inference calls on subsequent runs.

---

In Action: Ingesting Notebooks and Querying the Knowledge Graph

Using the connector in a Cognee pipeline takes just a few lines of asynchronous Python:

import asyncio
import cognee
from cognee_community_connector_jupyter import jupyter_source

async def main():
    # 1. Initialize connector pointing to data science repositories
    source = jupyter_source(
        workspace_dir="./analytics_workspace",
        include_outputs=True,
        max_output_chars=2000,
        redact_secrets=True,
    )

    # 2. Add source to Cognee staging and cognify into knowledge graph
    print("Ingesting notebooks...")
    await cognee.add(source)

    print("Extracting entities and building graph relationships...")
    await cognee.cognify()

    # 3. Query the unified notebook knowledge graph
    query = "What data preprocessing steps and evaluation metrics were used for customer churn?"
    print(f"\nQuerying: '{query}'")
    results = await cognee.search(query)

    for idx, match in enumerate(results, 1):
        print(f"\n[Result {idx}]:\n{match}")

if __name__ == "__main__":
    asyncio.run(main())

When Cognee processes the notebooks, it identifies concepts like RandomForestClassifier, AUC-ROC, and data_leakage_mitigation, creating cross-notebook edges that connect experimental hypotheses with quantitative findings.

---

Verification & Maintainer Standards

Following the Cognee core maintainer review standards:

• 100% Offline Testing: 27 unit tests pass offline with zero external network or kernel dependencies (tests/test_source.py, tests/test_parser.py, tests/test_redaction.py).
• Dependency Hygiene: Clean separation using modern pyproject.toml configuration with flexible bounds (cognee>=1.6.3,<2).
• DCO Certified: Verified Developer Certificate of Origin (git commit -s).

---

Conclusion & What's Next

By combining passive JSON parsing, pre-ingestion redaction, and Cognee's GraphRAG lifecycle, data science teams can safely turn their institutional notebook repositories into interactive, searchable memory.

You can inspect the connector and open PR under the Cognee community repository:

• Pull Request: topoteretes/cognee-community#373
• Cognee Core: topoteretes/cognee

Have questions about GraphRAG, Cognee connector architecture, or notebook sanitization? Drop a comment below or connect on GitHub!