Your README is no longer just documentation.

It is no longer news how much our lives as developers, engineers, tinkerers, artisans, and coders have changed since the advent of AI. For me, AI has been a great leveler in my journey as an engineer. It became the pair programmer I always wished I had. Had I enjoyed the luxury of an LLM back in 2021, I genuinely believe one of my contract engagements would have lasted much longer.

Since then, we've watched the industry evolve rapidly from LLMs to MCP servers, then autonomous agents, loop engineering, and now graph engineering. Every few months, it feels like the conversation changes. More than ever before, it feels as though traditional SEO has taken a back seat. These days, optimizing for AI agents is becoming just as important as optimizing for search engines. I still use Google every day, but LLMs now make up a significant part of my workflow. One thing about change is this: if you refuse to move with it, it has a way of moving without you and leaving you obsolete.

A few weeks ago, while working on a project, I came across Vis.js Network for the first time. I had previously worked with Chart.js and D3.js, especially while building analytics dashboards, which remain some of the most enjoyable parts of my work. Not long after, I needed to transport a gaming chair from one location to my apartment. Instead of opening multiple apps, I simply asked an LLM. Within the conversation itself, it suggested transport options like UberXL and even recommended local delivery services that weren't particularly visible on either the App Store or Google Play. That experience stuck with me.

This write-up was born from a recent project beyond my usual frontend and backend work. I found myself diving deep into graph databases, ontology design, graph reasoning, semantic search, cosine similarity, knowledge graphs, Cypher queries, context graphs, deterministic systems, and probabilistic reasoning. The project aims to reason over fragmented professional data by bringing together a person's information from multiple sources into a unified knowledge graph. From there, the system generates insights about their career, identifies capability gaps, and suggests ways they can improve. It's easily one of the most exciting projects I've worked on, and I'll write much more about it in future posts.

One of the integrations we built was with GitHub. The platform targets engineers and developers, so GitHub repositories naturally became one of our richest sources of professional evidence. We wanted to infer as much as GitHub's OAuth permissions would allow. The results surprised us. Many developers have built genuinely impressive software. Some had solved difficult technical problems. Others had years of engineering experience hidden inside their repositories. Yet they remained almost invisible.

Why?

Because many repositories either had no README at all or had README files that barely described the project. They failed to explain the technologies involved, the architectural decisions made, the problems solved, or the engineering skills the repository was meant to demonstrate. While implementing our inference engine, we discovered something interesting. Among every file in a repository, the one with the highest value for understanding what the project actually does wasn't always the source code.

It was the README.md.

For an embedding-based reasoning engine, a well-written README became a goldmine. It allowed us to infer things that weren't explicitly present in the code itself:

• The business problem being solved
• Architectural decisions
• Technical trade-offs
• Engineering maturity
• Technologies used
• Deployment strategies

And even the author's depth of understanding. It wasn't just documenting software. It was documenting engineering. Repositories with excellent README files dramatically improved the quality of our knowledge graph because they gave context that code alone rarely communicates.

Under the Hood: Designing the Ingestion & Inference Pipeline

Getting from a GitHub repository to these capability claims wasn't simply a matter of fetching files and sending everything to an LLM. The ingestion pipeline itself had to be designed carefully.We needed to retrieve the evidence first, process it efficiently, and only then allow the inference stages to reason over what had been collected. In other words, scoring and inference should never happen while we're still fetching the evidence.

That led us to a pipeline built around Celery chords.

docs.celeryq.dev

There are two chords involved, and each one represents a different synchronisation boundary in the system:

Boundary 1: Pagination Fan-Out & Graph Sync

GitHub already gives us a natural unit of parallelism through pagination. Rather than creating one enormous task responsible for every repository, the repository list is split into pages, and each page becomes its own extraction task.The implementation is surprisingly lightweight:

# How many repos → how many pages
pages = list(range(1, (total_repos // CHUNK_SIZE) + 2))
chunk_count = len(pages)
job.batch_chunk_count = chunk_count
db.session.commit()

_publish(job.user_id, 10, f"Found ~{total_repos} repos. Splitting into {chunk_count} chunks...")

# ── FAN-OUT: one extract task per page ──────────────────────────
extraction_tasks = [
    github_extract_chunk_task.s(job_id, page, profile["login"])
    for page in pages
]

# ── CHORD: run ALL extract tasks in parallel, then ONE callback ──
chord(extraction_tasks)(github_graph_sync_task.s(job_id))

The important part here isn't just that Celery is being used—it's where the parallelism happens. Each extraction task owns one page and operates independently. That allows workers to process multiple pages concurrently. Why not per repository? If a developer has hundreds of repositories, that turns into hundreds of individual tasks, flooding queues with orchestration overhead.Why not one giant task? We lose the ability to process independent pieces of work concurrently.

A page provides the ideal middle ground: it aligns with GitHub's pagination model, bounds the Celery task count, and gives each worker a chunk of work worth processing. A Celery chord is essentially a fan-out followed by a join. The extraction tasks run independently. The chord guarantees a synchronisation point without polling or manual completion flags. Once the group finishes, the callback triggers.

Boundary 2: Parallel Inference Prerequisites

github_graph_sync_task doesn't immediately score what it has found. Instead, it starts a second chord. This time, the fan-out spans two distinct inference prerequisites

chord(
    [
        embed_chunks_task.s(job.user_id, job.id, all_chunk_payloads),
        intel_enrich_knowledge_task.s(job_id, all_unknown_items),
    ]
)(
    chain(
        seed_knowledge_nodes_task.si(),
        extract_claims_from_chunks_task.si(job.user_id, job_id),
        mark_job_complete_task.si(job_id=job_id),
    )
)

Each branch serves a unique responsibility:

1. emb_chks_task constructs the vector representations needed for semantic extraction.
2. enrich_knowledge_task handles knowledge resolution. The goal was not to ask the LLM to resolve everything. Deterministic and cached resolution handled the cheap cases first, with the LLM reserved for cases where the system had exhausted the cheaper sources of evidence. Neither branch needs to wait for the other. However, semantic extraction requires both. The callback chain begins only when both branches complete. The system then seeds the knowledge nodes, extracts capability claims from the evidence, and marks the job complete. This enforces a strict architectural pattern :

Evidence Extraction → Inference → Claims → Scoring

Why the Context Layer Changes Everything

This brings us right back to the README.md.
A README isn't valuable merely because a human might read it and think a repository looks professional. It becomes valuable because it provides a reasoning system with necessary architectural context:

-The source code reveals a PostgreSQL import; the README explains why PostgreSQL was chosen over a document store.

• The source code shows a Redis client; the README details the specific caching strategy and eviction trade-offs.
• The source code displays services and queues; the README connects those components into an architectural narrative.
• That context is what makes raw evidence actionable.

We're entering an era where machines often discover your work before humans do. In 2026, I genuinely believe it's easier for an AI agent to find and understand your GitHub repository than it is for a recruiter or another engineer to stumble upon it organically. Whether AI eventually "solves coding" or replaces software engineering is a debate I'll happily leave to time. But one thing already feels true: if intelligent systems are becoming the first consumers of our work, then our repositories need to be written for both humans and machines.

Your README is no longer just documentation. It is part of your professional identity. The next time you finish a project, spend as much care explaining why it exists, how it works, and what engineering decisions you made as you do writing the code itself. You aren't just helping the next engineer who reads your repository. You might be helping the next AI system understand your expertise. And in this new era, being understood is becoming just as important as being skilled.