Building My First RAG System: From Components to Knowledge and Query Pipelines - Part Two

In the last part, we worked up from the foundational problem to the key components of a RAG system. We examined eight components and how they fit together.

These eight components can be grouped into two main pipelines: the knowledge pipeline and the query pipeline.

The Knowledge Pipeline

This pipeline turns raw source information into knowledge the system can store and retrieve later. It runs when information enters or changes and aims to produce retrievable knowledge data. For example, when a new ADR document is created in Notion, we fetch it, parse its structure into a uniform format, convert it into a consistent form, store it, and index it so it can be retrieved efficiently later.

In essence, this pipeline prepares the knowledge that the retrieval part of the RAG system will eventually search. It works much like the write path in a database system.

The Query Pipeline

This pipeline takes a question and finds enough relevant existing knowledge to provide as context for the LLM to answer it. It runs when you ask a question. For example, if you ask, "Why is there a separate requery service besides the payout service?", the system fetches relevant knowledge from the store populated by the knowledge pipeline, ranks and selects the most relevant information, packages it with the original question, and sends it to the LLM for an answer.

In short:

Query pipeline = Retrieval + Augmentation + Generation.

It acts much like the read path of a database system.

Splitting the system into these two halves, the flow looks like:

 ------------------------------------------------------------------
|                      KNOWLEDGE PIPELINE                          |
| Sources → Connectors → Ingestion → Processing → Indexing/Storage |
 ------------------------------------------------------------------
                              |
                              |
                              ↓
 -------------------------------------------------------------------
|                       QUERY PIPELINE                              |
|Question → (Retrieval+ Ranking) → Context Preparation → LLM →Answer|
 -------------------------------------------------------------------

Separating these two parts solidifies a mental model: one prepares the knowledge, while the other consumes it.

A good analogy is search engines. Google does not crawl and process the entire internet every time you search. A background process crawls the web, processes it, and indexes the data. When you search, another part of the system searches those indexes, retrieves matching results, ranks them, and shows you the answers.

A RAG system follows a similar pattern but for your own knowledge and with an LLM at the end.

Let's go deeper into the knowledge pipeline.

Connector

The connector lets us interact with knowledge sources. It's source-specific: we might have a Slack connector, a Notion connector, a Google Drive connector, etc.—like separate database drivers for different systems.

For a Slack connector, for example, we need to:

• authenticate with Slack
• fetch channels
• fetch messages
• fetch threads
• page through results
• interpret Slack-specific IDs and timestamps

The output should remain close to the source, for example:

SourceRecord
- source_type: slack
- source_id: "1712345678.1234"
- raw_content: "We decided to use Redis..."
- source_metadata:
    channel: engineering
    thread_id: ...
    author: ...
    timestamp: ...

Ingestion

While the connector answers,

How do I talk to this source?

The ingestion layer answers:

Which source records should enter or re-enter my knowledge pipeline?

When dealing with source changes, the ingestion layer decides:

• when to fetch all relevant info
• when to fetch only new/changed records
• when to remove irrelevant data (e.g., when source data is deleted)
• when to ignore data (e.g., for unchanged records)

Using the Slack example, suppose you have this in a Slack channel:

#engineering

Timi:  
"We decided to use Redis locking because two workers could update the wallet concurrently."

The flow becomes:

Slack API → SlackConnector → SourceRecord {id, message, channel, author, timestamp, thread} → Ingestion service.

The ingestion service asks:

"Have I seen this Slack message before?"

• If not, it ingests it.
• If yes and unchanged, it ignores.
• If edited, it reprocesses.
• If deleted, cascade the deletion.

---

For the knowledge base we’re building, I’m choosing to keep connector and ingestion logic as separate architectural responsibilities. While combining them seems simpler, as sources grow, two dimensions start changing independently:

• how a particular source is accessed
• how changes are handled once data arrives

Because these have different reasons to change, separating them is important.

Why Separate Connector and Ingestion Responsibilities?

• Core ingestion logic should be source-agnostic. The ingestion engine doesn't need to know connector internals. If it did, we'd have to write new ingestion logic for every connector. Instead, the ingestion flow is largely the same, regardless of connectors:

Receive/discover source changes
      ↓
Determine create/update/delete
      ↓
Submit changes downstream
      ↓
Persist sync/change state if required

• Not all ingestion will be pull-based. Some systems deliver data in different ways:
  • Slack may push updates via webhooks
  • Logseq might use scheduled scans (since it’s file-based)
  • Google Drive could use scheduled API polling

The ingestion layer shouldn’t care about the transport; its job is deciding how new/updated/deleted data affects your knowledge store.

A common ingestion engine can also handle:

• sync state
• idempotency
• retries
• change detection
• deletion propagation
• job scheduling

Conclusion

In this part, we moved from seeing a RAG system as a single linear flow to separating it into two major pipelines: the knowledge pipeline and the query pipeline. This separation offers a clearer architectural picture—one side prepares knowledge, and the other consumes it when a question arrives.

We started breaking down the knowledge pipeline by exploring connector and ingestion layers. The connector is source-specific, while ingestion decides what to do with incoming, changing, or deleted data.

The key architectural takeaway: these are different responsibilities with different reasons to change. As knowledge sources grow, this separation should make the system easier to extend and manage.

In the next part, we’ll continue along the knowledge pipeline and examine what happens after data’s been ingested—how raw source records are processed into a form the rest of the system can use.