This post is also on cocoindex.io, with interactive figures. The code uses CocoIndex v1.
An AI agent that uses retrieval answers from its index. It searches the index for context, then writes code or a reply from what it finds. When the index lags behind the source, the agent works from an old copy of your code and docs, and nothing in the retrieved text tells it so.
Live mode keeps a CocoIndex app running after its first pass. It watches the source and applies each change to the index as it happens.
The code below uses the CocoIndex v1 API and follows the code_embedding example on CocoIndex v1.1.1.
Why an agent needs a live index
A person who reads a stale wiki page can often notice the date. An agent gets a chunk of text and a similarity score. If the chunk is out of date, the mistake moves into the agent's output, for example an import that fails, or a header the API no longer accepts.
Catch-up runs exit, live runs keep watching
A CocoIndex app runs in one of two modes. The pipeline code is the same; you choose the mode when you start it.
-
Catch-up (
cocoindex update main): the app scans the source, processes what changed since the last run, syncs the target and exits. Memoized functions skip unchanged work, so the processing cost follows what changed (the run still lists every item to find it). To pick up later changes, you run it again, often from a cron job. -
Live (
cocoindex update -L main): the app does the same catch-up pass, then keeps running. For sources that report their own changes, it reacts to each change without rescanning the source (see Three ways a source goes live).
Both modes cost less than a full rebuild. Live mode also removes the wait between a change and the next run.
A shorter cron interval lowers the lag, but each scheduled run still has to list the whole source to find what changed. In live mode the source tells the app what changed, so a change does not wait for the next listing.
A change re-runs one component, not the app
Live mode needs two things. The app must run with live=True (the -L flag), and a processing component (CocoIndex's unit of work, usually one per source item) must react to changes. The usual way is to pass a live source, one that can report its changes, to coco.mount_each(). It then runs three phases:
- Scan current state. It lists every item and mounts one processing component per item, keyed by the item's key (for files, the path).
- Ready. The initial scan is done and the target rows are synced. A parent can wait for this before it serves queries.
- Watch for changes. A new or modified item re-mounts its component. A deleted item removes its component, and CocoIndex deletes the rows that component declared.
Two layers of memoization keep the cost of a change small:
-
process_fileis memoized per file (@coco.fn(memo=True)). A file event re-runs only that file's function. The other files are not read again. -
process_chunk(inmain.py) callsembed(), which is memoized on the model and the text. When an edited file is re-chunked, the chunks whose text did not change reuse their vectors, so only new text reaches the model.
In a quick test with a 2-file folder and a transform with no model call, an edit reached the output about 0.1 s after the write. With an embedding model, add the time the model takes for the changed chunks.
Make an indexing app live
This is the indexing path of the code_embedding example: it walks a repo, splits each file along its syntax tree, embeds each chunk, and stores the vectors in Postgres with pgvector. The setup (the CodeEmbedding dataclass, the PG_DB and EMBEDDER context keys, and the lifespan that creates them) is in main.py.
```python title=main.py
@coco.fn(memo=True)
async def process_file(
file: FileLike,
table: postgres.TableTarget[CodeEmbedding],
) -> None:
text = await file.read_text()
language = detect_code_language(filename=str(file.file_path.path.name))
chunks = _splitter.split(
text,
chunk_size=1000,
min_chunk_size=300,
chunk_overlap=300,
language=language,
)
id_gen = IdGenerator()
await coco.map(process_chunk, chunks, file.file_path.path, id_gen, table)
@coco.fn
async def app_main(sourcedir: pathlib.Path) -> None:
target_table = await postgres.mount_table_target(
PG_DB,
table_name=TABLE_NAME,
table_schema=await postgres.TableSchema.from_class(
CodeEmbedding,
primary_key=["id"],
),
pg_schema_name=PG_SCHEMA_NAME,
)
target_table.declare_vector_index(column="embedding")
files = localfs.walk_dir(
sourcedir,
recursive=True,
path_matcher=PatternFilePathMatcher(
included_patterns=["**/*.py", "**/*.rs", "**/*.toml", "**/*.md", "**/*.mdx"],
excluded_patterns=["**/.*", "**/target", "**/node_modules"],
),
live=True, # source supports live watch; pass -L to `cocoindex update` to actually run live
)
await coco.mount_each(process_file, files.items(), target_table)
`live=True` on `walk_dir` is the only line that concerns live mode. With it, `files.items()` returns a [`LiveMapView`](https://cocoindex.io/docs/advanced_topics/live_component/#livemapfeed-and-livemapview), which `mount_each()` can scan and then watch. Without `-L` at run time, the same code does one catch-up pass and exits.
### Run it from the CLI
```sh
cocoindex update main # catch-up: scan, sync changes, exit
cocoindex update -L main # live: catch up, then keep watching for edits
-L is short for --live. The live run keeps going until you stop it.
Run it from Python
app.update_blocking(live=True)
# or, inside async code
handle = app.update(live=True)
await handle.result()
Serve the agent while the index stays live
An agent usually queries the index from a server process. The image_search example starts the live update inside the FastAPI lifespan. It waits for the ready signal, so the first query sees a complete index, then leaves the update running in the background:
```python title=api.py
@asynccontextmanager
async def lifespan(app: FastAPI) -> AsyncIterator[None]:
async with coco.runtime():
# (Qdrant client setup omitted.)
# Start a live update; block startup until the initial sweep finishes so
# the collection is queryable, then keep it running in the background.
update_handle = pipeline.app.update(live=True)
async for snap in update_handle.watch():
if snap.status is coco.UpdateStatus.READY:
break
update_task = asyncio.create_task(update_handle.result())
try:
yield
finally:
update_task.cancel()
with contextlib.suppress(asyncio.CancelledError):
await update_task
The agent's search tool and the indexer share one process. Each file saved in the watched checkout, by the agent or by a pull of a teammate's commit, goes through the watcher into the index. Searches see it once its changed chunks are embedded.
## Three ways a source goes live
A source goes live in one of three ways, depending on whether it can report its own changes:
| Mechanism | How changes arrive | Initial scan | Deletes | In v1 |
|---|---|---|---|---|
| `LiveMapView` | The source is scanned once, then watched | Yes | Reported by the watcher | `localfs.walk_dir(live=True)`; [OCI Object Storage](https://cocoindex.io/blogs/changelog-101-107/#source-oci-object-storage-with-live-bucket-watching-v101) fed by bucket events |
| `LiveMapFeed` | A stream of upserts and deletes per key | No | Kafka tombstones; for Iggy, an `is_deletion` predicate you pass | `kafka.topic_as_map()`, [Apache Iggy](https://cocoindex.io/blogs/changelog-101-107/#source-apache-iggy-v107) |
| `coco.auto_refresh` | Your function re-runs on an interval | Yes, each cycle | Rows the function stops declaring | Any source: Google Drive, a Postgres table, a REST API |
### `LiveMapView`: scan the source, then watch it
Use this for sources that have a current state you can list, such as a directory. `walk_dir(live=True)` watches the folder for file events with [watchdog](https://github.com/gorakhargosh/watchdog), as in the example above. It also runs a safety rescan once an hour by default (`rescan_interval=None` turns it off). See the [`localfs` connector docs](https://cocoindex.io/docs/connectors/localfs/#live-file-watching).
### `LiveMapFeed`: consume a stream with no snapshot
Use this for sources that only send changes. The [Kafka connector](https://cocoindex.io/docs/connectors/kafka/) treats a topic as a keyed map: each message is an upsert or a delete for its key.
```python
from confluent_kafka.aio import AIOConsumer
from cocoindex.connectors import kafka
@coco.fn
async def app_main() -> None:
consumer = AIOConsumer({
"bootstrap.servers": "localhost:9092",
"group.id": "my-group",
"enable.auto.commit": "false",
})
items = kafka.topic_as_map(consumer, "my-topic")
await coco.mount_each(process_message, items)
coco.auto_refresh: re-run a function on an interval
Use this when the source has no change events. The v1 Google Drive source, for example, does not support live mode. process_file and DocEmbedding below are the ones from the gdrive_text_embedding example, which runs once in catch-up mode. coco.auto_refresh wraps a function as a live component. It runs the function once, signals ready, then runs it again after each interval:
import datetime
import os
import cocoindex as coco
from cocoindex.connectors import google_drive, postgres
async def sync_drive(table: postgres.TableTarget[DocEmbedding]) -> None:
source = google_drive.GoogleDriveSource(
service_account_credential_path=os.environ["GOOGLE_SERVICE_ACCOUNT_CREDENTIAL"],
root_folder_ids=os.environ["GOOGLE_DRIVE_ROOT_FOLDER_IDS"].split(","),
)
await coco.mount_each(process_file, source.items(), table)
@coco.fn
async def app_main() -> None:
table = await postgres.mount_table_target(
PG_DB,
table_name=TABLE_NAME,
table_schema=await postgres.TableSchema.from_class(DocEmbedding, primary_key=["id"]),
pg_schema_name=PG_SCHEMA_NAME,
)
await coco.mount(
coco.auto_refresh(sync_drive, interval=datetime.timedelta(minutes=1)),
table,
)
Each cycle is reconciled against the previous one:
- A file that
sync_driveno longer sees is not declared, so its rows are deleted. You do not track deletions yourself. -
process_fileis still memoized, so an unchanged file is a memo hit and makes no model call (unless the code ofprocess_filechanged). - Cycles never overlap. The interval is a fixed delay after each cycle ends.
In catch-up mode, auto_refresh runs the function once and exits, so this app also works as a cron job.
Upgrading from the v0 API
The first version of this post used the v0 API. The v1 equivalents:
| v0 | v1 |
|---|---|
refresh_interval= on add_source()
|
coco.auto_refresh(fn, interval=...) |
recent_changes_poll_interval= on the Google Drive source |
No v1 equivalent; use coco.auto_refresh
|
cocoindex.FlowLiveUpdater(flow) |
app.update(live=True) and await handle.result()
|
cocoindex update -L main |
cocoindex update -L main (unchanged) |
Where to go next
- Live mode in the programming guide.
-
Live components, to write your own
process_live()or aLiveMapViewfor a custom source. - The index-codebase tutorial, the full walkthrough of the
code_embeddingexample. - The CocoIndex source on GitHub.
Top comments (1)
Some comments may only be visible to logged-in visitors. Sign in to view all comments.