DEV Community

Cover image for Let AI agents work on huge datasets without transferring them
Marius-Florin Cristian
Marius-Florin Cristian

Posted on Originally published at keibidrop.com

Let AI agents work on huge datasets without transferring them

KeibiDrop connects two machines directly and gives both sides the same folder, encrypted end to end, with no cloud storage in between. kdmcp lets AI agents drive it as an MCP server: the remote folder mounts as a normal path, the agent reads files where they sit, and only the bytes it reads cross the wire. In a measured run, reading 1 MB out of a 256 MB file moved 16.8 MB instead of the whole file, and a 2,184-file dataset was listed 0.53 seconds after pairing. Setup is one config entry, and pairing needs a code from each side, so a single leaked code connects nobody.

Two terminal panes. The left agent offers a dataset folder over KeibiDrop, the right agent lists 2,184 files and reads one off the mount, with a running clock along the bottom.

Measured, with the conditions on the original page:

  • 1 MB read from the middle of a 256 MB file | 16.8 MB on the wire, 6.25% of the file, content matched
  • 40 MB download cut at 8 MB, then resumed | Finished, checksums matched on both sides

Full version, with the limits and the method: Let AI agents work on huge datasets without transferring them

Source, MPL-2.0: github.com/KeibiSoft/KeibiDrop

Top comments (0)