Email is the largest unstructured dataset most products own, and the default approach — dump the whole raw message into a database column — becomes a problem at around a hundred thousand messages.
Split the record
Store the metadata in the database: message id, thread id, participants, subject, timestamps, direction, and a pointer to the body. Store the raw message and the rendered parts in object storage keyed by message id. Listing an inbox then reads small rows, and opening one message reads one object.
Skew the schema
Bodies are large, variable, and rarely queried. Metadata is small, uniform, and queried constantly. Putting them in the same table means every index carries the weight of the part you never search.
Retention has to be per mailbox
Support conversations and agent inboxes have different legal expectations. If retention is a global setting, someone will eventually ask you to delete one conversation and you will have to write a migration instead of a delete.
The detail that makes search work
Index the extracted text, not the HTML. Indexing raw HTML puts tag names in your search results and multiplies the index size for no benefit. Extract once at ingest, store the extracted text next to the metadata, and search that.
Volanea keeps message metadata queryable and bodies addressable, so inbox listing stays fast as a mailbox grows.
Top comments (0)