Unstructured Data in a Medallion Architecture: Storing and Managing Images, Diagrams, and Designs

Discover how to extend your data lakehouse to govern and activate unstructured assets like images, CAD models, and diagrams. This guide shows you how to integrate metadata-driven Bronze, Silver, and Gold layers for robust unstructured data management.

This diagram illustrates the adaptation of the Medallion Architecture for unstructured data, showing how files remain intact while undergoing refinement through AI-driven metadata extraction and vectorization across Bronze, Silver, and Gold layers.

Unstructured Data in a Medallion Architecture: How to Store, Govern, and Activate Images, Diagrams, and Design Files

Most unstructured data medallion architecture guidance assumes tidy rows and columns. But a growing share of enterprise value is locked inside unstructured data — product photos, engineering diagrams, building designs, CAD and BIM models, scanned contracts, and architectural renderings. These assets rarely fit a table, yet they carry exactly the context that AI and analytics teams now need.

This guide explains how to extend the Bronze, Silver, and Gold layers to handle unstructured data management without abandoning the governance and lineage that make the medallion pattern work. Within a data lakehouse, the goal is to treat a 200 MB building design the same way you treat a customer record — versioned, described by metadata, quality-checked, and ready to serve.

Why Unstructured Data Breaks the Classic Medallion Model

The unstructured data medallion architecture was designed to progressively refine data from raw to curated. With structured data, each layer transforms columns. With unstructured data, the file itself usually stays intact — a photo is still a photo in Gold — so the refinement happens in the metadata and derived signals that surround it, not in the bytes of the file.

That shift changes three things:

• Storage of record moves to object storage. When you storing images in data lake environments rely on, the binary lives in a bucket or volume; the table only holds a pointer, a checksum, and descriptive metadata.
• Refinement means enrichment, not rewriting. Effective unstructured data management in Silver and Gold adds extracted text, embeddings, classifications, and quality flags rather than reshaping the original asset.
• Governance follows the metadata. Access control, lineage, and retention are enforced on the catalog record that describes each file, which in turn governs the file behind it.

The Three Layers, Adapted for Files

Bronze — Raw Landing for Binaries

Bronze is the immutable landing zone. For unstructured data, that means the original file lands in object storage exactly as received, and a Bronze table captures one row per file with the bare facts:

• A stable object path or URI to the binary (never the binary inside the row).
• A content hash (for deduplication and integrity), file size, and MIME type.
• Source system, ingestion timestamp, and the original filename.

Nothing is interpreted yet. A blurry site photo, a superseded floor plan, and a final rendering all land side by side. Bronze's job is to guarantee you never lose the source of truth and can always replay downstream processing.

Silver — Cleaned, Described, and Enriched

Silver is where unstructured data management turns raw files into usable assets through metadata enrichment. The file stays put; you layer signals on top of it:

• Content extraction: OCR text from scanned drawings, EXIF and geolocation from photos, layer and object lists from CAD/BIM, page text from PDFs.
• Enrichment: AI-generated captions, tags, and object detection for images; entity extraction from contracts; classification of a diagram as "electrical," "structural," or "HVAC."
• Vector embeddings: numeric representations of each image or document that power semantic search and retrieval-augmented generation.
• Quality and conformance: deduplication by content hash, resolution and readability checks, and standardized metadata schemas so a "building design" from any source describes itself the same way.

The result is a Silver table that is fully queryable — you can filter, join, and search across files even though the assets themselves are unstructured.

Gold — Curated Products for Analytics and AI

Gold packages Silver into trusted, purpose-built products. For organizations that storing images in data lake repositories contain, this typically looks like:

• A curated asset catalog — every current building design with its project, discipline, revision, and approval status, ready for a dashboard or portal.
• A vector index of approved documents and diagrams that powers "find similar designs" or an AI assistant that answers questions grounded in your own drawings.
• Aggregated signals — counts of assets by project phase, defect photos by site, or coverage gaps in as-built documentation.

Because Gold is governed and versioned in the unstructured data medallion architecture, an AI model or business user consumes only approved, current assets — not the raw noise sitting in Bronze.

A Worked Example: Building Design and Architecture Files

Consider an engineering and construction organization managing thousands of drawings, renderings, and BIM models across active projects in a data lakehouse.

• Bronze: Every uploaded file — a revised floor plan, a drone photo of the site, a Revit model export — lands in object storage with a hash and a Bronze row recording where it came from and when.
• Silver: OCR pulls title-block text and revision numbers from drawings; AI captions and classifies photos ("facade, north elevation"); BIM metadata is extracted; embeddings are generated so any drawing can be found semantically. Superseded revisions are flagged, and duplicates collapse by hash.
• Gold: A "current approved drawings" product exposes only the latest revision per sheet per project, feeds a search portal, and grounds an AI assistant that answers "show me the latest structural plans for Tower B" with the right file — with full lineage back to the source.

Governance, Lineage, and Cost

The same disciplines that protect structured data apply here, enforced through the catalog:

• Access control on the metadata table gates who can retrieve the underlying file; sensitive designs stay restricted by role.
• Lineage ties every Gold asset back to its Bronze original and the enrichment steps in between, so you can audit how a file was classified or captioned.
• Retention and tiering move cold binaries to cheaper storage while keeping their metadata hot and searchable — you don't pay premium rates to store a five-year-old rendering you rarely open.
• Open table formats (Delta, Iceberg) give ACID transactions and time travel over the metadata layer, so the catalog of your unstructured data estate is as trustworthy as any other table in your data lakehouse.

Implementation Principles

• Store binaries in object storage, pointers in tables. Never embed large files in rows when you storing images in data lake style.
• Refine metadata, preserve the original. The Bronze binary is immutable; metadata enrichment lives alongside it.
• Standardize a metadata contract early so every asset type describes itself consistently.
• Add embeddings in Silver to make unstructured content searchable and AI-ready by default.
• Govern the catalog, and the files follow. Access, lineage, and retention all key off the metadata record in your unstructured data medallion architecture.

Frequently Asked Questions

Where do the actual image or design files get stored in a medallion architecture?

The binary files live in object storage (a data lake bucket or lakehouse volume), not inside database rows. Each medallion table stores a pointer to the file, a content hash, and descriptive metadata. This keeps tables fast and queryable while the large binaries stay in cost-efficient storage.

How does the Silver layer refine unstructured data if the file doesn't change?

For unstructured data management, refinement happens in the metadata and derived signals rather than the file itself. Silver adds extracted text (OCR), AI captions and classifications, vector embeddings, and quality flags — turning an opaque file into something you can filter, join, and search.

How do you make images and diagrams searchable?

Generate vector embeddings for each asset in the Silver layer and store them in a vector index in Gold. This enables semantic search ("find similar facade designs") and retrieval-augmented AI assistants grounded in your own diagrams and drawings.

Can the same governance apply to files and structured data?

Yes. Access control, lineage, and retention are enforced on the catalog record that describes each file. Because that record governs the file behind it, unstructured data assets inherit the same governance model as your structured tables in a data lakehouse.