For contextual advertising, AWS gives you finished labels and Twelve Labs gives you stronger video models to build on. Amazon Bedrock Data Automation (BDA) splits a video into chapters and tags each one with IAB categories, along with on-screen text, logos and seven moderation classes, so you get targeting labels with no model work. Twelve Labs sells two models: Marengo, which turns video into embeddings, and Pegasus, which turns video into text. Turning their output into IAB codes, brand-safety flags or ad breaks is a pipeline you write and run. Both Twelve Labs models are also sold inside Bedrock, so one AWS account can use both.
What AWS gives you
BDA's standard video output has:
- shots and chapters, with timestamps and a summary for each chapter
- IAB categories per chapter
- a full transcript with speaker labels
- text that appears on screen, with bounding boxes
- logo detection (off by default)
- seven moderation categories, such as violence, alcohol and hate symbols
The IAB list in the BDA docs has 24 tier 1 and 85 tier 2 categories. IAB Content Taxonomy 3.1, the current version, has 704 categories across four tiers, 37 of them at tier 1. If a buyer, SSP or curation deal asks for tier 3 codes or a specific taxonomy version, plan for a mapping step.
AWS also publishes guidance for contextual advertising with generative AI, which builds the same kind of labels from multimodal models you choose.
What Twelve Labs gives you
- Marengo turns video into embeddings, used for search and classification.
- Pegasus turns video into text, for summaries and structured descriptions.
Marengo 2.7 and Pegasus 1.2 became fully managed models in Bedrock in July 2025. Twelve Labs' own API carries newer versions.
Their contextual ad engine tutorial shows what building on these models looks like. Pegasus 1.5 writes structured metadata for each scene. Marengo 3.0 embeddings feed search. The candidate labels are matched against IAB 3.1 rows, and the pipeline picks ad breaks and writes FreeWheel-style key-values. It ships as a reference build with a repo, so the pipeline is yours to run.
How to choose
- You are on AWS and want labels this quarter: start with BDA. Check its category depth against what your partners expect.
- You need to search an archive by what happens on screen, such as every scene at a beach at sunset, or define your own segments: embeddings fit better. That means Marengo, or any multimodal embedding you can run.
- You need audited IAB 3.1 codes at full depth: either route needs a classification step you can measure. Label a few hundred of your own scenes by hand and compare top-1 accuracy before you commit.
A third option
Disclosure: I work on Mixpeek.
Mixpeek indexes video from your own S3 or GCS bucket, scene by scene. It classifies each scene into all 704 IAB Content Taxonomy 3.1 categories through a retriever you can inspect and change. On IAB Tech Lab's open 704-page benchmark, our 3.1 classifier scored 72.7% top-1 and 93.8% top-5. That benchmark is text only, and the write-up includes where it lost. A side-by-side with Twelve Labs is at mixpeek.com/comparisons/mixpeek-vs-twelvelabs.
Top comments (0)