Introduction
This article kicks off a series where I share lessons learned as an AI leader at a FinTech. Over the past few months, I’ve been involved in building an Intelligent Document Processing (IDP) product focused on processing payment receipts. Our goal was to extract structured information from PDFs and images in a reliable and scalable way.
To speed up development, we relied on AWS Bedrock Data Automation (BDA). The tool delivered some good results, but it also revealed unexpected limitations. Below, I share three challenges we faced—without overlooking the positives: acknowledging BDA’s strengths while exposing the pitfalls we encountered in real-world use.
Why We Chose BDA
BDA is an AWS service designed to automate data extraction from documents and images. A key feature for our use case is the creation of blueprints, artifacts that define how data should be extracted, normalized, and formatted. BDA offers two approaches for creating blueprints:
- Manual (via JSON Schema): developers explicitly define the structure.
- Automatic (via prompt): describe in natural language which fields to extract and how to handle them.
The second option seemed irresistible. Almost instantly, we had blueprints running with promising outputs for payment receipt extraction. That convenience—combined with the frequent updates AWS pushed during development—convinced us to go down this path.
Everything seemed fine at first. The functionality was live in production, and extractions were performing as expected. But eventually, we stumbled into a cost anomaly, and after a deeper analysis of the system’s behavior, we uncovered issues that had gone unnoticed during development.
Pitfall #1 – Blueprints Created via Prompt Are Always Document Type
One thing that took us a while to realize: every blueprint created via prompt is classified as Document, even if the base content is actually an image. During tests and development, this characteristic went unnoticed since it didn’t seem to affect extraction accuracy.
Later on, however, it came back to bite us—this time as a cost anomaly.
Pitfall #2 – Wrong Modality and Financial Impact
BDA has a feature called modality routing, which defines whether a file is processed as a document or an image. While it might sound like a minor technical detail, this setting directly affects costs:
- Document: USD 0.040 per page
- Image: USD 0.005 per image
Since all our blueprints were created via prompt, everything was treated as Document. During early testing, we didn’t notice a difference. But by August, reality hit: almost USD 1,000 in unexpected extra charges.
After investigating, we understood the reason: auto-generated blueprints don’t respect the image modality. The fix was to recreate them manually using JSON Schema, properly configuring Document vs. Image, and enabling routing.
I’ll own part of this: we could have validated the behavior earlier. But the critique of AWS stands—the documentation does not clearly highlight this limitation. That lack of clarity can lead to costly mistakes.
Pitfall #3 – Average Latency of 30 Seconds
The third issue was latency. Each BDA process took about 30 seconds on average.
From an architectural perspective, this was manageable. We built the system around asynchronous, event-driven flows, so the backend could handle the delay. But from the user’s perspective, 30 seconds feels painfully slow.
When a customer uploads a receipt, waiting half a minute for results undermines the experience. To address this, we adapted the application, but naturally we compared alternatives. In some cases, pipelines using Textract + LLMs or OCR + post-processing delivered similar results with lower latency and comparable costs.
The Positive Side of BDA
Despite these challenges, BDA did provide real benefits:
- Abstracts complexity: we didn’t have to build an IDP pipeline from scratch.
- Fast initial delivery: we quickly deployed a working prototype.
- Lower barrier to entry: smaller teams can experiment with IDP without heavy infrastructure investment.
In our case, BDA acted as an initial accelerator. Without it, launching the first version would have taken longer. The key takeaway is knowing when it makes sense to rely on it and when alternative approaches may be more effective.
Conclusion
After three months with AWS Bedrock Data Automation, these were our main takeaways:
- Blueprints created via prompt are always Document type.
- Incorrect modality settings can lead to major cost overruns.
- 30-second latency hurts the user experience.
These issues are not clearly stated in the documentation. Only real-world usage exposed them. For me, this reinforces an important lesson: AI leadership requires not only enthusiasm for new solutions but also critical thinking to weigh trade-offs.
BDA has value, but it’s not a universal solution. Knowing when to adopt it—and when to replace it—is part of technical maturity.
In upcoming articles, I plan to share more behind-the-scenes experiences from AI projects in the financial sector. Hopefully, these insights help others avoid similar pitfalls and make more informed decisions.
Top comments (0)