I recently passed the AWS Certified Generative AI Developer – Professional exam while building DocuAI, a document-processing app (upload PDF/DOCX/TXT, get summaries, then search and chat with citations). Building it taught me three things the exam guide only hints at.
1. Make failures visible
My embedding calls hit Voyage AI's free-tier rate limit (3 requests/min). The error was swallowed, and the document was marked completed before indexing ran. It had a summary but no embeddings, so it never showed up in search, and nothing in the UI said why.
The fix: batch the embeddings, retry only on rate-limit errors, and mark the document failed with a message if indexing can't finish. Only write completed after indexing succeeds.
2. Retrieval quality is decided before the model
My DOCX extractor joined paragraph text, so text inside Word tables was never extracted, chunked or embedded. The model couldn't answer questions about it, and no prompt change would fix that. Check your extraction output before you tune prompts.
3. Streaming changes persistence
Chat answers stream over SSE. Saving when the request arrives stores questions without answers. I save each turn (question, answer, sources) inside the generator, after the final event, so it only happens once the answer is complete.
Still on my list
Guardrails against prompt injection, an evaluation set, caching and CI tests. Passing the exam and shipping those are different things.
If you're studying for this cert, build something small alongside it. The problems above showed up on their own.
DocuAI is available as a deployable template: https://patoliya17.gumroad.com/l/ai-document-processor
Top comments (0)