Originally published on AI Tech Connect.
When prompting a VLM stops being enough: the fine-tune decision for document AI There are two ways to build document AI with vision-language models, and they are different enough to deserve separate guides. The first is to use an off-the-shelf VLM — frontier or open — with a strict output schema, null-preference prompting, confidence gating and a human review queue. We covered that pipeline end to end in our guide to document extraction with VLMs, and for most teams it is the right place to start: no training run, no dataset to label, and a model swap is a config change. This article is the counterpart — what to do when that approach plateaus and you decide to train the model itself. The plateau has a recognisable shape. Your error analysis shows mistakes concentrating on your documents…
Top comments (0)