<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Miguel Morente de las Heras</title>
    <description>The latest articles on DEV Community by Miguel Morente de las Heras (@miguel_morentedelasher).</description>
    <link>https://dev.to/miguel_morentedelasher</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4246680%2F6aaa4772-0494-4c6d-b0be-47458b45dcdd.png</url>
      <title>DEV Community: Miguel Morente de las Heras</title>
      <link>https://dev.to/miguel_morentedelasher</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/miguel_morentedelasher"/>
    <language>en</language>
    <item>
      <title>Cite only what you need: building an offline document RAG on AMD ROCm</title>
      <dc:creator>Miguel Morente de las Heras</dc:creator>
      <pubDate>Mon, 16 Nov 2026 00:18:02 +0000</pubDate>
      <link>https://dev.to/miguel_morentedelasher/cite-only-what-you-need-building-an-offline-document-rag-on-amd-rocm-o6a</link>
      <guid>https://dev.to/miguel_morentedelasher/cite-only-what-you-need-building-an-offline-document-rag-on-amd-rocm-o6a</guid>
      <description>&lt;p&gt;&lt;em&gt;Part of my work in the &lt;a href="https://lablab.build/ai-hackathons/amd-lablab-ai-academy-challenge" rel="noopener noreferrer"&gt;AMD AI League&lt;/a&gt; by lablab.ai and AMD. #AMD #lablab&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Most RAG demos answer a question and paste a list of "sources" underneath. That list is usually padding: the top five chunks, whether they helped or not. For the AMD AI League's RAG challenge the grading was stricter. You return the value and &lt;strong&gt;exactly&lt;/strong&gt; the files that value needs: no more, no less. On top of that the corpus is messy company data and the whole system has to run offline on an AMD GPU.&lt;/p&gt;

&lt;p&gt;These are the design decisions that mattered.&lt;/p&gt;

&lt;h2&gt;
  
  
  The corpus is hostile
&lt;/h2&gt;

&lt;p&gt;The input folder mixed PDFs (some scanned, some encrypted), DOCX, PPTX, XLSX, CSV, logs, source code and screenshots. Two rules kept the indexer alive:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Every file gets its own &lt;code&gt;try&lt;/code&gt;.&lt;/strong&gt; One corrupt PDF must never cost you the rest of the corpus. Unreadable, encrypted and unknown files are logged and skipped.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;An encrypted file is never a source.&lt;/strong&gt; If you weren't given the password, you didn't read it, so you can't cite it.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Office files are just zipped XML, so DOCX and PPTX are read straight from their XML with the standard library, tables included. Spreadsheets keep row and column labels, so "Q3 revenue" stays next to its number.&lt;/p&gt;

&lt;h2&gt;
  
  
  One model for text and images
&lt;/h2&gt;

&lt;p&gt;Screenshots and scanned pages carry real answers, such as an error code on a dashboard or a value in a chart. Instead of a separate OCR stack, a single vision-language model (Qwen2.5-VL-7B on ROCm) &lt;strong&gt;transcribes each image into label/value pairs&lt;/strong&gt; at index time. After that, images are searchable text like everything else.&lt;/p&gt;

&lt;h2&gt;
  
  
  Retrieval that respects identifiers
&lt;/h2&gt;

&lt;p&gt;Company data is full of identifiers like &lt;code&gt;ORR-1847&lt;/code&gt; or &lt;code&gt;THERM_ALERT#&lt;/code&gt;. A normal tokenizer splits them and they stop matching. Keeping them whole in BM25 fixed a whole class of misses.&lt;/p&gt;

&lt;p&gt;Some questions need two hops: "the log shows an incident; which release fixed it?" The first hop finds the log line. A second hop follows the identifiers found &lt;strong&gt;next to the question's words&lt;/strong&gt; to the files that define them, such as the bug database.&lt;/p&gt;

&lt;h2&gt;
  
  
  Superseded documents
&lt;/h2&gt;

&lt;p&gt;Real folders contain &lt;code&gt;policy_r1.pdf&lt;/code&gt;, &lt;code&gt;policy_r2.pdf&lt;/code&gt; and a file titled "WITHDRAWN". The indexer labels older revisions and withdrawn files as &lt;strong&gt;SUPERSEDED&lt;/strong&gt;, and the model sees that label. When two files disagree, the current one wins.&lt;/p&gt;

&lt;h2&gt;
  
  
  Citations by necessity
&lt;/h2&gt;

&lt;p&gt;The model answers in JSON with the value, the files it needed and the evidence line. Then code checks the result:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;a file is cited only if it actually &lt;strong&gt;contains&lt;/strong&gt; the answer (or the identifier that leads to it);&lt;/li&gt;
&lt;li&gt;current documents are preferred over superseded ones;&lt;/li&gt;
&lt;li&gt;a second file is added only when it supplies a piece the first one lacks.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;When nothing in the corpus answers the question, the system returns an empty answer with no citations. A confident wrong citation is the worst possible output.&lt;/p&gt;

&lt;h2&gt;
  
  
  Running it on ROCm
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Base image: the official &lt;code&gt;rocm/pytorch&lt;/code&gt;. &lt;code&gt;transformers&lt;/code&gt; is installed with &lt;code&gt;--no-deps&lt;/code&gt; so pip never swaps in a CUDA build of PyTorch.&lt;/li&gt;
&lt;li&gt;The model loads &lt;strong&gt;once&lt;/strong&gt; into a resident server behind a unix socket. Each query process only pays for retrieval plus one generation.&lt;/li&gt;
&lt;li&gt;Indexing and answering run without network access.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Code
&lt;/h2&gt;

&lt;p&gt;Open source (MIT): &lt;a href="https://github.com/automatizacionesmorente-png/amd-ai-league-mini-challenges" rel="noopener noreferrer"&gt;https://github.com/automatizacionesmorente-png/amd-ai-league-mini-challenges&lt;/a&gt; (the &lt;code&gt;rag/&lt;/code&gt; folder).&lt;/p&gt;

&lt;p&gt;Thanks to &lt;strong&gt;AMD&lt;/strong&gt; and &lt;strong&gt;lablab.ai&lt;/strong&gt; for the GPU credits and a challenge that rewards honesty over volume. Questions and critiques are welcome in the comments.&lt;/p&gt;

</description>
      <category>amd</category>
      <category>rocm</category>
      <category>rag</category>
      <category>lablab</category>
    </item>
    <item>
      <title>Three AI containers on AMD ROCm: what I learned building OCR, RAG and a web agent with no API keys</title>
      <dc:creator>Miguel Morente de las Heras</dc:creator>
      <pubDate>Mon, 16 Nov 2026 00:17:20 +0000</pubDate>
      <link>https://dev.to/miguel_morentedelasher/three-ai-containers-on-amd-rocm-what-i-learned-building-ocr-rag-and-a-web-agent-with-no-api-keys-2a3o</link>
      <guid>https://dev.to/miguel_morentedelasher/three-ai-containers-on-amd-rocm-what-i-learned-building-ocr-rag-and-a-web-agent-with-no-api-keys-2a3o</guid>
      <description>&lt;p&gt;&lt;em&gt;Written for the &lt;a href="https://lablab.build/ai-hackathons/amd-lablab-ai-academy-challenge" rel="noopener noreferrer"&gt;AMD AI League&lt;/a&gt; by lablab.ai and AMD. #AMD #lablab #ROCm&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;The AMD AI League sets one hard rule for every challenge: ship a Docker image that runs on AMD GPUs with ROCm, with the model inside the image. You get no API keys, no cloud LLM, and 30 seconds per question. I built three of these containers: an OCR reader for licence plates and road signs, a RAG system that cites the exact files behind each answer, and a web agent that returns one exact value and the page it came from.&lt;/p&gt;

&lt;p&gt;The models are small (4B to 8B). What decides the score is the engineering around them. These five lessons held for all three.&lt;/p&gt;

&lt;h2&gt;
  
  
  1. Load the model once, answer many times
&lt;/h2&gt;

&lt;p&gt;Loading a vision-language model takes tens of seconds, which is most of the 30-second budget. So the container starts a &lt;strong&gt;resident server&lt;/strong&gt; that loads the model onto the GPU once and listens on a local unix socket. Each question runs as a small, fresh process that talks to that server, so no question pays the loading cost.&lt;/p&gt;

&lt;h2&gt;
  
  
  2. Write a valid answer before you think
&lt;/h2&gt;

&lt;p&gt;The first thing each run does is write a valid, empty JSON answer. The real answer overwrites it later. A crash, a timeout or an out-of-memory error then leaves a well-formed "I don't know" behind, never a broken file. An internal deadline stops the work well before the 30 seconds run out.&lt;/p&gt;

&lt;h2&gt;
  
  
  3. Let the model choose, never invent
&lt;/h2&gt;

&lt;p&gt;For the web agent and the RAG system, the model extracts the answer, but &lt;strong&gt;the answer is only accepted if it appears literally in the passage it cites&lt;/strong&gt;. When it doesn't, the system returns an empty answer. That one check removed nearly all of the hallucinations in my tests, and an empty answer scores better than a wrong one.&lt;/p&gt;

&lt;h2&gt;
  
  
  4. Classical retrieval does most of the work
&lt;/h2&gt;

&lt;p&gt;BM25 with stemming, stop words in two languages and a bonus for the requested type of value (a phone number, a date, a version) finds the right passage before the model sees anything. The model then only reads a few hundred tokens, which keeps it fast and makes it harder to fool.&lt;/p&gt;

&lt;h2&gt;
  
  
  5. Always have a plan B without the model
&lt;/h2&gt;

&lt;p&gt;If time runs short, rule-based extractors (regular expressions for phones, emails, prices, dates and versions) answer from the best passage, and only when the words of the question are right next to the value. The answer is a little worse, but it is never empty because of a timeout.&lt;/p&gt;

&lt;h2&gt;
  
  
  ROCm notes
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Start from the official &lt;code&gt;rocm/pytorch&lt;/code&gt; image and install &lt;code&gt;transformers&lt;/code&gt; with &lt;code&gt;--no-deps&lt;/code&gt;. Otherwise pip may replace the ROCm build of PyTorch with a CUDA one.&lt;/li&gt;
&lt;li&gt;Keep the model weights inside the image and check the final image size early, since the league limits it.&lt;/li&gt;
&lt;li&gt;bf16 with greedy decoding gave me stable, repeatable answers.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Code
&lt;/h2&gt;

&lt;p&gt;The OCR and RAG containers are open source: &lt;a href="https://github.com/automatizacionesmorente-png/amd-ai-league-mini-challenges" rel="noopener noreferrer"&gt;https://github.com/automatizacionesmorente-png/amd-ai-league-mini-challenges&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Thanks to &lt;strong&gt;AMD&lt;/strong&gt; and &lt;strong&gt;lablab.ai&lt;/strong&gt; for the GPU credits and the challenges. If you are building on ROCm too, I'd like to hear what worked for you.&lt;/p&gt;

</description>
      <category>amd</category>
      <category>rocm</category>
      <category>ai</category>
      <category>lablab</category>
    </item>
  </channel>
</rss>
