<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Patrick Kelly</title>
    <description>The latest articles on DEV Community by Patrick Kelly (@patrickjohnkelly).</description>
    <link>https://dev.to/patrickjohnkelly</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4254096%2F95cd31b6-7bc4-4735-b081-4ef0465fb2c0.jpeg</url>
      <title>DEV Community: Patrick Kelly</title>
      <link>https://dev.to/patrickjohnkelly</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/patrickjohnkelly"/>
    <language>en</language>
    <item>
      <title>Exploring Decision Models, Part 1: Putting Gemini Decision 1 to the Test</title>
      <dc:creator>Patrick Kelly</dc:creator>
      <pubDate>Thu, 19 Nov 2026 16:00:34 +0000</pubDate>
      <link>https://dev.to/googleai/exploring-decision-models-part-1-putting-gemini-decision-1-to-the-test-50f0</link>
      <guid>https://dev.to/googleai/exploring-decision-models-part-1-putting-gemini-decision-1-to-the-test-50f0</guid>
      <description>&lt;p&gt;  &lt;iframe src="https://www.youtube.com/embed/ym-uCpmh8sA" width="710" height="399"&gt;
  &lt;/iframe&gt;
&lt;/p&gt;

&lt;p&gt;With all the excitement around large language models over the last few years, it's easy to forget that machine learning has been around a lot longer than chat models, and that other models are built for very different jobs.&lt;/p&gt;

&lt;p&gt;Today, most AI systems leave a lot of efficiency on the table by calling a full LLM just to get a simple true or false response or a basic classification.&lt;/p&gt;

&lt;p&gt;Google recently released &lt;a href="https://ai.google.dev/gemini-api/docs/decisions-api" rel="noopener noreferrer"&gt;Gemini Decision 1&lt;/a&gt;, also known as a 'System 1' model. Instead of generating text token by token like a chat model, it reads your input in one pass and returns one of three structured statistical responses:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;True or false&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;A score&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;A category&lt;/strong&gt; (classification)&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Why decision models are fast and cheap
&lt;/h2&gt;

&lt;p&gt;Because there are no generated text tokens, both the speed and the pricing look very different from a standard LLM.&lt;/p&gt;

&lt;p&gt;You can ask up to &lt;strong&gt;1,000 questions at once&lt;/strong&gt; in a single request, and the extra latency per question is tiny. In my tests, asking 1 question took about &lt;strong&gt;180ms&lt;/strong&gt;, while asking 100 questions at once took about &lt;strong&gt;250ms&lt;/strong&gt;. That makes it practical to run a whole rubric of small checks together and combine them in your own application logic.&lt;/p&gt;

&lt;p&gt;On pricing, Gemini Decision 1 costs &lt;strong&gt;$0.10 per million input tokens&lt;/strong&gt; with &lt;strong&gt;$0 output token cost&lt;/strong&gt; (plus up to &lt;strong&gt;10,000 free requests per day&lt;/strong&gt; on Google's free tier). If you're used to paying steep output-token rates on chat models, that's a big shift.&lt;/p&gt;

&lt;p&gt;When I saw the release, I wanted to see how it actually held up in real workflows. I went through my old coding sessions and public datasets looking for places to either replace a slow LLM call or add a fast check that improves the overall system, ending up with &lt;strong&gt;800 scenarios&lt;/strong&gt; across 24 categories.&lt;/p&gt;

&lt;h2&gt;
  
  
  Experiment 1: model routing
&lt;/h2&gt;

&lt;p&gt;One of the first places I tried it was model routing.&lt;/p&gt;

&lt;p&gt;Instead of running your heaviest frontier model on every single step of an agent session, a router decides whether a specific request can go to a lighter, faster, cheaper model or needs larger reasoning capabilities.&lt;/p&gt;

&lt;p&gt;The catch with LLM-based routers is that you're adding an extra LLM call (more latency and cost) in the hopes of saving more than you just spent. A fast decision model gives you a much cheaper way to grade the prompt up front.&lt;/p&gt;

&lt;p&gt;First, I treated it like a normal LLM and asked one broad, subjective question: &lt;em&gt;"Should this request go to a small model or a large model based on the contents of this prompt?"&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;The result was barely better than a coin flip at &lt;strong&gt;0.72 out of 1&lt;/strong&gt;. Not reliable enough to trust.&lt;/p&gt;

&lt;p&gt;So I split that single broad prompt into four or five narrow, concrete questions in the same request: does this prompt touch authentication or billing? Is it just a styling tweak or a documentation update?&lt;/p&gt;

&lt;p&gt;With that one change, accuracy jumped to &lt;strong&gt;0.88 out of 1&lt;/strong&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Five examples across 800 scenarios
&lt;/h2&gt;

&lt;p&gt;That swing made me curious where else decision models shine and where they fall flat. Across the 24 categories I tested on 800 real sessions, here are five examples running from best to worst:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Customer support team routing (~100% accurate):&lt;/strong&gt; Triaging incoming customer queries to the right team was nearly 100% accurate, averaging &lt;strong&gt;230ms&lt;/strong&gt; per request at &lt;strong&gt;$0.34 per 10,000 requests&lt;/strong&gt;. Basic classification like this is a great fit.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Prompt injection detection (99.7% accurate):&lt;/strong&gt; Checking incoming queries for prompt injection and text-based hijacking hit &lt;strong&gt;99.7%&lt;/strong&gt; accuracy.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Deciding what to remember (88% accurate):&lt;/strong&gt; Scanning recent conversation messages to spot a durable rule, instruction, or fact worth saving to memory hit &lt;strong&gt;88%&lt;/strong&gt; accuracy. Running a full LLM after every turn to check for memory updates gets expensive fast, so this is a useful filter.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Splitting code files for retrieval (plain code won):&lt;/strong&gt; I tried using the model to slice large code files into relevant chunks for retrieval, and it didn't perform well. Plain deterministic code logic was simpler and worked better.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Predicting user pushback (46% accurate):&lt;/strong&gt; This was the test I was most hopeful about. I wanted a persistent check that could spot when I was about to step in and correct my agent before it happened. Instead, it scored &lt;strong&gt;46%&lt;/strong&gt;, worse than a coin flip. That question was simply too broad and subjective for a single pass.&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  Where decision models fit
&lt;/h2&gt;

&lt;p&gt;All of these tests ran on my own sessions and a public dataset, so they aren't exhaustive. Edge cases always exist, and testing on your own data is still the best way to learn where a model holds up.&lt;/p&gt;

&lt;p&gt;The pattern across all 800 scenarios was consistent: break your check down into the narrowest, most specific, and least subjective questions you can, and a decision model like Gemini Decision 1 will be fast, cheap, and accurate. Save broad subjective judgment for full language models, and use plain deterministic code whenever simple code logic already solves the problem.&lt;/p&gt;

&lt;p&gt;This is Part 1 of my exploration of decision models. I'll follow up with more results as I build these checks into real projects.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>googlecloud</category>
      <category>gemini</category>
      <category>agents</category>
    </item>
  </channel>
</rss>
