<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Datafaro</title>
    <description>The latest articles on DEV Community by Datafaro (@data_faro).</description>
    <link>https://dev.to/data_faro</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4253533%2F7613022b-3c4d-40ef-8307-b32e6b678178.png</url>
      <title>DEV Community: Datafaro</title>
      <link>https://dev.to/data_faro</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/data_faro"/>
    <language>en</language>
    <item>
      <title>Dashboards were built for eyes. Here's how to make analytics work for AI agents</title>
      <dc:creator>Datafaro</dc:creator>
      <pubDate>Thu, 19 Nov 2026 10:20:36 +0000</pubDate>
      <link>https://dev.to/data_faro/dashboards-were-built-for-eyes-heres-how-to-make-analytics-work-for-ai-agents-2bgb</link>
      <guid>https://dev.to/data_faro/dashboards-were-built-for-eyes-heres-how-to-make-analytics-work-for-ai-agents-2bgb</guid>
      <description>&lt;p&gt;Digital analytics is a visual industry. We think in dashboards: line charts with a comparison period, funnels with drop-off bars, a red number that means "look here". A good analyst reads a dashboard in seconds because the design does half the work.&lt;/p&gt;

&lt;p&gt;AI agents can't use any of that. They don't glance at a chart. They read text and numbers, one token at a time, and they have no idea that the red tile matters more than the grey one.&lt;/p&gt;

&lt;p&gt;So when people ask "can we just connect an agent to our analytics?", the honest answer is: &lt;strong&gt;yes, but not to the dashboards.&lt;/strong&gt; You have to rebuild what the dashboard was doing for the human. Here's how I'd approach it.&lt;/p&gt;

&lt;h2&gt;
  
  
  What a dashboard actually does
&lt;/h2&gt;

&lt;p&gt;Before translating anything, it helps to list what a good dashboard gives a human that raw data doesn't:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Definitions.&lt;/strong&gt; "Conversion rate" on this tile means purchases / sessions, last-click, excluding internal traffic. Nobody says this out loud; it's baked into the chart.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Comparison.&lt;/strong&gt; Every number is shown against something: last week, same week last year, target.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Salience.&lt;/strong&gt; Colour, size and position tell you what matters.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Context.&lt;/strong&gt; Annotations: "campaign launched", "tracking broken 3–5 May".&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Guardrails.&lt;/strong&gt; You can only see the cuts someone decided were meaningful.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;An agent pointed at a raw analytics API gets none of this. It gets dimensions, metrics and numbers, and it will happily compute a conversion rate from the wrong denominator and explain it confidently.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 1: Write the semantic layer down
&lt;/h2&gt;

&lt;p&gt;The single most important thing is to make your metric definitions explicit and machine-readable. Something like:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;metrics&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;conversion_rate&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;description&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;Purchases divided by sessions, excluding internal traffic.&lt;/span&gt;
    &lt;span class="na"&gt;numerator&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;purchases&lt;/span&gt;
    &lt;span class="na"&gt;denominator&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;sessions&lt;/span&gt;
    &lt;span class="na"&gt;filters&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s"&gt;traffic_type&lt;/span&gt; &lt;span class="kt"&gt;!=&lt;/span&gt; &lt;span class="s2"&gt;"internal"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="na"&gt;good_direction&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;up&lt;/span&gt;
  &lt;span class="na"&gt;bounce_rate&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;description&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;Share of sessions with a single page view and no engagement events.&lt;/span&gt;
    &lt;span class="na"&gt;good_direction&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;down&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This is the analytics equivalent of "the chart title and the tooltip". Whether you keep it in YAML, dbt's semantic layer or a tool like Cube matters less than the fact that it &lt;strong&gt;exists outside people's heads&lt;/strong&gt;. An agent can only respect a definition it can read.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 2: Give the agent answers, not raw tables
&lt;/h2&gt;

&lt;p&gt;Don't expose "query any table" as the agent's main tool. Expose a few &lt;strong&gt;question-shaped tools&lt;/strong&gt; that return pre-digested answers:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;code&gt;metric_summary(metric, period, compare_to)&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;top_movers(metric, dimension, period)&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;funnel(steps, period)&lt;/code&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;And make each one return what a dashboard would have shown visually, as text:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nl"&gt;"metric"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"conversion_rate"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"period"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"last_7_days"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"value"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;0.021&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"compare_to"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"previous_7_days"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"previous"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;0.026&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"change_pct"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;-19.2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"direction"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"worse"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"summary"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Conversion rate fell 19% week over week (2.6% to 2.1%). The drop is concentrated in mobile traffic from paid social."&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That &lt;code&gt;direction: "worse"&lt;/code&gt; field is the red tile. The &lt;code&gt;summary&lt;/code&gt; is the analyst's first sentence. You're encoding the visual judgement into the data itself.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 3: Always ship a comparison
&lt;/h2&gt;

&lt;p&gt;A number with no comparison is useless to a human and dangerous for an agent, because the agent will invent one ("2.1% is a healthy conversion rate"). Make the comparison part of every response: previous period by default, and year-over-year where seasonality matters. If there's a target, include it.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 4: Put the annotations in the data
&lt;/h2&gt;

&lt;p&gt;Every analytics team has a mental list of "don't trust these dates": tracking outages, consent banner changes, bot spikes, a big campaign. On a dashboard these are annotations. For an agent, they need to be returned alongside the numbers:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="nl"&gt;"caveats"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
  &lt;span class="s2"&gt;"Tracking on checkout was partially broken 2026-05-03 to 2026-05-05."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="s2"&gt;"A paid social campaign started 2026-05-10; traffic mix changed."&lt;/span&gt;
&lt;span class="p"&gt;]&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This one change prevents a large share of confidently wrong answers.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 5: Keep the guardrails
&lt;/h2&gt;

&lt;p&gt;Dashboards quietly prevent bad analysis by only showing meaningful cuts. When you give an agent flexibility, add explicit limits:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Minimum sample sizes.&lt;/strong&gt; Don't report a conversion rate for a segment with 12 sessions; return "not enough data".&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Sensible dimension lists.&lt;/strong&gt; Offer the breakdowns analysts actually use, not all 200 dimensions.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Sampling and thresholding flags.&lt;/strong&gt; If the source applied sampling or privacy thresholds, say so in the response.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Step 6: Keep charts for humans, at the end
&lt;/h2&gt;

&lt;p&gt;None of this means charts go away. Charts are still the best way to show a human what changed. The shift is that the chart becomes the &lt;strong&gt;output&lt;/strong&gt; of the agent's work ("here's the chart of the drop you asked about"), not the &lt;strong&gt;input&lt;/strong&gt; it has to understand.&lt;/p&gt;

&lt;h2&gt;
  
  
  A quick test for your setup
&lt;/h2&gt;

&lt;p&gt;Ask your agent three questions you'd ask a junior analyst:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;"How did conversion do last week?"&lt;/li&gt;
&lt;li&gt;"What drove the change?"&lt;/li&gt;
&lt;li&gt;"Can we trust this number?"&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;If it answers the first with a number and no comparison, the second by listing every dimension, or the third with "yes", you have more translation work to do. If it answers like a good analyst would, you've successfully turned a visual product into one an agent can use.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;I spent years in digital analytics before building data tools for AI agents. If you're wiring agents into analytics stacks, I'm curious which of these steps has been hardest for you.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>analytics</category>
      <category>ai</category>
      <category>dataengineering</category>
      <category>llm</category>
    </item>
    <item>
      <title>From 44 seconds to 6: how I handle 429 rate limits without slowing down</title>
      <dc:creator>Datafaro</dc:creator>
      <pubDate>Thu, 19 Nov 2026 10:20:13 +0000</pubDate>
      <link>https://dev.to/data_faro/from-44-seconds-to-6-how-i-handle-429-rate-limits-without-slowing-down-mbf</link>
      <guid>https://dev.to/data_faro/from-44-seconds-to-6-how-i-handle-429-rate-limits-without-slowing-down-mbf</guid>
      <description>&lt;p&gt;I had a tool that usually answered in a few seconds, and sometimes took 44. For a batch job that's annoying. For a tool called by AI agents, which typically wait around 30 seconds for an answer, it's broken: the agent gets "still running" and the user gets nothing.&lt;/p&gt;

&lt;p&gt;The upstream service was rate-limiting me with &lt;code&gt;HTTP 429 Too Many Requests&lt;/code&gt;. My first instinct was "add better backoff". That was the wrong instinct. Here's what was actually happening, and what fixed it.&lt;/p&gt;

&lt;h2&gt;
  
  
  Where the 44 seconds went
&lt;/h2&gt;

&lt;p&gt;I pulled the run log and lined up timestamps. The time broke down roughly like this:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;~18 s:&lt;/strong&gt; opening the first connection through a proxy session &lt;em&gt;stalled&lt;/em&gt;. Nothing failed; it just hung.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;~12 s:&lt;/strong&gt; after a 429, opening a replacement session was slow again.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;~7.5 s:&lt;/strong&gt; at the very end, the process waited up to 5 seconds for background listeners before exiting.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The actual rate-limit waits were a small part of it. Most of the time was spent &lt;strong&gt;waiting on things that were never going to succeed quickly&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;The sneaky one was the first. I had a 15-second timeout configured, but it was a &lt;em&gt;read&lt;/em&gt; timeout. It limited each read, not the whole request. A connection that trickled bytes slowly, or stalled during setup, could blow straight past it.&lt;/p&gt;

&lt;h2&gt;
  
  
  Fix 1: Total-time budgets, not just read timeouts
&lt;/h2&gt;

&lt;p&gt;Most HTTP clients let you set separate timeouts for connect, read, write and pool. Set all of them, and also wrap the whole operation in an overall deadline:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;asyncio&lt;/span&gt;
&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;httpx&lt;/span&gt;

&lt;span class="no"&gt;TIMEOUT&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;httpx&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Timeout&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;connect&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mf"&gt;5.0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;read&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mf"&gt;7.0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;write&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mf"&gt;5.0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;pool&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mf"&gt;5.0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;fetch&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;httpx&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;AsyncClient&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;url&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;httpx&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Response&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="c1"&gt;# Hard cap on the whole request, including slow trickles.&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;asyncio&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;wait_for&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;url&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="no"&gt;TIMEOUT&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mf"&gt;8.0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;I also capped each &lt;em&gt;query&lt;/em&gt; (which needs several requests) at 16 seconds. If it can't finish, the tool returns a clear error item instead of hanging. A fast, honest failure is better than a slow success the caller never sees.&lt;/p&gt;

&lt;h2&gt;
  
  
  Fix 2: On 429, switch, don't wait
&lt;/h2&gt;

&lt;p&gt;Classic advice for 429 is exponential backoff: wait 1 s, 2 s, 4 s... That's right when &lt;em&gt;you&lt;/em&gt; are the problem and the server needs you to slow down globally. It's wrong when the limit is tied to an identity (an IP or session) and you have another identity available.&lt;/p&gt;

&lt;p&gt;My tool runs through rotating proxy sessions, so the fix was:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Keep a &lt;strong&gt;spare session already open&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;On a 429, a stall or a network error, &lt;strong&gt;switch to the spare immediately&lt;/strong&gt; and start opening a new spare in the background.&lt;/li&gt;
&lt;li&gt;Only back off (briefly) if all sessions are limited.
&lt;/li&gt;
&lt;/ul&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;get_with_failover&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;pool&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;url&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;attempt&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="nb"&gt;range&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;session&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;pool&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;current&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="n"&gt;resp&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nf"&gt;fetch&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;session&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;url&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;resp&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;status_code&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="mi"&gt;429&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
                &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;resp&lt;/span&gt;
        &lt;span class="k"&gt;except&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;httpx&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;TransportError&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;asyncio&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;TimeoutError&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="k"&gt;pass&lt;/span&gt;
        &lt;span class="n"&gt;pool&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;rotate&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;            &lt;span class="c1"&gt;# promote the warm spare&lt;/span&gt;
        &lt;span class="n"&gt;pool&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;warm_spare&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;        &lt;span class="c1"&gt;# start opening the next one, don't await&lt;/span&gt;
    &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;asyncio&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;sleep&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mf"&gt;0.5&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;     &lt;span class="c1"&gt;# everything limited: short pause, then last try&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nf"&gt;fetch&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;pool&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;current&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;url&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The important part is that the replacement is &lt;strong&gt;already warm&lt;/strong&gt;. Switching to a session you still have to open just moves the wait somewhere else.&lt;/p&gt;

&lt;h2&gt;
  
  
  Fix 3: Race the start
&lt;/h2&gt;

&lt;p&gt;Opening the first session was the slowest, least predictable step. So I open &lt;strong&gt;two at startup and use whichever is ready first&lt;/strong&gt;; the other becomes the spare.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;first_ready&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;coros&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;tasks&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;asyncio&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;create_task&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;coros&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="n"&gt;done&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;pending&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;asyncio&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;wait&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;tasks&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;return_when&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;asyncio&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="no"&gt;FIRST_COMPLETED&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;winner&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;done&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;pop&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;result&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;winner&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;pending&lt;/span&gt;  &lt;span class="c1"&gt;# keep the pending one as the spare&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This costs a little extra (one more connection per run) and removes most of the tail.&lt;/p&gt;

&lt;p&gt;Note what I &lt;em&gt;didn't&lt;/em&gt; parallelize: the main requests themselves. In my case the second request needs a token returned by the first, so they're inherently sequential. Racing works on independent steps, not on a dependency chain.&lt;/p&gt;

&lt;h2&gt;
  
  
  Fix 4: Don't let shutdown eat your budget
&lt;/h2&gt;

&lt;p&gt;The last 5–7 seconds were the process waiting for background tasks to finish before exiting. I capped that wait at about 1 second and made closing old sessions happen in the background. Exit now takes around 0.3 s.&lt;/p&gt;

&lt;p&gt;It sounds trivial, but for short tools, teardown can be a surprisingly large share of total time.&lt;/p&gt;

&lt;h2&gt;
  
  
  Results
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;Before&lt;/th&gt;
&lt;th&gt;After (10 test runs)&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Typical run&lt;/td&gt;
&lt;td&gt;a few seconds&lt;/td&gt;
&lt;td&gt;6.5 s (p50)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Slow runs&lt;/td&gt;
&lt;td&gt;up to 40–44 s&lt;/td&gt;
&lt;td&gt;8.0 s (p95)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Average cost per run&lt;/td&gt;
&lt;td&gt;$0.00035&lt;/td&gt;
&lt;td&gt;$0.00011&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Typical runs were already fine before. The &lt;strong&gt;tail&lt;/strong&gt; collapsed, and that's what matters for anything with a deadline. Cost also dropped by about two-thirds, because runs that finish faster use less compute.&lt;/p&gt;

&lt;h2&gt;
  
  
  Takeaways
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Measure where the time goes&lt;/strong&gt; before tuning backoff. My problem was stalls, not rate limits.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Set total-time budgets&lt;/strong&gt;, not just read timeouts.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;On 429, switch identity if you can&lt;/strong&gt;; back off only when you can't.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Keep a warm spare&lt;/strong&gt; so switching is instant.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Race independent slow steps&lt;/strong&gt;; keep dependent steps sequential.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Cap shutdown time.&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Fail fast and clearly&lt;/strong&gt; when the budget is gone.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;em&gt;This came out of building agent-friendly data tools (the one above is a &lt;a href="https://apify.com/datafaro/google-trends-api" rel="noopener noreferrer"&gt;Google Trends API on Apify&lt;/a&gt;). If you've found other ways to tame tail latency against rate-limited APIs, I'd love to hear them.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>python</category>
      <category>webdev</category>
      <category>performance</category>
      <category>api</category>
    </item>
    <item>
      <title>Designing tools for AI agents: why small outputs beat big ones</title>
      <dc:creator>Datafaro</dc:creator>
      <pubDate>Thu, 19 Nov 2026 10:19:36 +0000</pubDate>
      <link>https://dev.to/data_faro/designing-tools-for-ai-agents-why-small-outputs-beat-big-ones-1lln</link>
      <guid>https://dev.to/data_faro/designing-tools-for-ai-agents-why-small-outputs-beat-big-ones-1lln</guid>
      <description>&lt;p&gt;Most APIs were designed for humans and batch jobs. You ask for data, you get &lt;em&gt;all&lt;/em&gt; the data, and you process it later. That works fine for a script. It works badly for an AI agent.&lt;/p&gt;

&lt;p&gt;I've been building data tools that agents call through MCP (the Model Context Protocol), and the biggest lesson so far is simple: &lt;strong&gt;the shape of your output matters more than the amount of data you return.&lt;/strong&gt; Here's what I mean, with the rules I now follow.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why agents are different consumers
&lt;/h2&gt;

&lt;p&gt;When an agent calls your tool, three things happen that don't happen with a script:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Everything you return lands in the context window.&lt;/strong&gt; A 50 KB JSON response isn't "just data", it's tokens the model has to read, pay for, and keep around for the rest of the conversation.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The agent is waiting, and so is a human.&lt;/strong&gt; Many MCP clients wait around 30 seconds for a tool call by default. If your tool takes 45 seconds, the agent often gets "still running" instead of an answer, and the user sees a stalled conversation.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The model fills in your inputs.&lt;/strong&gt; Every extra parameter is one more thing the model can guess wrong.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;So a tool that is "complete" in the traditional sense can be a worse tool for an agent than one that does less, faster, with a clearer answer.&lt;/p&gt;

&lt;h2&gt;
  
  
  Rule 1: Lead with a one-line summary
&lt;/h2&gt;

&lt;p&gt;Every result I return includes a &lt;code&gt;summary&lt;/code&gt; field written in plain English, like:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Search interest in "claude" worldwide over the past 12 months is rising, up 299% from the first to the last quarter; it peaked at 28 on 2026-11-08 and averaged 17 on a 0-100 relative scale.
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The full time series is still there for anyone who wants to chart it. But in most conversations, the agent only needs that sentence to answer the user. It reads one line instead of 52 data points, and the answer is consistent because &lt;em&gt;I&lt;/em&gt; decided how to describe the trend, not the model on the fly.&lt;/p&gt;

&lt;h2&gt;
  
  
  Rule 2: Flat beats nested
&lt;/h2&gt;

&lt;p&gt;Compare:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"post"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"author"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"profile"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"handle"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"x.bsky.social"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"record"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"text"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"..."&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"metrics"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"likes"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;12&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;with:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"author"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"x.bsky.social"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"text"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"..."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nl"&gt;"likes"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;12&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Same information, fewer tokens, and nothing for the model to dig through. I convert raw API objects into flat records with obvious field names before returning them. If a field needs explaining, the name is wrong.&lt;/p&gt;

&lt;h2&gt;
  
  
  Rule 3: Cap the size by default
&lt;/h2&gt;

&lt;p&gt;Defaults matter more than limits, because agents rarely change them. A search tool that defaults to 1,000 results will return 1,000 results. I default to 10–50 items and let callers ask for more explicitly. For text content (like newsletter posts), full text is off by default and a ~300-character excerpt is on.&lt;/p&gt;

&lt;h2&gt;
  
  
  Rule 4: Design for the time budget
&lt;/h2&gt;

&lt;p&gt;I aim for a typical call under 10 seconds and a worst case under ~20. That shaped real decisions:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Hard per-request timeouts, so one slow upstream call can't eat the whole budget.&lt;/li&gt;
&lt;li&gt;Returning partial results with a clear flag rather than waiting for perfection.&lt;/li&gt;
&lt;li&gt;Documenting honestly when something is slow (bulk runs that take 1–2 minutes are fine for automations, not for chat).&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;One of my tools went from 44 seconds to about 6.5 seconds per call mostly by fixing timeouts and retries, not by making the data smaller. (I wrote about that separately.)&lt;/p&gt;

&lt;h2&gt;
  
  
  Rule 5: Narrow, forgiving inputs
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;One obvious required input&lt;/strong&gt; (a query, a URL, a list of keywords).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Sensible defaults for everything else&lt;/strong&gt;, and defaults that always return data. If someone calls the tool with an empty input, they should get a useful example, not an error.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Accept what models naturally send.&lt;/strong&gt; &lt;code&gt;@handle&lt;/code&gt;, &lt;code&gt;handle.bsky.social&lt;/code&gt; and a full profile URL should all work. A country written as &lt;code&gt;WORLD&lt;/code&gt; or left empty should both mean worldwide.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Fail softly per item.&lt;/strong&gt; If one of five keywords has no data, return four results and one clearly marked "no data" item. Don't fail the whole call.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Rule 6: Make errors useful to a model
&lt;/h2&gt;

&lt;p&gt;An error message is a prompt for the agent's next attempt. Compare:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;HTTP 400
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;with:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;"XX" is not an ISO 3166-1 country code. Use a 2-letter code like US, GB or ES (use GB for the United Kingdom).
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The second one gets fixed on the next try without the user ever noticing.&lt;/p&gt;

&lt;h2&gt;
  
  
  Rule 7: Price per retry, not per byte
&lt;/h2&gt;

&lt;p&gt;Agents retry. If a tool is expensive per call, every misunderstanding costs real money. I price per query or per result, keep the per-call cost low, and don't charge for failed or empty results. That makes the tool safe to call speculatively, which is exactly how agents use tools.&lt;/p&gt;

&lt;h2&gt;
  
  
  A checklist
&lt;/h2&gt;

&lt;p&gt;Before shipping a tool for agents, I now check:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;[ ] Every result has a one-line plain-English summary.&lt;/li&gt;
&lt;li&gt;[ ] Output is flat, with obvious field names.&lt;/li&gt;
&lt;li&gt;[ ] Default size is small; bigger is opt-in.&lt;/li&gt;
&lt;li&gt;[ ] Typical call &amp;lt; 10 s, worst case &amp;lt; ~20 s.&lt;/li&gt;
&lt;li&gt;[ ] Empty input returns a useful example.&lt;/li&gt;
&lt;li&gt;[ ] Inputs accept the formats a model is likely to send.&lt;/li&gt;
&lt;li&gt;[ ] Errors say how to fix the input.&lt;/li&gt;
&lt;li&gt;[ ] Failed and empty results are free.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;None of this is complicated. It's mostly about remembering who your user is: not a script that will dutifully parse whatever you send, but a model with a limited attention span and a human waiting on the other side.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;I'm applying these rules in a few small data tools for agents (Google Trends, Bluesky and Substack) published as &lt;a href="https://apify.com/datafaro" rel="noopener noreferrer"&gt;Apify Actors under Datafaro&lt;/a&gt;. If you're building agent tools too, I'd love to hear which rules you'd add.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>mcp</category>
      <category>api</category>
      <category>llm</category>
    </item>
    <item>
      <title>Google Trends data in 2026: pytrends is dead, here's what works</title>
      <dc:creator>Datafaro</dc:creator>
      <pubDate>Thu, 19 Nov 2026 10:07:33 +0000</pubDate>
      <link>https://dev.to/data_faro/google-trends-data-in-2026-pytrends-is-dead-heres-what-works-5fon</link>
      <guid>https://dev.to/data_faro/google-trends-data-in-2026-pytrends-is-dead-heres-what-works-5fon</guid>
      <description>&lt;p&gt;If you've tried to pull Google Trends data into a script recently, you've probably hit the same wall I did:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;pytrends&lt;/strong&gt; was archived in April 2025 and breaks regularly against Google's current endpoints.&lt;/li&gt;
&lt;li&gt;Google announced an &lt;strong&gt;official Trends API&lt;/strong&gt; in July 2025, but it's still an application-only alpha. Most developers can't get in.&lt;/li&gt;
&lt;li&gt;Calling Google's internal endpoints yourself works for a while, then you get &lt;code&gt;429 Too Many Requests&lt;/code&gt; and spend your evening on cookies, tokens and proxies.
This post shows a simple way to get Google Trends data from Python, from the command line and from an AI agent, with one call and no scraping code on your side.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What you can get
&lt;/h2&gt;

&lt;p&gt;Four modes, all on Google's usual 0–100 relative scale:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Mode&lt;/th&gt;
&lt;th&gt;What it returns&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;compare&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Interest over time for 1–5 keywords, plus average, peak and trend direction&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;by_region&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Which countries (or regions) search most for a keyword&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;related&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Related and rising queries for a keyword&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;trending_now&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;What a country is searching for right now, with approximate volume and news headlines&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Every result also includes a one-sentence &lt;code&gt;summary&lt;/code&gt;, which is handy if you're feeding the data to an LLM.&lt;/p&gt;

&lt;h2&gt;
  
  
  1. Quick test with curl
&lt;/h2&gt;

&lt;p&gt;Get a free API token from Apify (Console → Settings → API &amp;amp; Integrations), then:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl -X POST &lt;span class="s2"&gt;"https://api.apify.com/v2/acts/datafaro~google-trends-api/run-sync-get-dataset-items"&lt;/span&gt; \
  -H &lt;span class="s2"&gt;"Authorization: Bearer $APIFY_TOKEN"&lt;/span&gt; \
  -H &lt;span class="s2"&gt;"Content-Type: application/json"&lt;/span&gt; \
  -d &lt;span class="s1"&gt;'{"mode": "compare", "keywords": ["chatgpt", "claude", "gemini"], "geo": "WORLD", "timeframe": "past_12_months"}'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;run-sync-get-dataset-items&lt;/code&gt; waits for the run and returns the results directly. A typical comparison takes around 6–8 seconds.&lt;/p&gt;

&lt;h2&gt;
  
  
  2. From Python
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pip install apify-client
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;apify_client&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="nc"&gt;ApifyClient&lt;/span&gt;

&lt;span class="n"&gt;client&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;ApifyClient&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"YOUR_APIFY_TOKEN"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="n"&gt;run&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;actor&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"datafaro/google-trends-api"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;call&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;run_input&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="s2"&gt;"mode"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"compare"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"keywords"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"chatgpt"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"claude"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"gemini"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"geo"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"WORLD"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;            &lt;span class="c1"&gt;# or a 2-letter country code like "US", "ES"&lt;/span&gt;
    &lt;span class="s2"&gt;"timeframe"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"past_12_months"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;item&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;dataset&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;run&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"defaultDatasetId"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;iterate_items&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;item&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"keyword"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;item&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"trend"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;item&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"average"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"  "&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;item&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"summary"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;When I ran this in November 2026, the summaries looked like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;chatgpt stable 85
   Search interest in "chatgpt" worldwide over the past 12 months is stable (-2% last vs first quarter); it peaked at 100 on 2025-11-16 and averaged 85 ...
claude rising 17
   Search interest in "claude" worldwide over the past 12 months is rising, up 299% from the first to the last quarter ...
gemini rising 47
   Search interest in "gemini" worldwide over the past 12 months is rising, up 30% from the first to the last quarter ...
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Each item also has &lt;code&gt;points&lt;/code&gt; (the full weekly series as &lt;code&gt;{date, value}&lt;/code&gt;) if you want to chart it with pandas or matplotlib:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;pandas&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;pd&lt;/span&gt;

&lt;span class="n"&gt;items&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kt"&gt;list&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;dataset&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;run&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"defaultDatasetId"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;iterate_items&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;df&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;pd&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;concat&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;pd&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;DataFrame&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"points"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;assign&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;keyword&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"keyword"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;i&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;items&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;pivot&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;index&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"date"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;columns&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"keyword"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;values&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"value"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;df&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;plot&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;title&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"Google Trends: ChatGPT vs Claude vs Gemini"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  3. Which countries search most for something
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;run&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;actor&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"datafaro/google-trends-api"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;call&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;run_input&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="s2"&gt;"mode"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"by_region"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"keywords"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"bitcoin"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"geo"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"WORLD"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Result summary: &lt;em&gt;Top countries for "bitcoin" worldwide over the past 12 months: Switzerland (100), Austria (93), Germany (83), Netherlands (82), Slovenia (78).&lt;/em&gt; Useful for picking markets for a product or a campaign.&lt;/p&gt;

&lt;h2&gt;
  
  
  4. What's trending right now
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;run&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;actor&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"datafaro/google-trends-api"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;call&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;run_input&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="s2"&gt;"mode"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"trending_now"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="s2"&gt;"geo"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"ES"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;You get the top trending searches for that country, each with approximate search volume, when it started trending and the related news headlines, so you can see &lt;em&gt;why&lt;/em&gt; something is trending, not just that it is.&lt;/p&gt;

&lt;h2&gt;
  
  
  5. From an AI agent (Claude, ChatGPT, Cursor)
&lt;/h2&gt;

&lt;p&gt;This is the part I care most about. Apify runs an MCP server, so an agent can call the Actor as a tool. Add this MCP server URL to your client:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;https://mcp.apify.com/?tools=datafaro/google-trends-api
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Then just ask in plain language:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;em&gt;"Compare ChatGPT, Claude and Gemini on Google Trends worldwide over the last year."&lt;/em&gt;&lt;/li&gt;
&lt;li&gt;&lt;em&gt;"What's trending on Google in Spain right now, and why?"&lt;/em&gt;&lt;/li&gt;
&lt;li&gt;
&lt;em&gt;"Which countries search most for 'electric bikes'?"&lt;/em&gt;
The outputs are deliberately small and flat (a summary line plus a few fields), so they fit in an agent's context without blowing up your token count.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  A few things to know about Trends data
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Values are relative, not absolute.&lt;/strong&gt; 100 is the peak in the selected time range and region. You can't compare numbers across separate queries; put the keywords you want to compare in the same &lt;code&gt;compare&lt;/code&gt; call (up to 5).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Low-volume keywords are noisy.&lt;/strong&gt; If most points are 0, the summary flags it as sparse data.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Trending Now is per country.&lt;/strong&gt; There's no worldwide trending list in Google Trends; if you don't pass a country, it uses the US and says so.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Wrapping up
&lt;/h2&gt;

&lt;p&gt;If you just need Trends data occasionally, the Google Trends website is fine. If you need it in a script, a scheduled job or an AI agent, a hosted tool saves you from maintaining the rate-limit and token handling yourself.&lt;/p&gt;

&lt;p&gt;The Actor is here: &lt;a href="https://apify.com/datafaro/google-trends-api" rel="noopener noreferrer"&gt;apify.com/datafaro/google-trends-api&lt;/a&gt;. I'd love feedback, especially on what other Trends data you'd want an agent to have access to.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Disclosure:&lt;/strong&gt; I built the tool used above, a Google Trends Actor on the Apify platform. I wrote it because I needed reliable Trends data for agents and kept fighting rate limits. It's pay-per-query (about $0.0035 per query) and Apify's free plan includes $5/month of credit, so you can try everything here for free.&lt;/p&gt;
&lt;/blockquote&gt;

</description>
      <category>python</category>
      <category>api</category>
      <category>datascience</category>
      <category>ai</category>
    </item>
  </channel>
</rss>
