<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Alexandr Rešetňak</title>
    <description>The latest articles on DEV Community by Alexandr Rešetňak (@resetnak).</description>
    <link>https://dev.to/resetnak</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F512925%2F917f9e06-1f11-4588-b188-b9e4acc7f811.jpeg</url>
      <title>DEV Community: Alexandr Rešetňak</title>
      <link>https://dev.to/resetnak</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/resetnak"/>
    <language>en</language>
    <item>
      <title>GitHub's traffic API has a 14 day memory. Here's what I learned archiving it.</title>
      <dc:creator>Alexandr Rešetňak</dc:creator>
      <pubDate>Fri, 21 Aug 2026 19:00:26 +0000</pubDate>
      <link>https://dev.to/resetnak/githubs-traffic-api-has-a-14-day-memory-heres-what-i-learned-archiving-it-1lc0</link>
      <guid>https://dev.to/resetnak/githubs-traffic-api-has-a-14-day-memory-heres-what-i-learned-archiving-it-1lc0</guid>
      <description>&lt;p&gt;Open any repo you own and click Insights → Traffic.&lt;/p&gt;

&lt;p&gt;You get views, clones, unique visitors, top referrers, and popular paths. It's a genuinely good little dashboard.&lt;/p&gt;

&lt;p&gt;It also forgets everything older than 14 days.&lt;/p&gt;

&lt;p&gt;Not "&lt;strong&gt;archived somewhere&lt;/strong&gt;." Not "available on request." Gone.&lt;/p&gt;

&lt;p&gt;There is no API parameter that reaches further back, no export button, and no support ticket that recovers it.&lt;/p&gt;

&lt;p&gt;If your project got picked up by a newsletter three weeks ago, the only record you have is the memory of seeing a spike.&lt;/p&gt;

&lt;p&gt;That bothered me enough to start polling the endpoints myself and writing everything into Postgres.&lt;/p&gt;

&lt;p&gt;Four endpoints. Every six hours. Per repository.&lt;/p&gt;

&lt;p&gt;Sounds like an afternoon of work.&lt;/p&gt;

&lt;p&gt;It was not an afternoon of work.&lt;/p&gt;

&lt;p&gt;Here's what got me.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;1. The traffic endpoints sit behind Administration permission&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;If you build this as a GitHub App, GET /repos/{owner}/{repo}/traffic/views requires Administration: read.&lt;/p&gt;

&lt;p&gt;That's it.&lt;/p&gt;

&lt;p&gt;That's the only scope GitHub offers for traffic data. There's no "Insights: read" and no narrower alternative.&lt;/p&gt;

&lt;p&gt;Which means the installation screen shows your users a permission that reads, roughly:&lt;/p&gt;

&lt;p&gt;This app can administer your repositories.&lt;/p&gt;

&lt;p&gt;For an app whose entire job is counting page views.&lt;/p&gt;

&lt;p&gt;I spent a while trying to find a way around it. There isn't one.&lt;/p&gt;

&lt;p&gt;What I did instead was request nothing else:&lt;/p&gt;

&lt;p&gt;no Contents&lt;br&gt;
no Issues&lt;br&gt;
no Actions&lt;br&gt;
no Secrets&lt;/p&gt;

&lt;p&gt;Just read-only Administration and read-only Metadata.&lt;/p&gt;

&lt;p&gt;I also added a page to the site explaining exactly why the permission is required, because I knew that screen was going to be the number one reason someone bounces.&lt;/p&gt;

&lt;p&gt;If you're building anything that touches traffic data, budget time for explaining that permission.&lt;/p&gt;

&lt;p&gt;It's a product problem, not a technical one.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. Daily uniques do not add up&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;This one I got wrong, shipped, and then had to go back and fix in five places.&lt;/p&gt;

&lt;p&gt;The views endpoint with per=day gives you a row per day with count and uniques.&lt;/p&gt;

&lt;p&gt;Views are a count, so summing them across days is fine.&lt;/p&gt;

&lt;p&gt;Uniques are deduplicated within each day, so summing them across days gives you a number that doesn't really mean anything.&lt;/p&gt;

&lt;p&gt;Someone who clones your repo on Monday, Tuesday, and Wednesday is one unique visitor in each of those three rows.&lt;/p&gt;

&lt;p&gt;Add them together and you get:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;3 unique visitors&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Which is wrong.&lt;/p&gt;

&lt;p&gt;And you can't fix it after the fact because GitHub never gave you the identity needed to deduplicate across days.&lt;/p&gt;

&lt;p&gt;There is no correct sum.&lt;/p&gt;

&lt;p&gt;So I stopped pretending there was one and labelled the column honestly:&lt;/p&gt;

&lt;p&gt;Daily uniques, 30-day sum&lt;/p&gt;

&lt;p&gt;Ugly label. Accurate label.&lt;/p&gt;

&lt;p&gt;Every export also includes a limitations field explaining the same thing, because a CSV someone opens six months from now has no tooltip to hover over.&lt;/p&gt;

&lt;p&gt;Worth checking your own dashboards for this.&lt;/p&gt;

&lt;p&gt;I'd bet a lot of them quietly sum uniques.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;3. Today's row is a moving target&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Poll at 06:00 UTC and today's row is only a few hours old.&lt;/p&gt;

&lt;p&gt;Poll again at 12:00 and the same date comes back with bigger numbers.&lt;/p&gt;

&lt;p&gt;So the write can't be a simple insert.&lt;/p&gt;

&lt;p&gt;It has to be an upsert keyed on:&lt;/p&gt;

&lt;p&gt;(repository_id, metric_date)&lt;/p&gt;

&lt;p&gt;And the whole pipeline has to agree on what a "day" means.&lt;/p&gt;

&lt;p&gt;I run the scheduler in UTC and store UTC dates because GitHub's day boundaries are UTC. Any translation layer between those two is a bug waiting for a DST change.&lt;/p&gt;

&lt;p&gt;Small thing.&lt;/p&gt;

&lt;p&gt;But it'll cost you a day of confused debugging if you get it wrong, because the numbers are only slightly off.&lt;/p&gt;

&lt;p&gt;And slightly-off numbers are the worst kind.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;4. Referrers have no dates at all&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;traffic/popular/referrers returns a top 10 for the last 14 days.&lt;/p&gt;

&lt;p&gt;No timestamps.&lt;/p&gt;

&lt;p&gt;No per-day breakdown.&lt;/p&gt;

&lt;p&gt;It's a single snapshot of a rolling window, and next week it'll be a different snapshot with no direct relationship to the previous one.&lt;/p&gt;

&lt;p&gt;The only way to build a time series is to write down what you saw and when you saw it.&lt;/p&gt;

&lt;p&gt;So I snapshot the top 10 daily, keyed by date.&lt;/p&gt;

&lt;p&gt;The subtle part is deletion.&lt;/p&gt;

&lt;p&gt;If Hacker News was in yesterday's top 10 and drops out today, a plain upsert leaves yesterday's HN row sitting in today's snapshot with a stale count.&lt;/p&gt;

&lt;p&gt;Today's snapshot has to be replaced as a set:&lt;/p&gt;

&lt;p&gt;delete whatever isn't in the new list&lt;br&gt;
upsert the rest&lt;br&gt;
do it all in one transaction&lt;/p&gt;

&lt;p&gt;Same for popular paths.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;5. Release downloads are cumulative&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The releases API gives you download_count per asset.&lt;/p&gt;

&lt;p&gt;It only ever goes up.&lt;/p&gt;

&lt;p&gt;Useful for a total.&lt;/p&gt;

&lt;p&gt;Useless for the actual question:&lt;/p&gt;

&lt;p&gt;Did this release land better than the previous one?&lt;/p&gt;

&lt;p&gt;To get velocity, you have to store the cumulative number every day and diff consecutive days yourself.&lt;/p&gt;

&lt;p&gt;Nobody hands you a rate.&lt;/p&gt;

&lt;p&gt;The part I'd tell past me&lt;/p&gt;

&lt;p&gt;The API itself is small:&lt;/p&gt;

&lt;p&gt;four endpoints&lt;br&gt;
an installation token&lt;br&gt;
a scheduled job&lt;/p&gt;

&lt;p&gt;The work was almost entirely in the semantics:&lt;/p&gt;

&lt;p&gt;which numbers can be added&lt;br&gt;
which day a row belongs to&lt;br&gt;
what to do when a value disappears from a list&lt;/p&gt;

&lt;p&gt;And then there's the one thing you can't engineer around:&lt;/p&gt;

&lt;p&gt;You cannot backfill.&lt;/p&gt;

&lt;p&gt;GitHub does not give you day 15.&lt;/p&gt;

&lt;p&gt;Whatever you didn't record, you don't have.&lt;/p&gt;

&lt;p&gt;My first sync grabs the roughly 14 days GitHub still holds. Everything after that accumulates from the moment tracking starts.&lt;/p&gt;

&lt;p&gt;Every chart is labelled with the date tracking started so nobody mistakes a gap for a zero.&lt;/p&gt;

&lt;p&gt;Which is a slightly uncomfortable thing to build a product on.&lt;/p&gt;

&lt;p&gt;The value only shows up later.&lt;/p&gt;

&lt;p&gt;I turned this into RepoMeter: &lt;a href="https://repometer.online" rel="noopener noreferrer"&gt;https://repometer.online&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;It's a hosted GitHub App that does all of the above for your repositories and keeps the history.&lt;/p&gt;

&lt;p&gt;It's free, with no billing in the app right now.&lt;/p&gt;

&lt;p&gt;It's read-only, and you can export everything to CSV and JSON so the archive isn't hostage to whether I keep the thing running.&lt;/p&gt;

&lt;p&gt;But if you just want the data, the endpoints are right there, and this post contains most of what you need to know.&lt;/p&gt;

&lt;p&gt;Start recording today rather than discovering in December that you have nothing from this autumn.&lt;/p&gt;

&lt;p&gt;If you've hit other quirks in these endpoints, I'd like to hear them.&lt;/p&gt;

&lt;p&gt;I'm fairly sure I haven't found them all.&lt;/p&gt;

</description>
      <category>api</category>
      <category>github</category>
      <category>opensource</category>
      <category>webdev</category>
    </item>
  </channel>
</rss>
