<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Preetam S Chaudhary</title>
    <description>The latest articles on DEV Community by Preetam S Chaudhary (@preetam960).</description>
    <link>https://dev.to/preetam960</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4131421%2F5e0e07fa-e64d-44ec-872b-5c1af2556899.png</url>
      <title>DEV Community: Preetam S Chaudhary</title>
      <link>https://dev.to/preetam960</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/preetam960"/>
    <language>en</language>
    <item>
      <title>Archiving a Telegram channel with Telethon, and re-running it without downloading twice</title>
      <dc:creator>Preetam S Chaudhary</dc:creator>
      <pubDate>Sat, 24 Oct 2026 06:54:02 +0000</pubDate>
      <link>https://dev.to/preetam960/archiving-a-telegram-channel-with-telethon-and-re-running-it-without-downloading-twice-4ckh</link>
      <guid>https://dev.to/preetam960/archiving-a-telegram-channel-with-telethon-and-re-running-it-without-downloading-twice-4ckh</guid>
      <description>&lt;p&gt;Downloading a Telegram channel's media the first time is mostly plumbing. The design work is in the second run, a week or a month later, when the goal is to download what was posted since and leave every file already on disk alone.&lt;/p&gt;

&lt;p&gt;TGLoader is a free desktop app I built for that job (&lt;a href="https://tgloader.preetam.org?utm_source=devto&amp;amp;utm_medium=devto&amp;amp;utm_campaign=tgloader-build-log&amp;amp;utm_content=tgl-1022-1" rel="noopener noreferrer"&gt;tgloader.preetam.org&lt;/a&gt;). It saves a channel's photos, videos, audio, voice messages, stickers and files to disk, on 64-bit Windows 10 or later and on Apple Silicon Macs. Telethon signs it in as you, using your own &lt;code&gt;api_id&lt;/code&gt; and &lt;code&gt;api_hash&lt;/code&gt; registered at my.telegram.org, so the app can read a channel only when your account can. Version 0.3.1 handles the second run in two layers, and only one of them is fully wired.&lt;/p&gt;

&lt;h2&gt;
  
  
  The message cursor
&lt;/h2&gt;

&lt;p&gt;Telethon has the standard tool for fetching only what is new. Message ids in a channel increase over time, and &lt;code&gt;iter_messages&lt;/code&gt; (like &lt;code&gt;get_messages&lt;/code&gt;) takes a &lt;code&gt;min_id&lt;/code&gt; that excludes every message at or below it:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;newest&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;last_seen_id&lt;/span&gt;
&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;msg&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;iter_messages&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;channel&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;min_id&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;last_seen_id&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="n"&gt;newest&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;max&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;newest&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;msg&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nb"&gt;id&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="c1"&gt;# queue msg.media for download
# save `newest` only after this run's downloads finish
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;In TGLoader this lives in &lt;code&gt;src/api/channel.py&lt;/code&gt;. Each configured channel gets its own &lt;code&gt;Channel&lt;/code&gt; object, and its &lt;code&gt;get_new_messages()&lt;/code&gt; reads &lt;code&gt;last_message_id&lt;/code&gt; from that channel's metadata (0 if absent) and passes it on as &lt;code&gt;min_id&lt;/code&gt;. Underneath, the client pages through the channel 100 messages at a time with &lt;code&gt;offset_id&lt;/code&gt;, pausing half a second between pages.&lt;/p&gt;

&lt;p&gt;In 0.3.1 only the read side is wired. &lt;code&gt;channel.py&lt;/code&gt; also has an &lt;code&gt;update_last_message_id()&lt;/code&gt; method, and nothing calls it, so the stored id stays at 0 and every run lists the channel again from the newest message back. A second run still doesn't re-download the channel, because the check that does the real work sits one level down, on each file.&lt;/p&gt;

&lt;h2&gt;
  
  
  The per-file tracker
&lt;/h2&gt;

&lt;p&gt;Each channel folder holds a &lt;code&gt;channel_info.json&lt;/code&gt;. When a file finishes, the app records it there under an id built from Telegram's own media id, &lt;code&gt;photo_&amp;lt;id&amp;gt;&lt;/code&gt; or &lt;code&gt;document_&amp;lt;id&amp;gt;&lt;/code&gt;. Before a file joins the queue, three checks run in order:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;If the id is in &lt;code&gt;channel_info.json&lt;/code&gt;, skip it.&lt;/li&gt;
&lt;li&gt;If a file already sits at the target path with exactly the size Telegram reports, skip it and add it to the tracker as &lt;code&gt;verified_existing&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;If a file sits there at a different size, treat it as a partial from an interrupted run, delete it and download again.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The second check means a lost or deleted &lt;code&gt;channel_info.json&lt;/code&gt; doesn't cost you a full re-download. It depends on the target path being the same every time, which is why the filename carries the media id: &lt;code&gt;&amp;lt;original name&amp;gt;-document_&amp;lt;id&amp;gt;.mp4&lt;/code&gt; for a named document, &lt;code&gt;photo_&amp;lt;id&amp;gt;.jpg&lt;/code&gt; for a photo. Two different files both called &lt;code&gt;video.mp4&lt;/code&gt; land at two paths, and the same file always lands at one.&lt;/p&gt;

&lt;p&gt;The tracker can be switched off per channel ("Remember downloaded files for this channel"). The disk check in step 2 runs either way.&lt;/p&gt;

&lt;h2&gt;
  
  
  Sorting by type
&lt;/h2&gt;

&lt;p&gt;The folder comes from the media attributes Telegram sends with each message. A &lt;code&gt;MessageMediaPhoto&lt;/code&gt; goes to &lt;code&gt;media/Pictures/&lt;/code&gt;. A document with a video attribute goes to &lt;code&gt;media/Videos/&lt;/code&gt;, one with an audio attribute to &lt;code&gt;media/Audio/&lt;/code&gt; (or &lt;code&gt;media/Voice Messages/&lt;/code&gt; when the voice flag is set), and an &lt;code&gt;image/webp&lt;/code&gt; or &lt;code&gt;application/x-tgsticker&lt;/code&gt; MIME type to &lt;code&gt;media/Stickers/&lt;/code&gt;. Everything else is a plain document and goes to &lt;code&gt;documents/&lt;/code&gt;, next to &lt;code&gt;media/&lt;/code&gt; rather than inside it.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;&amp;lt;channel folder&amp;gt;/
├── channel_info.json
├── documents/
└── media/
    ├── Pictures/
    ├── Videos/
    ├── Audio/
    ├── Voice Messages/
    └── Stickers/
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Each channel can point at its own download folder, so this tree lands wherever you chose for that channel.&lt;/p&gt;

&lt;h2&gt;
  
  
  Big files, and a 2 GB cap that shipped by accident
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;download.max_file_size&lt;/code&gt; is in MB. The app multiplies it by 1024 × 1024 and skips a file only when the limit is above zero and the file is bigger. The default is &lt;code&gt;0&lt;/code&gt;, which applies no cap at all, so a 4 GB video takes the same path as a 40 KB sticker.&lt;/p&gt;

&lt;p&gt;Version 0.3.0 got this wrong. A &lt;code&gt;config.yaml&lt;/code&gt; with &lt;code&gt;max_file_size: 2000&lt;/code&gt; ended up inside the package, and a packaged build runs with the extracted payload as its working directory, so that file became each new user's config on first launch. Anything over 2 GB was skipped with a line in the log and nothing in the window. 0.3.1 fixed the package, but a config saved by 0.3.0 keeps its value, so if you upgraded, open Settings and set the Max File Size field back to 0.&lt;/p&gt;

&lt;h2&gt;
  
  
  Three at a time, 1800 seconds each
&lt;/h2&gt;

&lt;p&gt;The defaults are 3 concurrent downloads and a 1800-second timeout per file. Concurrency comes from an &lt;code&gt;asyncio.Semaphore&lt;/code&gt; sized to the setting, and the queue is also cut into batches of that size, each awaited before the next one starts. One slow video therefore holds the other two slots idle until it finishes or gives up.&lt;/p&gt;

&lt;p&gt;Each attempt gets the full timeout. A failed file gets 3 attempts, 10 seconds apart; when Telegram answers with a flood wait, the app sleeps the seconds Telegram asks for plus 2 instead. The batch as a whole gets three times the per-file timeout before stragglers are cancelled.&lt;/p&gt;

&lt;p&gt;The 1800 seconds are wall-clock time. A 4 GB file has to average about 2.3 MB/s to finish inside it, so on a slower connection raise &lt;code&gt;download.timeout_seconds&lt;/code&gt; before you queue a channel of long videos.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;download&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;max_file_size&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;0&lt;/span&gt;          &lt;span class="c1"&gt;# MB, 0 = no limit&lt;/span&gt;
  &lt;span class="na"&gt;concurrent_downloads&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;3&lt;/span&gt;
  &lt;span class="na"&gt;timeout_seconds&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;1800&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Limits on a large channel
&lt;/h2&gt;

&lt;p&gt;Listing messages has a 120-second budget. When it runs out, the app logs a warning and carries on with what it fetched, which is the newest part of the channel. Since the cursor isn't saved, the next run lists the same newest stretch again, and the messages older than that window never come in, however many times you run it.&lt;/p&gt;

&lt;p&gt;The per-channel date range gets you past it. The To date goes to Telegram as &lt;code&gt;offset_date&lt;/code&gt;, so listing starts at the end of that day and walks back. The From date is applied after the fetch, so it trims what gets downloaded but doesn't shorten the listing. For a channel with a long back catalogue, set the To date to an older day, run it, and move the date back again for the next stretch.&lt;/p&gt;

</description>
      <category>api</category>
      <category>python</category>
      <category>software</category>
      <category>tools</category>
    </item>
  </channel>
</rss>
