<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Sasha Kosobutsky</title>
    <description>The latest articles on DEV Community by Sasha Kosobutsky (@_sasha_2040_).</description>
    <link>https://dev.to/_sasha_2040_</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3941926%2Fc31d63e9-08a8-4819-b6e3-2881ed1c5e5b.png</url>
      <title>DEV Community: Sasha Kosobutsky</title>
      <link>https://dev.to/_sasha_2040_</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/_sasha_2040_"/>
    <language>en</language>
    <item>
      <title>Your SFTP Client Should Not Assume "/" - Lessons From a Home Router</title>
      <dc:creator>Sasha Kosobutsky</dc:creator>
      <pubDate>Wed, 18 Nov 2026 08:47:36 +0000</pubDate>
      <link>https://dev.to/_sasha_2040_/your-sftp-client-should-not-assume-lessons-from-a-home-router-4ano</link>
      <guid>https://dev.to/_sasha_2040_/your-sftp-client-should-not-assume-lessons-from-a-home-router-4ano</guid>
      <description>&lt;p&gt;&lt;em&gt;I am building &lt;a href="https://m-apps.net/syncgallery/html/en/index.html" rel="noopener noreferrer"&gt;SyncGallery&lt;/a&gt;, an Android gallery with configurable cloud sync. This post comes from one user report and the changes we made after reading it.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;A user plugged a USB disk into their Keenetic router, turned on its built-in SFTP server, and pointed their computers at it. Sync worked. Then they added the same server to SyncGallery on their phone.&lt;/p&gt;

&lt;p&gt;The login succeeded. Choosing a folder failed at once with "Permission denied".&lt;/p&gt;

&lt;p&gt;The router was fine. Our app was asking the wrong question.&lt;/p&gt;

&lt;h2&gt;
  
  
  1. "Permission denied" is a precise answer
&lt;/h2&gt;

&lt;p&gt;In SFTP, &lt;code&gt;SSH_FX_PERMISSION_DENIED&lt;/code&gt; does not mean the login failed. The &lt;a href="https://datatracker.ietf.org/doc/html/draft-ietf-secsh-filexfer-02#section-7" rel="noopener noreferrer"&gt;protocol draft&lt;/a&gt; returns it "when the authenticated user does not have sufficient permissions to perform the operation". One request was refused, and nothing else.&lt;/p&gt;

&lt;p&gt;The refused request was "list the contents of &lt;code&gt;/&lt;/code&gt;".&lt;/p&gt;

&lt;p&gt;On Keenetic routers, SFTP shows the real file system. A USB disk is mounted deep inside it:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;/tmp/mnt/&amp;lt;disk id or label&amp;gt;/&amp;lt;your folder&amp;gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Users on Keenetic's community forum report the same thing: some apps try to open the root folder, get refused, and drop the connection. FileZilla on a desktop connects and lands straight in the right folder.&lt;/p&gt;

&lt;p&gt;The difference is where each client starts.&lt;/p&gt;

&lt;h2&gt;
  
  
  2. Ask the server where to start
&lt;/h2&gt;

&lt;p&gt;The SFTP draft already answers this. Relative paths &lt;a href="https://datatracker.ietf.org/doc/html/draft-ietf-secsh-filexfer-02#section-6.2" rel="noopener noreferrer"&gt;are relative to "the user's default directory (home directory)"&lt;/a&gt;, and &lt;a href="https://datatracker.ietf.org/doc/html/draft-ietf-secsh-filexfer-02#section-6.11" rel="noopener noreferrer"&gt;&lt;code&gt;SSH_FXP_REALPATH&lt;/code&gt;&lt;/a&gt; turns any path into an absolute one. Ask for the real path of &lt;code&gt;.&lt;/code&gt; and the server tells you where you are supposed to be.&lt;/p&gt;

&lt;p&gt;The reference clients do exactly that:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;OpenSSH &lt;code&gt;sftp&lt;/code&gt;&lt;/strong&gt; requests the real path of &lt;code&gt;.&lt;/code&gt; at startup and uses it as the start folder (&lt;a href="https://github.com/openssh/openssh-portable/blob/V_9_9_P1/sftp.c" rel="noopener noreferrer"&gt;source&lt;/a&gt;).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;PuTTY &lt;code&gt;psftp&lt;/code&gt;&lt;/strong&gt; returns to "your home directory on the server" on &lt;code&gt;cd&lt;/code&gt; with no argument (&lt;a href="https://the.earth.li/~sgtatham/putty/0.83/htmldoc/Chapter6.html" rel="noopener noreferrer"&gt;manual&lt;/a&gt;).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Go's &lt;code&gt;pkg/sftp&lt;/code&gt;&lt;/strong&gt;, used by rclone, implements its working-directory call as a real-path request for &lt;code&gt;.&lt;/code&gt; (&lt;a href="https://github.com/pkg/sftp/blob/master/client.go" rel="noopener noreferrer"&gt;source&lt;/a&gt;).&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;A client that assumes everything starts at &lt;code&gt;/&lt;/code&gt; skips this step. On most servers that is harmless, because &lt;code&gt;/&lt;/code&gt; can be listed. On a server that refuses &lt;code&gt;/&lt;/code&gt;, it is the whole bug.&lt;/p&gt;

&lt;h2&gt;
  
  
  3. Routers are not the only ones
&lt;/h2&gt;

&lt;p&gt;Once we knew the pattern, we found it documented in other common setups:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Server&lt;/th&gt;
&lt;th&gt;What you see&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;OpenSSH with &lt;code&gt;ChrootDirectory&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;The jail root must be root-owned and not writable by others; after the chroot, sshd moves you to your home folder (&lt;a href="https://man.openbsd.org/sshd_config" rel="noopener noreferrer"&gt;sshd_config&lt;/a&gt;)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;OpenSSH &lt;code&gt;sftp-server -d&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;An "alternate starting directory" that only a client asking the server will find (&lt;a href="https://man.openbsd.org/sftp-server" rel="noopener noreferrer"&gt;sftp-server&lt;/a&gt;)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Hetzner Storage Box&lt;/td&gt;
&lt;td&gt;"Only the directory /home/ is writable"; relative paths recommended (&lt;a href="https://docs.hetzner.com/storage/storage-box/access/access-ssh-rsync-borg" rel="noopener noreferrer"&gt;docs&lt;/a&gt;)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Windows built-in OpenSSH&lt;/td&gt;
&lt;td&gt;Paths carry the drive letter, like &lt;code&gt;/C:/...&lt;/code&gt; (&lt;a href="https://github.com/PowerShell/Win32-OpenSSH/issues/730" rel="noopener noreferrer"&gt;Win32-OpenSSH #730&lt;/a&gt;, &lt;a href="https://github.com/curl/curl/commit/1abb087a9c8f1e613b0b38b7afeffb61c18c2fed" rel="noopener noreferrer"&gt;curl commit&lt;/a&gt;)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Synology NAS&lt;/td&gt;
&lt;td&gt;SFTP shows shared folders at the top; a &lt;code&gt;/volume1/...&lt;/code&gt; path copied from SSH does not exist over SFTP (&lt;a href="https://forum.restic.net/t/new-user-sftp-issues/165" rel="noopener noreferrer"&gt;restic forum&lt;/a&gt;, &lt;a href="https://ixsystems.atlassian.net/browse/NAS-131815" rel="noopener noreferrer"&gt;TrueNAS report&lt;/a&gt;)&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The lesson: &lt;code&gt;/&lt;/code&gt; is not a safe default. It is just the most common one.&lt;/p&gt;

&lt;h2&gt;
  
  
  4. Create folders from the bottom up
&lt;/h2&gt;

&lt;p&gt;Fixing the browser was not enough. Before an upload, SyncGallery made sure the target folder existed by walking the path from &lt;code&gt;/&lt;/code&gt; and creating each level. On this router, that meant asking about &lt;code&gt;/tmp&lt;/code&gt; and &lt;code&gt;/tmp/mnt&lt;/code&gt; - folders the user may not touch.&lt;/p&gt;

&lt;p&gt;An SFTP &lt;code&gt;MKDIR&lt;/code&gt; creates exactly one folder and &lt;a href="https://datatracker.ietf.org/doc/html/draft-ietf-secsh-filexfer-02#section-6.6" rel="noopener noreferrer"&gt;fails if it already exists&lt;/a&gt;. Creating missing parents is the client's job, and the order matters.&lt;/p&gt;

&lt;p&gt;Now SyncGallery checks the target first, climbs up only while folders are missing, then creates the missing ones on the way back down:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;/tmp/mnt/disk/Photos/2026/Camera   missing
/tmp/mnt/disk/Photos/2026          missing
/tmp/mnt/disk/Photos               exists -&amp;gt; stop climbing
create  .../Photos/2026
create  .../Photos/2026/Camera
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;It never looks above the first folder that exists. If a create fails, it checks once more and accepts the folder if it is now there, since another rule may have just made it. Climbing stops at &lt;code&gt;/&lt;/code&gt; and at Windows drive roots like &lt;code&gt;/C:&lt;/code&gt;. This is the same order &lt;a href="https://github.com/rclone/rclone/blob/master/backend/sftp/sftp.go" rel="noopener noreferrer"&gt;rclone's SFTP backend&lt;/a&gt; uses.&lt;/p&gt;

&lt;p&gt;A side benefit: uploading into a folder that already exists now costs one check instead of one request per path level.&lt;/p&gt;

&lt;h2&gt;
  
  
  5. Let people type the path
&lt;/h2&gt;

&lt;p&gt;The other half of the fix was in the account editor. The start folder could only be picked with Browse, and Browse opened at &lt;code&gt;/&lt;/code&gt;. There was no way to enter a long path by hand.&lt;/p&gt;

&lt;p&gt;Now:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;The folder browser lists &lt;code&gt;/&lt;/code&gt; if it can, and otherwise opens at the real path of &lt;code&gt;.&lt;/code&gt;. For FTP, &lt;code&gt;PWD&lt;/code&gt; plays the same role (&lt;a href="https://www.rfc-editor.org/rfc/rfc959" rel="noopener noreferrer"&gt;RFC 959&lt;/a&gt;).&lt;/li&gt;
&lt;li&gt;The start directory is a text field for SFTP and FTP. When you finish typing, the app opens that folder and tells you whether it does not exist or access was denied.&lt;/li&gt;
&lt;li&gt;Adding an SFTP, FTP or SMB account is split into steps: &lt;strong&gt;Connect&lt;/strong&gt;, then &lt;strong&gt;Start directory&lt;/strong&gt; (SMB adds &lt;strong&gt;Share&lt;/strong&gt;). Saving lists the chosen folder, so a wrong path shows up now, not during the first sync.&lt;/li&gt;
&lt;li&gt;If a saved start folder later stops working, the Files tab offers &lt;strong&gt;Change start directory&lt;/strong&gt; next to the error.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  6. Bonus: do not trust the first address
&lt;/h2&gt;

&lt;p&gt;While testing this case we hit an unrelated problem. A router behind a dynamic DNS name can have both IPv4 and IPv6 addresses, and on some networks the IPv6 route goes nowhere - for example, when the router forwards SSH over IPv4 only. Phones that prefer IPv6 waited until the connection timed out, while the numeric IPv4 address worked.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://www.rfc-editor.org/rfc/rfc6724#section-2" rel="noopener noreferrer"&gt;RFC 6724&lt;/a&gt; says applications "SHOULD NOT simply use the first address returned". &lt;a href="https://www.rfc-editor.org/rfc/rfc8305#section-4" rel="noopener noreferrer"&gt;RFC 8305 (Happy Eyeballs)&lt;/a&gt; adds that clients should interleave address families.&lt;/p&gt;

&lt;p&gt;SFTP, FTP and SMB connections in SyncGallery now try every resolved address, alternating IPv6 and IPv4. They move on only when the network fails - a wrong password is not retried at another address - and remember the address that answered while the app runs. Attempts run one after another rather than overlapping, which matters less for a backup connection than for opening a web page.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I took away
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;A refused &lt;code&gt;/&lt;/code&gt; is a normal server configuration, not an error state.&lt;/li&gt;
&lt;li&gt;Ask the server for its start folder with &lt;code&gt;REALPATH "."&lt;/code&gt; (or &lt;code&gt;PWD&lt;/code&gt; on FTP) instead of assuming one.&lt;/li&gt;
&lt;li&gt;Create missing folders from the deepest existing one downward, never from the root.&lt;/li&gt;
&lt;li&gt;Let users type a path and validate it immediately.&lt;/li&gt;
&lt;li&gt;Try every address a hostname resolves to.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;None of this was new. The SFTP drafts, the OpenSSH client, and rclone all handle it already. We only found it because one user sent a clear report with the exact path that worked in other tools.&lt;/p&gt;

&lt;p&gt;If you have shipped an SFTP or FTP client, I would love to hear which server layout surprised you the most.&lt;/p&gt;




&lt;p&gt;This article was edited with AI assistance and reviewed for technical accuracy by the author.&lt;/p&gt;

</description>
      <category>sftp</category>
      <category>networking</category>
      <category>android</category>
      <category>selfhosted</category>
    </item>
    <item>
      <title>Scaling an Android Gallery from 100,000 to 1,000,000 Files</title>
      <dc:creator>Sasha Kosobutsky</dc:creator>
      <pubDate>Mon, 24 Aug 2026 22:21:45 +0000</pubDate>
      <link>https://dev.to/_sasha_2040_/scaling-an-android-gallery-from-100000-to-1000000-files-430h</link>
      <guid>https://dev.to/_sasha_2040_/scaling-an-android-gallery-from-100000-to-1000000-files-430h</guid>
      <description>&lt;p&gt;&lt;em&gt;I am building &lt;a href="https://m-apps.net/syncgallery/html/en/index.html" rel="noopener noreferrer"&gt;SyncGallery&lt;/a&gt;, an Android gallery with configurable cloud sync. The examples below come from real customer failures, heap dumps, device profiles, and the code we changed after reading them.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Our original scale target was 100,000 media files.&lt;/p&gt;

&lt;p&gt;That sounded generous. At 30 new photos per day, it is about nine years of photos. Then real libraries showed up with old archives, phone migrations, imported camera cards, and large SD cards. The app did not suddenly fail at file 100,001. It became slower and heavier in lots of small places.&lt;/p&gt;

&lt;p&gt;We are now designing for one million files.&lt;/p&gt;

&lt;p&gt;Important disclaimer: one million is our new design target, not a benchmark we have completed across the whole app. The measurements in this post come from real scenarios between 1,000 and 100,000 records. The point is to explain what those measurements taught us, and what had to change before a million could be realistic.&lt;/p&gt;

&lt;p&gt;The snippets are simplified from production code.&lt;/p&gt;

&lt;h2&gt;
  
  
  1. A count should not build a screen
&lt;/h2&gt;

&lt;p&gt;The first expensive mistake looked innocent. A filter needed to show how many files matched a sync rule, so it reused the same projection that built the visible rows.&lt;/p&gt;

&lt;p&gt;Conceptually, the old path did this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight kotlin"&gt;&lt;code&gt;&lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="py"&gt;count&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;buildExplorerEntries&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;rule&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="n"&gt;size&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That one line hid a lot of work. For one rule, the app materialized 11,112 &lt;code&gt;ExplorerEntry&lt;/code&gt; objects and checked 8,810 paths on disk. It took 4,049 ms to produce one number.&lt;/p&gt;

&lt;p&gt;We replaced the projection with a batched aggregate:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="n"&gt;ruleId&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="k"&gt;COUNT&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="k"&gt;count&lt;/span&gt;
&lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="n"&gt;cloud_sync_mapping&lt;/span&gt;
&lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="n"&gt;excluded&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;
  &lt;span class="k"&gt;AND&lt;/span&gt; &lt;span class="n"&gt;ruleId&lt;/span&gt; &lt;span class="k"&gt;IN&lt;/span&gt; &lt;span class="p"&gt;(:&lt;/span&gt;&lt;span class="n"&gt;ruleIds&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;GROUP&lt;/span&gt; &lt;span class="k"&gt;BY&lt;/span&gt; &lt;span class="n"&gt;ruleId&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;In the measured sessions, the filter count dropped from about 1,350 ms to 8-230 ms, depending on the state of the rule and database.&lt;/p&gt;

&lt;p&gt;There was a second problem in the same path. Presence checking called &lt;code&gt;File(path).exists()&lt;/code&gt; for every row. A rule with 3,760 entries spent 1,720 ms doing individual file stats.&lt;/p&gt;

&lt;p&gt;Most files lived in a small number of directories, so we grouped paths by parent, listed each directory once, and answered membership from an in-memory name set. In one capture, 8,810 paths were spread across 34 directories. That changed 8,810 file stats into 34 directory listings.&lt;/p&gt;

&lt;p&gt;The rule I wrote down after this: if the UI needs a scalar, compute a scalar. If it needs membership, batch around the storage shape instead of asking the filesystem the same kind of question thousands of times.&lt;/p&gt;

&lt;h2&gt;
  
  
  2. "Latest result wins" does not cancel old work
&lt;/h2&gt;

&lt;p&gt;We had several background recomputations protected by generation counters. The idea was reasonable: start some work, and only publish if no newer generation has appeared.&lt;/p&gt;

&lt;p&gt;The shape looked like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight kotlin"&gt;&lt;code&gt;&lt;span class="n"&gt;nextGeneration&lt;/span&gt; &lt;span class="p"&gt;+=&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;
&lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="py"&gt;generation&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="n"&gt;nextGeneration&lt;/span&gt;

&lt;span class="n"&gt;viewModelScope&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;launch&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="py"&gt;result&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;recomputeBadges&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;generation&lt;/span&gt; &lt;span class="p"&gt;==&lt;/span&gt; &lt;span class="n"&gt;nextGeneration&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="nf"&gt;publish&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;result&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This prevented stale results from reaching the UI. It did not stop stale work.&lt;/p&gt;

&lt;p&gt;A heap dump at 25,000 mappings showed:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;ExplorerBadgeTarget       513,893 live instances
ExplorerBadgeIdentity      57,106 live instances
LinkedHashMap.Entry      1,244,611 live instances
App heap                        217 MB shallow
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;There were only about 57,000 distinct badge identities, but around nine full generations were alive at once. There was no classic leak and no single object dominating the heap. Several valid coroutines were all doing obsolete work and holding their own copies.&lt;/p&gt;

&lt;p&gt;We changed global recomputation to a conflated channel with one long-lived consumer:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight kotlin"&gt;&lt;code&gt;&lt;span class="k"&gt;private&lt;/span&gt; &lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="py"&gt;recomputeSignals&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Channel&lt;/span&gt;&lt;span class="p"&gt;&amp;lt;&lt;/span&gt;&lt;span class="nc"&gt;Unit&lt;/span&gt;&lt;span class="p"&gt;&amp;gt;(&lt;/span&gt;&lt;span class="nc"&gt;Channel&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;CONFLATED&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;private&lt;/span&gt; &lt;span class="k"&gt;fun&lt;/span&gt; &lt;span class="nf"&gt;requestRecompute&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="n"&gt;recomputeSignals&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;trySend&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;Unit&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="k"&gt;private&lt;/span&gt; &lt;span class="k"&gt;fun&lt;/span&gt; &lt;span class="nf"&gt;consumeRecomputes&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="n"&gt;viewModelScope&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;launch&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;Dispatchers&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;IO&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;signal&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="n"&gt;recomputeSignals&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="nf"&gt;recomputeBadges&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Now there can be at most one recomputation in flight and one queued signal. A burst collapses into the latest pending request. For work scoped to one gallery pane, we cancel that pane's previous job before starting its replacement.&lt;/p&gt;

&lt;p&gt;Generation checks are still useful for publication correctness. They are not a concurrency limit. If stale work is expensive, it needs cancellation, conflation, or both.&lt;/p&gt;

&lt;h2&gt;
  
  
  3. Boxed IDs are not free
&lt;/h2&gt;

&lt;p&gt;Android gallery code passes media IDs everywhere, so &lt;code&gt;Long&lt;/code&gt; collections are easy to ignore. Heap dumps made them hard to ignore.&lt;/p&gt;

&lt;p&gt;At 54,483 items, two collections used only for EXIF progress bookkeeping retained 5.99 MB of a 106.5 MB live heap:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight kotlin"&gt;&lt;code&gt;&lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="py"&gt;versions&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;HashMap&lt;/span&gt;&lt;span class="p"&gt;&amp;lt;&lt;/span&gt;&lt;span class="nc"&gt;Long&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nc"&gt;Long&lt;/span&gt;&lt;span class="p"&gt;&amp;gt;()&lt;/span&gt;
&lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="py"&gt;freshIds&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;HashSet&lt;/span&gt;&lt;span class="p"&gt;&amp;lt;&lt;/span&gt;&lt;span class="nc"&gt;Long&lt;/span&gt;&lt;span class="p"&gt;&amp;gt;()&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The useful data was tiny. Most of the cost came from boxed &lt;code&gt;Long&lt;/code&gt; objects and hash table nodes. We kept the same state and changed only its representation:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight kotlin"&gt;&lt;code&gt;&lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="py"&gt;versions&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;MutableLongLongMap&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="py"&gt;freshIds&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;MutableLongSet&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The expected retained size fell from 5.99 MB to roughly 1.7 MB, and clearing and refilling the collections could reuse flat arrays instead of rebuilding a node graph.&lt;/p&gt;

&lt;p&gt;Another heap dump showed a related problem. Two reverse indexes retained 21.5 MB at rest. They were derived transposes of data we already had, and their only live reader asked questions about the bounded set of rows visible on screen.&lt;/p&gt;

&lt;p&gt;We deleted the whole-library reverse indexes and replaced them with chunked database queries for visible rows. That trades a little database latency for a much smaller process-wide live set. The query runs on a conflated IO consumer, so it does not block the main thread or pile up unbounded copies.&lt;/p&gt;

&lt;p&gt;The lesson is not "replace every Kotlin collection with a primitive one." It is to do the memory math for any collection with one entry per media file. At one million files, even a small per-entry tax becomes architecture.&lt;/p&gt;

&lt;h2&gt;
  
  
  4. "Select all" should select a predicate
&lt;/h2&gt;

&lt;p&gt;Three customers on devices with a 256 MiB heap hit &lt;code&gt;OutOfMemoryError&lt;/code&gt; with about 90,000 active tasks and 90,000 parked rows.&lt;/p&gt;

&lt;p&gt;The parked approval screen was one source of pressure. Confirming a large selection held the same IDs in seven forms: the selected set, a defensive copy, a sorted list, discovered IDs, per-rule batches, committed IDs, and hidden IDs.&lt;/p&gt;

&lt;p&gt;The UI said "select all matching rows." The implementation heard "load every matching row ID and keep copying it."&lt;/p&gt;

&lt;p&gt;We changed selection from a materialized set to a frozen query specification:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight kotlin"&gt;&lt;code&gt;&lt;span class="kd"&gt;data class&lt;/span&gt; &lt;span class="nc"&gt;AllMatchingSelection&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="py"&gt;filters&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;SelectionFilters&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="py"&gt;maxRowId&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;Long&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="py"&gt;excludedIds&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;Set&lt;/span&gt;&lt;span class="p"&gt;&amp;lt;&lt;/span&gt;&lt;span class="nc"&gt;Long&lt;/span&gt;&lt;span class="p"&gt;&amp;gt;,&lt;/span&gt;
    &lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="py"&gt;includedIds&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;Set&lt;/span&gt;&lt;span class="p"&gt;&amp;lt;&lt;/span&gt;&lt;span class="nc"&gt;Long&lt;/span&gt;&lt;span class="p"&gt;&amp;gt;,&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The row ID watermark freezes membership at the moment the user taps Select all. New tasks that arrive later stay visible for the next confirmation instead of silently joining an action the user already approved.&lt;/p&gt;

&lt;p&gt;Processing then walks the selection in keyset pages:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="n"&gt;id&lt;/span&gt;
&lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="n"&gt;parked_tasks&lt;/span&gt;
&lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="n"&gt;id&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="n"&gt;afterId&lt;/span&gt;
  &lt;span class="k"&gt;AND&lt;/span&gt; &lt;span class="n"&gt;id&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;=&lt;/span&gt; &lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="n"&gt;maxRowId&lt;/span&gt;
  &lt;span class="k"&gt;AND&lt;/span&gt; &lt;span class="p"&gt;...&lt;/span&gt;
&lt;span class="k"&gt;ORDER&lt;/span&gt; &lt;span class="k"&gt;BY&lt;/span&gt; &lt;span class="n"&gt;id&lt;/span&gt;
&lt;span class="k"&gt;LIMIT&lt;/span&gt; &lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="n"&gt;pageSize&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The important part is not Room or this exact SQL. It is the model: a large selection is usually a predicate plus exceptions, not a giant set of IDs.&lt;/p&gt;

&lt;p&gt;That model also gives better crash behavior. A stable watermark makes the approved population explicit, and keyset paging gives each batch a clear continuation point.&lt;/p&gt;

&lt;h2&gt;
  
  
  5. Choose the list architecture before optimizing it
&lt;/h2&gt;

&lt;p&gt;There are several valid ways to show a large list. They make very different tradeoffs.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Model&lt;/th&gt;
&lt;th&gt;What stays in memory&lt;/th&gt;
&lt;th&gt;Main advantage&lt;/th&gt;
&lt;th&gt;Main risk&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Full list&lt;/td&gt;
&lt;td&gt;Every materialized row&lt;/td&gt;
&lt;td&gt;Simple code, instant access, exact total and position&lt;/td&gt;
&lt;td&gt;Memory grows directly with the library&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Explicit pages&lt;/td&gt;
&lt;td&gt;One page, sometimes nearby pages&lt;/td&gt;
&lt;td&gt;Strong memory bound and simple server queries&lt;/td&gt;
&lt;td&gt;Page buttons are awkward in a gallery&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Continuous loading&lt;/td&gt;
&lt;td&gt;Every chunk loaded as the user reaches the end&lt;/td&gt;
&lt;td&gt;Natural endless-scroll experience&lt;/td&gt;
&lt;td&gt;If old chunks stay, memory eventually becomes a full list&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Indexed window&lt;/td&gt;
&lt;td&gt;Compact keys for the full list, rich rows for one window&lt;/td&gt;
&lt;td&gt;Continuous scrolling with an exact total and bounded row memory&lt;/td&gt;
&lt;td&gt;The complete key index still has a memory cost&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F52npgtkxj4wbuh2raf85.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F52npgtkxj4wbuh2raf85.png" alt="Four UI patterns for a large gallery: full list, explicit pages, continuous loading, and an indexed window." width="800" height="480"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;The four models are not just different loading animations. They decide what can stay alive in memory.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Continuous loading is often called pagination, but the UX and memory behavior are different from explicit pages. Loading 100 more rows near the end does not make a list memory-safe if every earlier row remains alive. Dropping old chunks bounds memory, but then the app needs stable anchors and a way to recover the user's absolute position.&lt;/p&gt;

&lt;p&gt;For our gallery, explicit page buttons were the wrong experience. A fully materialized list was the wrong memory model. We chose an indexed window as the compromise.&lt;/p&gt;

&lt;h3&gt;
  
  
  Why we keep the complete index
&lt;/h3&gt;

&lt;p&gt;One gallery filter could not use our indexed path, so it always built the full projection. At 100,000 favorites, one refresh materialized about 55 MB.&lt;/p&gt;

&lt;p&gt;But the screen did not need 100,000 rich row objects. It needed a stable order, a total count, and enough objects to draw the current window.&lt;/p&gt;

&lt;p&gt;With a complete index, the UI knows the exact collection size and the absolute position of the visible window. It can say item 450,000 of 1,000,000 and drive a real scrollbar or position progress indicator. A pure endless list only knows how many rows have arrived so far. It cannot know the real denominator until it reaches the end or runs a separate count query.&lt;/p&gt;

&lt;p&gt;The pane therefore stores compact identity for every position and materializes content only for the current window. The simplified production shape is:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight kotlin"&gt;&lt;code&gt;&lt;span class="kd"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;RuleFilterIndex&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="py"&gt;ids&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;LongArray&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="py"&gt;kinds&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;ByteArray&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nf"&gt;init&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="nf"&gt;require&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ids&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;size&lt;/span&gt; &lt;span class="p"&gt;==&lt;/span&gt; &lt;span class="n"&gt;kinds&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;size&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The kind byte tells us which database table owns an ID. A window load slices both primitive arrays, fetches only those rows, and publishes the full index size alongside the window:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight kotlin"&gt;&lt;code&gt;&lt;span class="k"&gt;suspend&lt;/span&gt; &lt;span class="k"&gt;fun&lt;/span&gt; &lt;span class="nf"&gt;loadWindow&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;start&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;Int&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;size&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;Int&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="py"&gt;end&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;start&lt;/span&gt; &lt;span class="p"&gt;+&lt;/span&gt; &lt;span class="n"&gt;size&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;coerceAtMost&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;index&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;ids&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;size&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="py"&gt;ids&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="n"&gt;index&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;ids&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;copyOfRange&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;start&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;end&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="py"&gt;kinds&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="n"&gt;index&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;kinds&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;copyOfRange&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;start&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;end&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="py"&gt;rows&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="n"&gt;source&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;materializeWindow&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ruleId&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;ids&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;kinds&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="nf"&gt;publish&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="n"&gt;rowCount&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="n"&gt;index&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;ids&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;size&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;windowStart&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="n"&gt;start&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;window&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="n"&gt;rows&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  The index needs its own memory budget
&lt;/h3&gt;

&lt;p&gt;Windowing does not make the full-list cost disappear. It changes the cost from rich objects to compact keys, so the key representation becomes load-bearing architecture.&lt;/p&gt;

&lt;p&gt;For one million rows, the raw payload is easy to calculate:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;LongArray: 1,000,000 * 8 bytes = 7.63 MiB
ByteArray: 1,000,000 * 1 byte  = 0.95 MiB
Combined raw payload                 = 8.58 MiB
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;There is a little extra array and object overhead, but the payload dominates at this size. If a list needs only one ID, the base index is about 7.63 MiB. Our rule-filter index needs an ID plus a one-byte kind, so its raw budget is about 8.58 MiB.&lt;/p&gt;

&lt;p&gt;The representation matters. In this code path, we budget roughly 32 bytes per entry for a boxed &lt;code&gt;List&amp;lt;Long&amp;gt;&lt;/code&gt;. At one million rows, that would be about 30.5 MiB for the IDs alone instead of 7.63 MiB. A parallel boxed kind collection would make it worse.&lt;/p&gt;

&lt;p&gt;The full projection gives the other end of the tradeoff. If 100,000 rich rows take about 55 MB, a rough linear extrapolation to one million is about 550 MB, before temporary copies or concurrent recomputations. That does not fit a 256 MiB Android heap. An 8.58 MiB index plus a bounded content window can.&lt;/p&gt;

&lt;p&gt;This is not "paging fixes everything." The index still needs invalidation, loading it still costs time, and window fetches need stable ordering. The useful boundary is that the whole collection is represented by primitive identity, while expensive objects exist only for the visible window.&lt;/p&gt;

&lt;h2&gt;
  
  
  What one million means for us
&lt;/h2&gt;

&lt;p&gt;We have not reached a point where every screen can handle one million files. Some paths still contain assumptions from the 100,000-file design, and the rollout is happening screen by screen.&lt;/p&gt;

&lt;p&gt;The number is useful anyway because it changes design reviews now.&lt;/p&gt;

&lt;p&gt;If a screen keeps one object per file for its whole lifetime, we ask for the memory math. If an event rebuilds the whole collection, we ask how often that event can fire and whether old work stops. If a button says Select all, we ask whether it stores a predicate or a list. If the UI needs a count, a badge, or a visible page, we ask why it would hydrate anything else.&lt;/p&gt;

&lt;p&gt;The production results so far are encouraging:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;One filter workload dropped from 74.5 MB/s to 39.2 MB/s of allocation.&lt;/li&gt;
&lt;li&gt;Blocking garbage collections dropped from 326 to 19 in comparable captures.&lt;/li&gt;
&lt;li&gt;A separate cache change reduced a measured heap peak from 216 MB to 137 MB and eliminated observed main-thread GC waits in that capture.&lt;/li&gt;
&lt;li&gt;The 100,000-favorite filter moved from a roughly 55 MB full projection to an index plus one materialized window.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Those are not one-million-file benchmark results. They are evidence that removing work proportional to the whole library, especially repeated or concurrent work, is the right direction.&lt;/p&gt;

&lt;p&gt;My short version is:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Never materialize a collection to compute a count.&lt;/li&gt;
&lt;li&gt;Prevent stale work from running, not only from publishing.&lt;/li&gt;
&lt;li&gt;Measure boxed IDs and derived indexes as real product memory.&lt;/li&gt;
&lt;li&gt;Represent large selections as predicates plus exceptions.&lt;/li&gt;
&lt;li&gt;Keep rich objects inside a bounded visible window.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The surprising part was that no single clever algorithm fixed the app. The big improvement came from finding many ordinary places where convenient code quietly copied, boxed, hydrated, or recomputed the whole world.&lt;/p&gt;

&lt;p&gt;If you have scaled a media app, file manager, or offline database past 100,000 local records, I would love to hear which assumption broke first for you.&lt;/p&gt;




&lt;p&gt;This article was edited with AI assistance and reviewed for technical accuracy by the author.&lt;/p&gt;

</description>
      <category>android</category>
      <category>mobile</category>
      <category>kotlin</category>
      <category>performance</category>
    </item>
  </channel>
</rss>
