The latest articles on DEV Community by sensen0025 (@sensen0025). https://dev.to/sensen0025 https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4116854%2Fbd0add92-009e-4dc7-a091-f9f4fdd7487b.png https://dev.to/sensen0025 en sensen0025 Wed, 09 Sep 2026 05:55:43 +0000 https://dev.to/sensen0025/resources-downloader-4b7g https://dev.to/sensen0025/resources-downloader-4b7g <h1> I Built a Downloader That Doesn't Ask for Permission </h1> <p>Here's a small frustration we all share.</p> <p>You find a file. You click download. And then:</p> <ul> <li>Login required (for a public file)</li> <li>14 pop-ups before the button appears</li> <li>"Please verify you're human" — with 47 trackers loading in the background</li> <li>A captcha that takes 3 attempts</li> <li>Finally, an HTML page disguised as a PDF</li> </ul> <p>It's not malicious. It's just how the web works now. Data collection, ad impressions, engagement metrics — everything comes before the user's actual goal.</p> <p>I got tired of it. So I built a tool.</p> <h2> What I Built </h2> <p><strong>resources_downloader</strong> — a DSH skill pack that downloads resources from virtually any site, using an LLM agent to plan and execute the strategy.</p> <p>The code doesn't hardcode site-specific scrapers. Instead, it gives the agent a set of generic tools and lets it figure out the approach.</p> <div class="table-wrapper-paragraph"><table> <thead> <tr> <th>Tool</th> <th>Purpose</th> </tr> </thead> <tbody> <tr> <td><code>browser</code></td> <td>Playwright automation — login, modals, CAPTCHA solving, cookie export</td> </tr> <tr> <td><code>download_file</code></td> <td>Range-resume downloads</td> </tr> <tr> <td><code>download_hls</code></td> <td>M3U8 + AES-128 stream download</td> </tr> <tr> <td><code>probe_file</code></td> <td>Magic bytes verification — ensures the file is real, not an error page</td> </tr> <tr> <td><code>run_code</code></td> <td>Execute one-off Python scripts for undocumented sites</td> </tr> <tr> <td><code>memory_remember/query</code></td> <td>Persist working strategies per domain</td> </tr> </tbody> </table></div> <p><strong>21 skill cards</strong> currently cover Bilibili (BBDown), YouTube (yt-dlp), Anna's Archive, Project Gutenberg, HuggingFace, Quark netdisk, LittleSkin, and more.</p> <p>If your site isn't covered, the agent writes a crawler on the spot.</p> <h2> The Execution Flow </h2> <p>The agent follows a structured workflow:</p> <ol> <li>Check the catalog and query memory for past successes</li> <li>Prefer mature CLIs (BBDown, yt-dlp) before falling back to custom scripts</li> <li>Write and execute Python code if needed</li> <li>Verify the downloaded file — magic bytes, size, SHA-256</li> <li>Persist the winning strategy for next time</li> <li>If 3+ strategies fail, report failure with evidence and terminate gracefully</li> <li>Deliver full metadata — path, size, hash, source URL</li> </ol> <p>The goal is deterministic delivery. No empty files. No error pages passed off as downloads.</p> <h2> Free API Access — No Strings Attached </h2> <p>Most projects require you to bring your own LLM API key. That's fine, but it adds friction.</p> <p>So I'm providing <strong>free API keys</strong> for this service. No credit card required. No "start your free trial" nonsense. Just use it.<br> <a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fiirq6yk87tddemlferjn.png" class="article-body-image-wrapper"><img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fiirq6yk87tddemlferjn.png" alt=" " width="800" height="516"></a></p> deepseek ai resources 爬虫