<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Michael Shadpour</title>
    <description>The latest articles on DEV Community by Michael Shadpour (@realmichaelshad).</description>
    <link>https://dev.to/realmichaelshad</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4217706%2Fdead0359-0811-4e8c-ad92-3215a65a5f7f.jpg</url>
      <title>DEV Community: Michael Shadpour</title>
      <link>https://dev.to/realmichaelshad</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/realmichaelshad"/>
    <language>en</language>
    <item>
      <title>I'm really stuck with my app. Can someone help me figure out what to do?</title>
      <dc:creator>Michael Shadpour</dc:creator>
      <pubDate>Sun, 01 Nov 2026 23:51:38 +0000</pubDate>
      <link>https://dev.to/realmichaelshad/im-really-stuck-with-my-app-can-someone-help-me-figure-out-what-to-do-4h78</link>
      <guid>https://dev.to/realmichaelshad/im-really-stuck-with-my-app-can-someone-help-me-figure-out-what-to-do-4h78</guid>
      <description>&lt;p&gt;I'll be honest: I'm stuck, and I could really use help from people who've been here.&lt;/p&gt;

&lt;p&gt;A few weeks ago I posted about trying to sell Starling, an app I built that nobody had used yet. A lot of you told me the same thing: the price wasn't the problem, proof was. Get real users first.&lt;/p&gt;

&lt;p&gt;So I listened. I cancelled the Flippa auction (582 views, 4 watchers, 0 bids), and on 31 October I put Starling online as a free beta.&lt;/p&gt;

&lt;p&gt;What it does: Starling drafts replies on X in your own voice. You give it a few of your old posts, it learns how you write, and you get drafts that sound like you. There's a Chrome extension and a page that works on your phone. Nothing posts unless you press Post yourself. It works, and I'm proud of it.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Here's where I'm stuck&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The more I look into it, the worse it looks:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;X reply tools are everywhere, and anyone can ask ChatGPT or Grok to write a reply for free.&lt;/li&gt;
&lt;li&gt;I looked at 11 X writing tools that launched in 2025-2026 and show their revenue publicly. The median made about $10 in its first 60 days, and some of those founders had 20,000+ followers. I have almost no followers. My X posts get single-digit views.&lt;/li&gt;
&lt;li&gt;My honest guess for the next two months, even if I push hard: 30-90 sign-ups and 0-2 paying users.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;So I don't know what to do with it, and I keep going back and forth between:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Selling it now for whatever it gets&lt;/strong&gt; (probably $50-250) and moving on.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Changing who it's for:&lt;/strong&gt; people who reply online for work, like founders, salespeople and recruiters. Add LinkedIn and a right-click "reply in my voice" that works on any site, charge $19-29 a month, and give it until 1 February. If nobody pays by then, sell it.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Finding a partner with an audience&lt;/strong&gt; who'd sell it for a real share. But people with real audiences don't usually team up with apps that have no users.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Shutting it down.&lt;/strong&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;I'm leaning toward 2, but I really don't want to spend three more months on something that was never going to work.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;If you've been here before, please help&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;What would you do in my shoes?&lt;/li&gt;
&lt;li&gt;Is LinkedIn a real step up, or just as crowded as X?&lt;/li&gt;
&lt;li&gt;Would you pay $19 a month for replies in your own voice? What would make it worth it?&lt;/li&gt;
&lt;li&gt;Has anyone found a partner with an audience for a small product? How?&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Any advice helps, even "shut it down." I'd rather hear it now than in three months.&lt;/p&gt;

&lt;p&gt;If you want to try it, it's free at starlingwriter.com, and honest feedback on the drafts would mean a lot.&lt;/p&gt;

&lt;p&gt;Thank you. Your advice last time is the reason it's live at all.&lt;/p&gt;

</description>
      <category>discuss</category>
      <category>sideprojects</category>
      <category>startup</category>
      <category>ai</category>
    </item>
    <item>
      <title>Page headers and footers in RAG chunks: what I found in 25 government PDFs</title>
      <dc:creator>Michael Shadpour</dc:creator>
      <pubDate>Sat, 31 Oct 2026 23:39:39 +0000</pubDate>
      <link>https://dev.to/realmichaelshad/page-headers-and-footers-in-rag-chunks-what-i-found-in-25-government-pdfs-24fg</link>
      <guid>https://dev.to/realmichaelshad/page-headers-and-footers-in-rag-chunks-what-i-found-in-25-government-pdfs-24fg</guid>
      <description>&lt;p&gt;When you pull the text out of a PDF, the running header, the footer and the page number come out as ordinary lines, mixed in with the body. Nothing marks them as different. A RAG pipeline then cuts that text into chunks, and two things happen. The same footer lands in chunk after chunk, which makes those chunks look alike to the embedding model. And any chunk that crosses a page break gets a header in the middle of a sentence.&lt;/p&gt;

&lt;p&gt;I wanted to know how much of a real document this is, so I measured it on 25 public PDFs from US federal agencies. Disclosure up front: I make Agnos, a Windows app from Gidion that cleans documents before they go into AI tools, and the study measured what it removed. What it got wrong is in here too. That turned out to be the most useful part.&lt;/p&gt;

&lt;h2&gt;
  
  
  The setup
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;25 PDFs, 1,342 pages, from agency websites: handbooks, program guides, technical standards, regulatory guidance, annual reports, procurement, research reports and form instructions. All of them are text-based. None is a scan.&lt;/li&gt;
&lt;li&gt;I picked them by hand, not at random. I chose 21 because they have running headers or footers on most pages, and 4 because they have little. So the shares below describe these documents, not PDFs in general.&lt;/li&gt;
&lt;li&gt;The baseline is what a simple loader passes on: pdfplumber 0.11.10, &lt;code&gt;page.extract_text()&lt;/code&gt; with its default options. That's 64,682 non-empty lines and 3,509,754 characters.&lt;/li&gt;
&lt;li&gt;The removed lines come from the export log (&lt;code&gt;rejected.jsonl&lt;/code&gt;) of Agnos 2.3.2, run with its RAG / Retrieval Corpus purpose and default settings.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What came out
&lt;/h2&gt;

&lt;p&gt;Agnos removed 2,063 lines as page furniture, from 24 of the 25 documents. That's 50,928 characters, 1.45% of the plain text. Per document, the median was 1.32%, and the range ran from 0% to 3.17%.&lt;/p&gt;

&lt;p&gt;By volume, that's small. The problem is where it sits. The median was 1.18 lines of furniture per page. Furniture was removed from every text page of 3 documents, and from at least 90% of the text pages in 15. It's at every page break, so it lands in the middle of any chunk that crosses one.&lt;/p&gt;

&lt;p&gt;The most common kinds (one line can carry more than one label):&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Kind&lt;/th&gt;
&lt;th&gt;Lines&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Running title with a page number&lt;/td&gt;
&lt;td&gt;635&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Date or year&lt;/td&gt;
&lt;td&gt;590&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Page number on its own&lt;/td&gt;
&lt;td&gt;497&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Document number or form id&lt;/td&gt;
&lt;td&gt;488&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Agency or organisation name&lt;/td&gt;
&lt;td&gt;458&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Running title or document name, no page number&lt;/td&gt;
&lt;td&gt;457&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;"Page X of Y"&lt;/td&gt;
&lt;td&gt;305&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Some real ones: "Page 1 of 89" from NASA's software engineering requirements. "PAGE 3 OF 34 1333ND19QNB680025" from a NIST solicitation. And a line from an IRS printer's proof that starts "Page 2 of 59 Fileid: …" and runs on into a file path and a timestamp. None of that helps anyone answer a question.&lt;/p&gt;

&lt;h2&gt;
  
  
  Find them in your own PDFs
&lt;/h2&gt;

&lt;p&gt;You don't need any app to see this. The idea is simple. Compare lines with their digits masked, so "Page 3 of 12" and "Page 4 of 12" count as the same line. Then count how many pages each one is on.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;re&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;sys&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;collections&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;defaultdict&lt;/span&gt;

&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;pdfplumber&lt;/span&gt;  &lt;span class="c1"&gt;# pip install pdfplumber
&lt;/span&gt;
&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;shape&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;line&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="c1"&gt;# "Page 2 of 4" and "Page 3 of 4" have the same shape: "page # of #"
&lt;/span&gt;    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;re&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;sub&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;r&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;\d+&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;#&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt; &lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;join&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;line&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;lower&lt;/span&gt;&lt;span class="p"&gt;().&lt;/span&gt;&lt;span class="nf"&gt;split&lt;/span&gt;&lt;span class="p"&gt;()))&lt;/span&gt;

&lt;span class="k"&gt;with&lt;/span&gt; &lt;span class="n"&gt;pdfplumber&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;open&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;sys&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;argv&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;])&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;pdf&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;pages&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[(&lt;/span&gt;&lt;span class="n"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;extract_text&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="sh"&gt;""&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;splitlines&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;page&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;pdf&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;pages&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;

&lt;span class="n"&gt;on_pages&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;defaultdict&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;set&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;  &lt;span class="c1"&gt;# shape -&amp;gt; the pages it is on
&lt;/span&gt;&lt;span class="n"&gt;first_seen&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{}&lt;/span&gt;              &lt;span class="c1"&gt;# shape -&amp;gt; the line as first printed
&lt;/span&gt;&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;number&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;lines&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="nf"&gt;enumerate&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;pages&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;start&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;line&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;lines&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;line&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;strip&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
            &lt;span class="n"&gt;on_pages&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nf"&gt;shape&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;line&lt;/span&gt;&lt;span class="p"&gt;)].&lt;/span&gt;&lt;span class="nf"&gt;add&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;number&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="n"&gt;first_seen&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;setdefault&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;shape&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;line&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; &lt;span class="n"&gt;line&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;strip&lt;/span&gt;&lt;span class="p"&gt;())&lt;/span&gt;

&lt;span class="n"&gt;need&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;max&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mf"&gt;0.3&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="nf"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;pages&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;  &lt;span class="c1"&gt;# on 3+ pages, and on 30%+ of them
&lt;/span&gt;&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;s&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;seen&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="nf"&gt;sorted&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;on_pages&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;items&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt; &lt;span class="n"&gt;key&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="k"&gt;lambda&lt;/span&gt; &lt;span class="n"&gt;kv&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="nf"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;kv&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;])):&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="nf"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;seen&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;=&lt;/span&gt; &lt;span class="n"&gt;need&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;on &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="nf"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;seen&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt; of &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="nf"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;pages&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt; pages: &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;first_seen&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;s&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="si"&gt;!r}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Run it as &lt;code&gt;python repeated_lines.py your.pdf&lt;/code&gt;. On a four-page sample handbook I made for my guides, it prints:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;on 4 of 4 pages: 'Northwind Tools | Staff Handbook | March 2026'
on 4 of 4 pages: 'Confidential - Internal Use Only'
on 4 of 4 pages: 'Page 1 of 4'
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That's the easy case. Real documents are where it gets interesting.&lt;/p&gt;

&lt;h2&gt;
  
  
  Trap 1: masked digits match things that aren't furniture
&lt;/h2&gt;

&lt;p&gt;Masking the digits is what makes this work. It's also what makes it wrong:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;re&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;shape&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;line&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;re&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;sub&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;r&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;\d+&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;#&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt; &lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;join&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;line&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;lower&lt;/span&gt;&lt;span class="p"&gt;().&lt;/span&gt;&lt;span class="nf"&gt;split&lt;/span&gt;&lt;span class="p"&gt;()))&lt;/span&gt;

&lt;span class="n"&gt;pairs&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
    &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Page 3 of 12&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Page 4 of 12&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;      &lt;span class="c1"&gt;# furniture: these should match
&lt;/span&gt;    &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Table A-1.&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Table A-2.&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;          &lt;span class="c1"&gt;# two different table labels
&lt;/span&gt;    &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;1-800-555-0100&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;1-800-555-0199&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;  &lt;span class="c1"&gt;# two different phone numbers
&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;b&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;pairs&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="si"&gt;!r}&lt;/span&gt;&lt;span class="s"&gt; = &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;b&lt;/span&gt;&lt;span class="si"&gt;!r}&lt;/span&gt;&lt;span class="s"&gt;? &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="nf"&gt;shape&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="nf"&gt;shape&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;b&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;, both &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="nf"&gt;shape&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="si"&gt;!r}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;'Page 3 of 12' = 'Page 4 of 12'? True, both 'page # of #'
'Table A-1.' = 'Table A-2.'? True, both 'table a-#.'
'1-800-555-0100' = '1-800-555-0199'? True, both '#-#-#-#'
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;(The phone numbers are made-up 555 numbers.)&lt;/p&gt;

&lt;p&gt;The study ran into exactly this. We read every removed line, and checked its page when the line alone wasn't clear. 200 of the 2,063 looked like content. There were 36 table cells, 34 column headings repeated on each page, 22 figure and table notes, 21 table labels and 17 phone numbers, plus smaller groups. In the Census income report, "Table A-1." was removed while the table's title on the next line was kept. In NASA's requirements, "3.1.10 042 The project manager shall" was removed at a page edge. The rest of the sentence stayed, so that requirement lost its number.&lt;/p&gt;

&lt;p&gt;None of those lines is furniture. They have the same shape as furniture, at the edge of the page, on 3 or more pages.&lt;/p&gt;

&lt;h2&gt;
  
  
  Trap 2: an edge window misses things
&lt;/h2&gt;

&lt;p&gt;The safer way to use this rule is to look only at the edges of each page. Agnos looks at the first 2 and last 2 lines. There it removes page markers, page numbers that count up with the pages, and short lines (80 characters or fewer, not sentences) that repeat at the edges of 3 or more pages. That protects the body of the page. But it also means Agnos kept about as much repeated text as it removed.&lt;/p&gt;

&lt;p&gt;The biggest case: NASA NPR 7150.2D prints a four-line notice at the foot of all 89 pages. In the order Agnos reads the page, the page number and a header line come after the notice. Those two were removed, so the notice sits just outside the window. All four lines stayed: 26,433 characters, 12.8% of that document's plain text. An IRS proof notice stayed too. At 121 characters, it's over the length limit.&lt;/p&gt;

&lt;h2&gt;
  
  
  Trap 3: a percentage threshold finds nothing in some documents
&lt;/h2&gt;

&lt;p&gt;A common rule is "repeated on at least 30% of pages". Medicare &amp;amp; You 2027 changes its running header with each section. No single line is on 30% of its 128 pages, so that rule finds nothing at all. Agnos needs a line on only 3 pages, and it removed a header or page number from 104 of the document's 124 text pages. Neither threshold is right for every file.&lt;/p&gt;

&lt;p&gt;One more thing to watch: rotated pages. The Census report has 18 pages of rotated tables. Agnos's reader returned each of those pages as one or two long lines, so no furniture could be found on them. And pdfplumber read the rotated tables backwards: "emocni" for "income".&lt;/p&gt;

&lt;h2&gt;
  
  
  What I'd check before indexing
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;List the lines that repeat on 3 or more pages, and read them before you delete anything.&lt;/li&gt;
&lt;li&gt;Count the lines in your footers. A footer longer than your edge window stays.&lt;/li&gt;
&lt;li&gt;Look for long running headers. A length limit that protects sentences also protects an 85-character header.&lt;/li&gt;
&lt;li&gt;Log every removed line with its page. Without a log, a missing phone number or table label is very hard to find later.&lt;/li&gt;
&lt;li&gt;Open a few chunks that cross a page break, and read them.&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  What Agnos does
&lt;/h2&gt;

&lt;p&gt;Agnos does this for whole folders of text-based PDFs, Word, Excel, PowerPoint and web pages, on your own Windows 10 or 11 PC. It lists what it removed with the rule behind each removal. Review shows up to 500 removals per file, and the Everything export has the full log as &lt;code&gt;rejected.jsonl&lt;/code&gt;. Its limits: it doesn't do OCR, so scanned pages are skipped and named in its warnings. It runs on Windows only, and the Store version has no command line yet. If you'd rather stay in Python, Docling is free and open source. It labels page headers and footers and leaves them out of its default exports.&lt;/p&gt;

&lt;h2&gt;
  
  
  The full study
&lt;/h2&gt;

&lt;p&gt;The study page has every document, the method, all 200 lines that looked like content, and two scripts to reproduce it. The per-document numbers, 46 columns of them, are in a CSV under CC BY 4.0: &lt;a href="https://agnos.cloud/guides/page-furniture-study/" rel="noopener noreferrer"&gt;https://agnos.cloud/guides/page-furniture-study/&lt;/a&gt; To try this on your own PDFs without code, there's a free tool that runs in your browser and uploads nothing: &lt;a href="https://agnos.cloud/tools/pdf-repeated-lines/" rel="noopener noreferrer"&gt;https://agnos.cloud/tools/pdf-repeated-lines/&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;How do you check what your cleaning step removed? I'd like to know what other people do.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Drafted with help from an AI assistant (Claude). The numbers were checked against the study page, and both scripts were run on the sample PDF, before publishing.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>rag</category>
      <category>pdf</category>
      <category>llm</category>
      <category>python</category>
    </item>
    <item>
      <title>What I learned trying to sell an app nobody has used</title>
      <dc:creator>Michael Shadpour</dc:creator>
      <pubDate>Fri, 30 Oct 2026 16:54:04 +0000</pubDate>
      <link>https://dev.to/realmichaelshad/what-i-learned-trying-to-sell-an-app-nobody-has-used-4jdp</link>
      <guid>https://dev.to/realmichaelshad/what-i-learned-trying-to-sell-an-app-nobody-has-used-4jdp</guid>
      <description>&lt;p&gt;This summer I built Starling, a Chrome extension that drafts replies, quote posts and posts on X in your own voice. It learns how you write from your old posts, and nothing goes out until you press Post. Behind it there's a FastAPI backend, Stripe billing, an owner dashboard and about 5,500 automated tests.&lt;/p&gt;

&lt;p&gt;Then I realized I don't want to run a subscription business: support, payments, servers, all of it. So I tried to sell it instead. It has never launched, so it has 0 users and $0 revenue.&lt;/p&gt;

&lt;p&gt;Here's what a month of trying to sell it taught me.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;1. Cold email gets you read, not bought.&lt;/strong&gt; I sent about 700 personal emails to companies that build X tools or AI writing tools, and to people who buy small apps. 13 people wrote back. Most were polite no's, a couple were curious, and nobody made an offer.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. Check your spam folder.&lt;/strong&gt; The most serious buyer replied within a day of my email. His reply went straight to spam, and I found it 10 days later. By then he'd moved on.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;3. Price wasn't the problem.&lt;/strong&gt; I went from $1,950 to $1,200, then to an auction starting at $299. Views went up. Bids didn't. For a product with no users, a buyer isn't really pricing the code, they're pricing the risk that it won't work for them.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;4. Proof beats pitch.&lt;/strong&gt; I re-ran the full test suite on the exact version a buyer would get (4,838 tests passed, 0 failed) and put the result in the listing. What buyers really want, though, is to see it running for someone who isn't me, and that's the proof I'm still missing.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;5. Communities beat outreach.&lt;/strong&gt; One honest post on Indie Hackers about not being able to sell it got more than 20 comments and better advice than weeks of email.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;6. Have a backup.&lt;/strong&gt; A developer I emailed offered to launch it with me for free if it doesn't sell. That took a lot of pressure off.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Update (30 October):&lt;/strong&gt; I've stopped the sale and taken Starling off Flippa. Every piece of advice I got pointed the same way: the missing piece is real users, not a lower price. So I'm launching Starling as a real product instead, and I'll share how that goes.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;90-second demo: &lt;a href="https://www.youtube.com/watch?v=QJUr3JpdA3s" rel="noopener noreferrer"&gt;https://www.youtube.com/watch?v=QJUr3JpdA3s&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Website: &lt;a href="https://starlingwriter.com" rel="noopener noreferrer"&gt;https://starlingwriter.com&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If you've launched something with no audience, I'd love to hear what worked for you.&lt;/p&gt;

</description>
      <category>sideprojects</category>
      <category>showdev</category>
      <category>webdev</category>
      <category>ai</category>
    </item>
  </channel>
</rss>
