<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Hao</title>
    <description>The latest articles on DEV Community by Hao (@liuhao04).</description>
    <link>https://dev.to/liuhao04</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4044236%2F289e92a2-c37e-480f-aad8-830aa21d09d3.png</url>
      <title>DEV Community: Hao</title>
      <link>https://dev.to/liuhao04</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/liuhao04"/>
    <language>en</language>
    <item>
      <title>Your YouTube screenshot is black because you screenshotted the wrong layer</title>
      <dc:creator>Hao</dc:creator>
      <pubDate>Sat, 25 Jul 2026 03:55:50 +0000</pubDate>
      <link>https://dev.to/liuhao04/your-youtube-screenshot-is-black-because-you-screenshotted-the-wrong-layer-14cm</link>
      <guid>https://dev.to/liuhao04/your-youtube-screenshot-is-black-because-you-screenshotted-the-wrong-layer-14cm</guid>
      <description>&lt;p&gt;You hit PrintScreen on a video, paste, and get a black rectangle where the picture was. The window&lt;br&gt;
frame is there. The controls are there. The video is a hole.&lt;/p&gt;

&lt;p&gt;The video was never in the pixels you copied.&lt;/p&gt;
&lt;h2&gt;
  
  
  What actually happened
&lt;/h2&gt;

&lt;p&gt;A hardware-accelerated video does not get painted into the same surface as the rest of the window.&lt;br&gt;
The decoder hands frames to the GPU, which composites them into an overlay plane at the very last&lt;br&gt;
step, after the window contents have been assembled. A screen capture that reads the window surface&lt;br&gt;
reads what is underneath that plane, which is the punch-out colour: black.&lt;/p&gt;

&lt;p&gt;This is why the same PrintScreen works fine on some machines and fails on others, why it starts&lt;br&gt;
failing after a GPU driver update, and why "disable hardware acceleration" is the advice that keeps&lt;br&gt;
getting handed out. Turning off acceleration does fix it, by making the video render into the normal&lt;br&gt;
paint path, along with the battery cost.&lt;/p&gt;

&lt;p&gt;The better move is to stop capturing the window and ask the &lt;code&gt;&amp;lt;video&amp;gt;&lt;/code&gt; element for the frame instead.&lt;/p&gt;
&lt;h2&gt;
  
  
  Grab the frame from the element
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;drawImage&lt;/code&gt; accepts a &lt;code&gt;&amp;lt;video&amp;gt;&lt;/code&gt; and reads the frame it is currently showing:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;v&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;document&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;querySelector&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;video&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;c&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;document&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;createElement&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;canvas&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="nx"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;width&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;v&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;videoWidth&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;          &lt;span class="c1"&gt;// NOT v.clientWidth&lt;/span&gt;
&lt;span class="nx"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;height&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;v&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;videoHeight&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="nx"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getContext&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;2d&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;drawImage&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;v&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="nx"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;toBlob&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;b&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="nf"&gt;open&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;URL&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;createObjectURL&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;b&lt;/span&gt;&lt;span class="p"&gt;)),&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;image/png&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Paste that into the console on a YouTube tab and you get the exact frame, no black.&lt;/p&gt;

&lt;p&gt;The one line worth staring at is &lt;code&gt;v.videoWidth&lt;/code&gt;. That is the resolution of the decoded stream, which&lt;br&gt;
is what you want. &lt;code&gt;clientWidth&lt;/code&gt; is the size of the CSS box, so using it silently downsamples a 4K&lt;br&gt;
frame to whatever your player happens to be sized at in that window. This is the difference between a&lt;br&gt;
usable still and a soft 640-pixel-wide one, and it is easy to get wrong because both produce an image.&lt;/p&gt;

&lt;p&gt;Set the player to 2160p first if you want 2160p out. &lt;code&gt;videoWidth&lt;/code&gt; reports the stream YouTube is&lt;br&gt;
currently feeding you, not the best one available.&lt;/p&gt;
&lt;h2&gt;
  
  
  Straight to the clipboard
&lt;/h2&gt;

&lt;p&gt;Skipping the file is often what you want, since the frame is usually headed for a slide or a note:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;png&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;Promise&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;r&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="nx"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;toBlob&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;r&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;image/png&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;));&lt;/span&gt;
&lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nb"&gt;navigator&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;clipboard&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;write&lt;/span&gt;&lt;span class="p"&gt;([&lt;/span&gt;&lt;span class="k"&gt;new&lt;/span&gt; &lt;span class="nc"&gt;ClipboardItem&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;image/png&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;png&lt;/span&gt; &lt;span class="p"&gt;})]);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Two constraints that will bite you. The clipboard only takes PNG here, so if you produced a JPEG&lt;br&gt;
blob you have to re-encode before writing. And &lt;code&gt;clipboard.write&lt;/code&gt; needs a user gesture and a focused&lt;br&gt;
document, so it works from a click handler and fails from a &lt;code&gt;setTimeout&lt;/code&gt;.&lt;/p&gt;
&lt;h2&gt;
  
  
  When it still comes out black
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;drawImage&lt;/code&gt; on a DRM-protected video throws instead of returning pixels. Widevine-protected content —&lt;br&gt;
a rented film, most paid streaming — is out of reach by design, and no extension or console snippet&lt;br&gt;
changes that. YouTube's ordinary videos are not protected, so they are fine.&lt;/p&gt;

&lt;p&gt;Wrap it, because an unhandled throw here looks identical to a broken tool:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="k"&gt;try&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nx"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;drawImage&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;v&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;catch&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="cm"&gt;/* protected stream */&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Naming the file
&lt;/h2&gt;

&lt;p&gt;If you are saving rather than copying, put the timestamp in the name while you still have it:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;t&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;Math&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;floor&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;v&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;currentTime&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;stamp&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;`&lt;/span&gt;&lt;span class="p"&gt;${&lt;/span&gt;&lt;span class="nc"&gt;String&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;t&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="mi"&gt;3600&lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;padStart&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;0&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)}&lt;/span&gt;&lt;span class="s2"&gt;-&lt;/span&gt;&lt;span class="p"&gt;${&lt;/span&gt;&lt;span class="nc"&gt;String&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;t&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="mi"&gt;60&lt;/span&gt;&lt;span class="o"&gt;%&lt;/span&gt;&lt;span class="mi"&gt;60&lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;padStart&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;0&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)}&lt;/span&gt;&lt;span class="s2"&gt;-&lt;/span&gt;&lt;span class="p"&gt;${&lt;/span&gt;&lt;span class="nc"&gt;String&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;t&lt;/span&gt;&lt;span class="o"&gt;%&lt;/span&gt;&lt;span class="mi"&gt;60&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;padStart&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;0&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)}&lt;/span&gt;&lt;span class="s2"&gt;`&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Frames without a timestamp in the filename are unfindable a week later, which anyone who has taken&lt;br&gt;
lecture notes this way has learned the hard way.&lt;/p&gt;

&lt;h2&gt;
  
  
  The extension
&lt;/h2&gt;

&lt;p&gt;I packaged this as &lt;strong&gt;&lt;a href="https://chromewebstore.google.com/detail/hcffndaocfbpfjfcdmdnkdnacjbdnbnk" rel="noopener noreferrer"&gt;Screenshot for YouTube — Video Frame&lt;br&gt;
Capture&lt;/a&gt;&lt;/strong&gt;. Disclosure:&lt;br&gt;
mine, free, no account, no network requests.&lt;/p&gt;

&lt;p&gt;It puts a camera button in the player controls, binds Alt+S, saves PNG or JPEG or copies to the&lt;br&gt;
clipboard, and names files with the video title plus the timestamp above. There is a popup capture&lt;br&gt;
button as a fallback for Shorts and embeds, where the standard control bar is absent. Getting the&lt;br&gt;
button to survive YouTube's SPA navigation needed a &lt;code&gt;MutationObserver&lt;/code&gt; plus the &lt;code&gt;yt-navigate-finish&lt;/code&gt;&lt;br&gt;
event, which is the least interesting and most fiddly part of the whole thing.&lt;/p&gt;

&lt;p&gt;If you only need one frame today, the console snippet above is the entire mechanism. The extension&lt;br&gt;
exists because the twentieth time you do it by hand, &lt;code&gt;videoWidth&lt;/code&gt; is not the thing you want to be&lt;br&gt;
remembering.&lt;/p&gt;

</description>
      <category>javascript</category>
      <category>chrome</category>
      <category>webdev</category>
      <category>video</category>
    </item>
    <item>
      <title>Why the most-installed "PDF dark mode" Chrome extensions do nothing on a web PDF</title>
      <dc:creator>Hao</dc:creator>
      <pubDate>Sat, 25 Jul 2026 03:54:35 +0000</pubDate>
      <link>https://dev.to/liuhao04/why-most-pdf-dark-mode-chrome-extensions-do-nothing-on-a-web-pdf-on5</link>
      <guid>https://dev.to/liuhao04/why-most-pdf-dark-mode-chrome-extensions-do-nothing-on-a-web-pdf-on5</guid>
      <description>&lt;p&gt;Chrome still ships no dark mode for its built-in PDF viewer. Open a white paper at 1am and you get a&lt;br&gt;
flashbang. So you go to the Web Store, install the extension with the most installs, click it, and…&lt;br&gt;
nothing happens. The page stays white.&lt;/p&gt;

&lt;p&gt;I went and read the manifests of the top results to find out why. Two reasons, and both are boring.&lt;/p&gt;
&lt;h2&gt;
  
  
  Reason 1: the popular ones only handle &lt;code&gt;file://&lt;/code&gt;
&lt;/h2&gt;

&lt;p&gt;The extension named "PDF Dark Mode" (about 10,000 users, rated 2.5) declares exactly this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="nl"&gt;"permissions"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"scripting"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"declarativeContent"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="err"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="nl"&gt;"host_permissions"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"file:///*.pdf"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The runner-up, "PDF Dark Theme" (about 9,000 users, rated 2.9), does the same thing with a content&lt;br&gt;
script:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="nl"&gt;"content_scripts"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"matches"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"file://*.pdf"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"js"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"content-script.js"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}]&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;file:///*.pdf&lt;/code&gt; matches a PDF you dragged in from your own disk. It does not match&lt;br&gt;
&lt;code&gt;https://arxiv.org/pdf/1706.03762&lt;/code&gt;, or the invoice your bank linked, or the syllabus on a course&lt;br&gt;
site. That is where almost everyone actually meets a PDF. So the extension is installed, enabled,&lt;br&gt;
and structurally incapable of touching the document in front of you.&lt;/p&gt;

&lt;p&gt;To be fair about the scope of this: it is the two &lt;em&gt;most-installed&lt;/em&gt; ones that are scoped this way,&lt;br&gt;
not the whole category. Several smaller ones (a few hundred to a few thousand users) request&lt;br&gt;
&lt;code&gt;&amp;lt;all_urls&amp;gt;&lt;/code&gt; or use &lt;code&gt;activeTab&lt;/code&gt; and do handle web PDFs. The ranking is weighted by install count, so&lt;br&gt;
the two that cannot do the common case are the two you see first.&lt;/p&gt;

&lt;p&gt;This is also why the reviews are full of people being told to flip "Allow access to file URLs" and&lt;br&gt;
reporting back that it changed nothing. It was never the missing piece.&lt;/p&gt;

&lt;p&gt;You can check any extension for this in ten seconds: &lt;code&gt;chrome://extensions&lt;/code&gt; → Details → look at&lt;br&gt;
"Site access". If it says nothing beyond file URLs, that is your answer.&lt;/p&gt;
&lt;h2&gt;
  
  
  Reason 2: the CSS target moved
&lt;/h2&gt;

&lt;p&gt;The other approach is a CSS filter on the viewer element:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight css"&gt;&lt;code&gt;&lt;span class="nt"&gt;embed&lt;/span&gt;&lt;span class="o"&gt;[&lt;/span&gt;&lt;span class="nt"&gt;type&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s1"&gt;"application/x-google-chrome-pdf"&lt;/span&gt;&lt;span class="o"&gt;]&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nl"&gt;filter&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;invert&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="m"&gt;90%&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="n"&gt;hue-rotate&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="m"&gt;180deg&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That used to be right. When you navigate straight to a PDF today, the document you are styling has&lt;br&gt;
no &lt;code&gt;&amp;lt;embed&amp;gt;&lt;/code&gt; in it. The viewer lives in an out-of-process child frame that your CSS cannot reach.&lt;br&gt;
Your selector matches zero elements and fails silently, which is the worst way for CSS to fail.&lt;/p&gt;

&lt;p&gt;What does reach it is a filter on the root element of the PDF document itself:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight css"&gt;&lt;code&gt;&lt;span class="nt"&gt;html&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nl"&gt;filter&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;invert&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="m"&gt;90%&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="n"&gt;hue-rotate&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="m"&gt;180deg&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt; &lt;span class="nl"&gt;background-color&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="m"&gt;#17171a&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;So the two cases need different code, and you can tell them apart cheaply:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;topLevelPdf&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;document&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;contentType&lt;/span&gt; &lt;span class="o"&gt;===&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;application/pdf&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;css&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;topLevelPdf&lt;/span&gt;
  &lt;span class="p"&gt;?&lt;/span&gt; &lt;span class="s2"&gt;`html { filter: &lt;/span&gt;&lt;span class="p"&gt;${&lt;/span&gt;&lt;span class="nx"&gt;f&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="s2"&gt; !important; background-color: #17171a !important; }`&lt;/span&gt;
  &lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;`embed[type="application/x-google-chrome-pdf"],
     embed[type="application/pdf"],
     object[type="application/pdf"] { filter: &lt;/span&gt;&lt;span class="p"&gt;${&lt;/span&gt;&lt;span class="nx"&gt;f&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="s2"&gt; !important; }`&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;An embedded PDF (a viewer inside a normal page) still wants the second form, because filtering&lt;br&gt;
&lt;code&gt;html&lt;/code&gt; there would invert the surrounding page too.&lt;/p&gt;
&lt;h2&gt;
  
  
  Try it without installing anything
&lt;/h2&gt;

&lt;p&gt;Open a PDF in Chrome, open DevTools on it, and paste this into the console:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="nb"&gt;document&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;documentElement&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;style&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;filter&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;invert(90%) hue-rotate(180deg)&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="nb"&gt;document&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;documentElement&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;style&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;backgroundColor&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;#17171a&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Two notes on the numbers, because they matter more than they look.&lt;/p&gt;

&lt;p&gt;&lt;code&gt;invert(90%)&lt;/code&gt; rather than &lt;code&gt;invert(100%)&lt;/code&gt;: full inversion turns black text into pure white on pure&lt;br&gt;
black, which vibrates unpleasantly at night. 90% lands on a soft grey.&lt;/p&gt;

&lt;p&gt;&lt;code&gt;hue-rotate(180deg)&lt;/code&gt; after the inversion: inverting alone also inverts hue, so a blue hyperlink comes&lt;br&gt;
out orange and a red warning box comes out cyan. Rotating the hue 180° puts colours back where they&lt;br&gt;
started while keeping the lightness flipped. Skip it and PDFs with charts become unreadable.&lt;/p&gt;

&lt;p&gt;Sepia is the same trick with one more stage:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight css"&gt;&lt;code&gt;&lt;span class="nt"&gt;filter&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt; &lt;span class="nt"&gt;invert&lt;/span&gt;&lt;span class="o"&gt;(&lt;/span&gt;&lt;span class="err"&gt;93&lt;/span&gt;&lt;span class="o"&gt;%)&lt;/span&gt; &lt;span class="nt"&gt;hue-rotate&lt;/span&gt;&lt;span class="o"&gt;(&lt;/span&gt;&lt;span class="err"&gt;180&lt;/span&gt;&lt;span class="nt"&gt;deg&lt;/span&gt;&lt;span class="o"&gt;)&lt;/span&gt; &lt;span class="nt"&gt;sepia&lt;/span&gt;&lt;span class="o"&gt;(&lt;/span&gt;&lt;span class="err"&gt;35&lt;/span&gt;&lt;span class="o"&gt;%);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  What this approach cannot do
&lt;/h2&gt;

&lt;p&gt;Be honest about the limits before you build on it:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Photographs and diagrams invert along with the text. A CSS filter has no idea which pixels are
content and which are background. Nobody solves this with &lt;code&gt;filter&lt;/code&gt;; it needs the PDF parsed and
re-rendered.&lt;/li&gt;
&lt;li&gt;DRM-protected PDFs are outside your reach entirely.&lt;/li&gt;
&lt;li&gt;The console trick dies on reload. Persisting it is the entire reason extensions exist here.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  The extension
&lt;/h2&gt;

&lt;p&gt;I got tired of this and wrote the small version: &lt;strong&gt;&lt;a href="https://chromewebstore.google.com/detail/cbdeadhjbdomkldbfedfphhgfaegpboc" rel="noopener noreferrer"&gt;PDF Dark Mode — Night Reader for&lt;br&gt;
PDFs&lt;/a&gt;&lt;/strong&gt;. Disclosure: mine,&lt;br&gt;
free, no account, no network calls at all.&lt;/p&gt;

&lt;p&gt;It does the branch above, so web PDFs work and not only local ones; it uses &lt;code&gt;activeTab&lt;/code&gt; so it reads&lt;br&gt;
one tab when you click it and holds no standing permission on your browsing; the three themes are&lt;br&gt;
soft dark, true black and sepia, with brightness and contrast sliders.&lt;/p&gt;

&lt;p&gt;If you would rather not install a thing, the console snippet is genuinely the whole idea. Take it.&lt;br&gt;
The frustrating part of this category was never the CSS. It was ten extensions all quietly scoped&lt;br&gt;
to &lt;code&gt;file://&lt;/code&gt;.&lt;/p&gt;

</description>
      <category>chrome</category>
      <category>css</category>
      <category>webdev</category>
      <category>productivity</category>
    </item>
    <item>
      <title>Fragrance notes, accords and ratings as JSON, without fighting Fragrantica's Cloudflare</title>
      <dc:creator>Hao</dc:creator>
      <pubDate>Fri, 24 Jul 2026 04:47:54 +0000</pubDate>
      <link>https://dev.to/liuhao04/fragrance-notes-accords-and-ratings-as-json-without-fighting-fragranticas-cloudflare-2gp8</link>
      <guid>https://dev.to/liuhao04/fragrance-notes-accords-and-ratings-as-json-without-fighting-fragranticas-cloudflare-2gp8</guid>
      <description>&lt;p&gt;A dupe/clone finder, a fragrance shop's product enrichment, a rating tracker, a&lt;br&gt;
recommendation engine based on notes: they all need the same raw material, which is&lt;br&gt;
&lt;strong&gt;structured notes, accords, and ratings per fragrance&lt;/strong&gt;, not prose reviews. Demand for that&lt;br&gt;
data is well proven, since Fragrantica scrapers on Apify have &lt;strong&gt;1,700+ combined runs&lt;/strong&gt;&lt;br&gt;
between them. Sourcing it has gotten harder, though, and that is worth understanding before&lt;br&gt;
you pick a source.&lt;/p&gt;
&lt;h2&gt;
  
  
  Why the obvious source (Fragrantica) is the wrong first pick
&lt;/h2&gt;

&lt;p&gt;Fragrantica is the biggest fragrance database, so it's the first thing everyone scrapes.&lt;br&gt;
But it now sits behind &lt;strong&gt;Cloudflare&lt;/strong&gt;, and most of the scrapers built against it are&lt;br&gt;
fighting that wall. You'll find a handful on Apify, several with visibly stale runs or&lt;br&gt;
low success rates, because the site actively blocks plain requests. You end up paying in&lt;br&gt;
proxy cost, retries, and broken runs for data that's conceptually simple: notes, accords,&lt;br&gt;
a rating.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Parfumo&lt;/strong&gt; is the other large, community-driven fragrance encyclopedia, with comparable&lt;br&gt;
scale and depth of data (notes, accords, per-dimension community ratings, perfumer credits),&lt;br&gt;
and it still serves plain, uncached HTML with no Cloudflare challenge. As of this writing it&lt;br&gt;
had &lt;strong&gt;zero dedicated scrapers on Apify&lt;/strong&gt;. So the demand signal for this class of data is&lt;br&gt;
proven (1,700+ runs on the Fragrantica side alone), while almost all of the tooling points&lt;br&gt;
at the one source that is actively fighting back.&lt;/p&gt;
&lt;h2&gt;
  
  
  What the source data looks like, cleaned up
&lt;/h2&gt;

&lt;p&gt;One record per fragrance, real output for Dior Sauvage EDT:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"name"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Sauvage"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"brand"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Dior"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"releaseYear"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;2015&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"concentration"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Eau de Toilette"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"gender"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"men"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"rating"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"value"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mf"&gt;7.5&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"best"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"ratingCount"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;6514&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"reviewCount"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;365&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"perfumers"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"François Demachy"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"accords"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"Fresh"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Spicy"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Synthetic"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Citrus"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Woody"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"notes"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"Ambrox"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Sichuan pepper"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Calabrian bergamot"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Pink pepper"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"..."&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"communityRatings"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"scent"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt;     &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"average"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mf"&gt;7.30&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"votes"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;6514&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"longevity"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"average"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mf"&gt;7.79&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"votes"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;6129&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"sillage"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt;   &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"average"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mf"&gt;7.73&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"votes"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;6106&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"bottle"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"average"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mf"&gt;7.72&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"votes"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;6164&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"value"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt;     &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"average"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mf"&gt;7.40&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"votes"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;4561&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;When a page groups notes into a pyramid (most designer releases do), you also get:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="nl"&gt;"notesPyramid"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"top"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt;   &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"Bergamot"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Apple"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Blackcurrant"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Lemon"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Pink pepper"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"heart"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"Pineapple"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Indonesian patchouli"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Jasmine"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"base"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"Birch"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Cedarwood"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Musk"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Oakmoss"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Ambergris"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The pyramid, rather than a flat note list, is what a dupe/clone finder actually needs. Two&lt;br&gt;
fragrances can share a base note and read completely differently because their top notes&lt;br&gt;
diverge, so comparing pyramids beats comparing flat sets.&lt;/p&gt;
&lt;h2&gt;
  
  
  Getting it without hand-copying spec pages
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://apify.com/minty_modesty/parfumo-perfume-data-scraper" rel="noopener noreferrer"&gt;&lt;strong&gt;Parfumo Fragrance Scraper&lt;/strong&gt;&lt;/a&gt;&lt;br&gt;
on Apify takes a list of perfume URLs (or &lt;code&gt;Brand/slug&lt;/code&gt; pairs) and returns one clean record&lt;br&gt;
per fragrance:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl &lt;span class="nt"&gt;-s&lt;/span&gt; &lt;span class="nt"&gt;-X&lt;/span&gt; POST &lt;span class="s2"&gt;"https://api.apify.com/v2/acts/minty_modesty~parfumo-perfume-data-scraper/runs?token=&lt;/span&gt;&lt;span class="nv"&gt;$APIFY_TOKEN&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;-H&lt;/span&gt; &lt;span class="s1"&gt;'content-type: application/json'&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;-d&lt;/span&gt; &lt;span class="s1"&gt;'{"perfumeUrls":["Dior/sauvage","Creed/aventus","Chanel/No_5"],"includeCommunityRatings":true}'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;It reads only public &lt;code&gt;/Perfumes/&lt;/code&gt; pages, which Parfumo's &lt;code&gt;robots.txt&lt;/code&gt; allows. No login, no&lt;br&gt;
CAPTCHA, no Cloudflare challenge to route around, so a run doesn't turn into a proxy bill.&lt;br&gt;
URLs that don't resolve come back as &lt;code&gt;{"url": "...", "error": "not_found"}&lt;/code&gt; instead of&lt;br&gt;
silently dropping the line item, so a batch of a few hundred references doesn't need&lt;br&gt;
babysitting.&lt;/p&gt;

&lt;h2&gt;
  
  
  When this is (and isn't) the right layer
&lt;/h2&gt;

&lt;p&gt;If you already have a working Fragrantica pipeline and it isn't breaking, there's no reason&lt;br&gt;
to switch sources mid-project. But if you're starting a new dupe-finder, enrichment job, or&lt;br&gt;
rating tracker from scratch, the 1,700+ runs on the Fragrantica side tell you the &lt;em&gt;data&lt;/em&gt; is&lt;br&gt;
worth having. They don't tell you Fragrantica is the only place to get it. Parfumo is the&lt;br&gt;
same class of data from a source that isn't actively trying to block you from reading it.&lt;/p&gt;

</description>
      <category>api</category>
      <category>webscraping</category>
      <category>javascript</category>
      <category>datasets</category>
    </item>
    <item>
      <title>Turn a watch reference number into full specs (caliber, case, dial) as JSON</title>
      <dc:creator>Hao</dc:creator>
      <pubDate>Fri, 24 Jul 2026 04:18:32 +0000</pubDate>
      <link>https://dev.to/liuhao04/turn-a-watch-reference-number-into-full-specs-caliber-case-dial-as-json-g0g</link>
      <guid>https://dev.to/liuhao04/turn-a-watch-reference-number-into-full-specs-caliber-case-dial-as-json-g0g</guid>
      <description>&lt;p&gt;A marketplace, a valuation tool, an insurance quoting flow, a collection tracker: build&lt;br&gt;
anything around watch data and you eventually hit the same wall. You have a &lt;strong&gt;reference&lt;br&gt;
number&lt;/strong&gt; (&lt;code&gt;126500LN-0001&lt;/code&gt;, scraped from a listing or copied off an invoice) and you need the&lt;br&gt;
&lt;strong&gt;spec sheet behind it&lt;/strong&gt;: movement/caliber, case material, dimensions, water resistance,&lt;br&gt;
dial color and finish. The listing itself never has all of this. Where&lt;br&gt;
does it actually live, and how do you get it as clean data instead of copy-pasting spec&lt;br&gt;
tables by hand?&lt;/p&gt;
&lt;h2&gt;
  
  
  The gap in the watch-data ecosystem
&lt;/h2&gt;

&lt;p&gt;Watch &lt;strong&gt;marketplace&lt;/strong&gt; scraping is a solved, proven market. Chrono24 scrapers on Apify have&lt;br&gt;
&lt;strong&gt;16,000+ runs&lt;/strong&gt;, so people clearly pay for watch listing data. But a listing gives you&lt;br&gt;
price, seller, condition, and a reference number. It does not give you the manufacturer's&lt;br&gt;
full technical spec for that reference. That data lives on dedicated&lt;br&gt;
reference databases, not on the marketplaces themselves.&lt;/p&gt;

&lt;p&gt;The reference-grade source for this is &lt;strong&gt;WatchBase&lt;/strong&gt;, a structured database organized&lt;br&gt;
brand → model family → individual reference, with every spec Rolex, Omega, Seiko, and&lt;br&gt;
hundreds of other brands publish for each watch. It has no public API and no search&lt;br&gt;
endpoint, only a browsable page hierarchy, which is exactly why this data usually ends&lt;br&gt;
up copy-pasted into spreadsheets one watch at a time.&lt;/p&gt;
&lt;h2&gt;
  
  
  What the source data looks like, cleaned up
&lt;/h2&gt;

&lt;p&gt;One record per reference, normalized to the fields you actually compare across watches:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"brand"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Rolex"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"family"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Daytona"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"reference"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"126500LN-0001"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"name"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Cosmograph Daytona Stainless Steel - Cerachrom / White / Oyster"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"caliber"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Rolex caliber 4131"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"movement"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Rolex caliber 4131 Hours, Minutes, Small Seconds | Chronograph, Column wheel | Chronometer"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"material"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Ceramic, Stainless Steel"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"glass"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Sapphire"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"diameterMm"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;40&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"heightMm"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mf"&gt;11.9&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"waterResistanceM"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;100&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"dialColor"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"White"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"indexes"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Stick / Dot"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"specs"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"..."&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"every raw label/value pair WatchBase publishes for this reference"&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The &lt;code&gt;specs&lt;/code&gt; object matters as much as the normalized top-level fields: field coverage&lt;br&gt;
varies a lot by watch (a current Rolex has far more documented fields than a 1960s&lt;br&gt;
piece), and WatchBase's own labels vary slightly between brands (&lt;code&gt;Material&lt;/code&gt; vs&lt;br&gt;
&lt;code&gt;Materials&lt;/code&gt;). Keeping the raw label/value pairs alongside the normalized fields means you&lt;br&gt;
never lose data to an over-eager parser.&lt;/p&gt;
&lt;h2&gt;
  
  
  Getting it without hand-copying spec tables
&lt;/h2&gt;

&lt;p&gt;Rather than walking WatchBase's brand → family → reference hierarchy yourself, you can&lt;br&gt;
run a hosted actor that already does the discovery and normalization:&lt;br&gt;
&lt;a href="https://apify.com/minty_modesty/watchbase-scraper" rel="noopener noreferrer"&gt;&lt;strong&gt;WatchBase Scraper&lt;/strong&gt;&lt;/a&gt; on Apify.&lt;/p&gt;

&lt;p&gt;Give it a brand (or a list of specific reference URLs if you already have them):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"brands"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"rolex"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"maxItems"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;200&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"includeMovement"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"includeCaseSpecs"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"includeDialSpecs"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl &lt;span class="nt"&gt;-s&lt;/span&gt; &lt;span class="nt"&gt;-X&lt;/span&gt; POST &lt;span class="s2"&gt;"https://api.apify.com/v2/acts/minty_modesty~watchbase-scraper/runs?token=&lt;/span&gt;&lt;span class="nv"&gt;$APIFY_TOKEN&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;-H&lt;/span&gt; &lt;span class="s1"&gt;'content-type: application/json'&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;-d&lt;/span&gt; &lt;span class="s1"&gt;'{"brands":["rolex"],"maxItems":200}'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;It reads only public reference pages (plain server-rendered HTML, allowed by&lt;br&gt;
&lt;code&gt;robots.txt&lt;/code&gt;, no login) and returns one clean record per watch. If you already have&lt;br&gt;
reference numbers from a marketplace dataset, pass &lt;code&gt;referenceUrls&lt;/code&gt; instead of a whole&lt;br&gt;
brand and skip straight to the records you need.&lt;/p&gt;

&lt;h2&gt;
  
  
  When this is (and isn't) the right layer
&lt;/h2&gt;

&lt;p&gt;If you only need one or two watches, just open the WatchBase page. If you're enriching a&lt;br&gt;
marketplace dataset in bulk, hydrating thousands of Chrono24-style listings with the&lt;br&gt;
manufacturer spec behind each reference, that is exactly the gap this fills. Listing&lt;br&gt;
scrapers get you price and reference number; this gets you everything after that.&lt;/p&gt;

</description>
      <category>api</category>
      <category>webscraping</category>
      <category>javascript</category>
      <category>datasets</category>
    </item>
    <item>
      <title>How to get LCSC component prices and stock as JSON (no login, no API key)</title>
      <dc:creator>Hao</dc:creator>
      <pubDate>Thu, 23 Jul 2026 18:23:28 +0000</pubDate>
      <link>https://dev.to/liuhao04/how-to-get-lcsc-component-prices-and-stock-as-json-no-login-no-api-key-2ekb</link>
      <guid>https://dev.to/liuhao04/how-to-get-lcsc-component-prices-and-stock-as-json-no-login-no-api-key-2ekb</guid>
      <description>&lt;p&gt;If you build hardware, you have hit this: you have a BOM full of LCSC part codes&lt;br&gt;
(&lt;code&gt;C1591&lt;/code&gt;, &lt;code&gt;C25804&lt;/code&gt;, the &lt;code&gt;C#####&lt;/code&gt; numbers on every LCSC and JLCPCB line item), and you&lt;br&gt;
want the &lt;strong&gt;live price ladder and stock&lt;/strong&gt; as structured data, so you can cost a build,&lt;br&gt;
watch prices, or enrich a component database. LCSC has no public pricing API, and&lt;br&gt;
the search endpoints are protected. So how do you get the data cleanly?&lt;/p&gt;

&lt;p&gt;This post shows the pragmatic approach, why the obvious ones break, and a zero-setup&lt;br&gt;
way to get one clean JSON record per part.&lt;/p&gt;
&lt;h2&gt;
  
  
  Why the obvious approaches break
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Scraping the search API.&lt;/strong&gt; It is rate-limited and CAPTCHA-protected; reverse-
engineering it means fighting anti-bot systems that change often. High-maintenance.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Headless-browser clicking.&lt;/strong&gt; Heavy, slow, and brittle against layout changes.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Manual copy-paste.&lt;/strong&gt; Fine for three parts, not for a 200-line BOM you re-cost weekly.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The stable surface is the &lt;strong&gt;public product-detail page&lt;/strong&gt; (&lt;code&gt;/product-detail/...&lt;/code&gt;). It is&lt;br&gt;
allowed by LCSC's &lt;code&gt;robots.txt&lt;/code&gt;, it carries the full data (price tiers, stock, package,&lt;br&gt;
MOQ, parametric specs, datasheet URL) in the page's own JSON, and it does not require a&lt;br&gt;
login. Read &lt;em&gt;that&lt;/em&gt;, one lightweight request per part, and you get reliable results&lt;br&gt;
without an anti-bot arms race.&lt;/p&gt;
&lt;h2&gt;
  
  
  The data you actually want, per part
&lt;/h2&gt;


&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"productCode"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"C1591"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"mpn"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"CL10B104KB8NNNC"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"brand"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Samsung"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"description"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"100nF ±10% 50V X7R 0603 MLCC"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"stock"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;512340&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"package"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"0603"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"minOrderQty"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;50&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"priceTiers"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"qty"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;50&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"usd"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mf"&gt;0.0019&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"qty"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;500&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"usd"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mf"&gt;0.0011&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"qty"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;5000&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"usd"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mf"&gt;0.0007&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"datasheetUrl"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"https://..."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"url"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"https://www.lcsc.com/product-detail/C1591.html"&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;That is spreadsheet-ready and diff-able: store it, re-fetch weekly, and you can watch&lt;br&gt;
price and stock move over time.&lt;/p&gt;
&lt;h2&gt;
  
  
  The zero-setup route
&lt;/h2&gt;

&lt;p&gt;Rather than maintain scraper code yourself, you can run a hosted actor that does exactly&lt;br&gt;
the above. It reads only public product pages and returns one clean record per code:&lt;br&gt;
&lt;a href="https://apify.com/minty_modesty/lcsc-parts-scraper" rel="noopener noreferrer"&gt;&lt;strong&gt;LCSC Parts Scraper&lt;/strong&gt;&lt;/a&gt; on Apify.&lt;/p&gt;

&lt;p&gt;Input is just your list of codes:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"productCodes"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"C1591"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"C25804"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"C7593"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"includePriceTiers"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"includeSpecs"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Call it from the API / CLI and pull the dataset as JSON or CSV:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# start a run and wait, then fetch the dataset&lt;/span&gt;
curl &lt;span class="nt"&gt;-s&lt;/span&gt; &lt;span class="nt"&gt;-X&lt;/span&gt; POST &lt;span class="s2"&gt;"https://api.apify.com/v2/acts/minty_modesty~lcsc-parts-scraper/runs?token=&lt;/span&gt;&lt;span class="nv"&gt;$APIFY_TOKEN&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;-H&lt;/span&gt; &lt;span class="s1"&gt;'content-type: application/json'&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;-d&lt;/span&gt; &lt;span class="s1"&gt;'{"productCodes":["C1591","C25804"],"includePriceTiers":true}'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Because it reads public pages at low volume, it keeps working where search-API scrapers&lt;br&gt;
get blocked. Feed it your BOM, get back cost data.&lt;/p&gt;

&lt;h2&gt;
  
  
  When to roll your own instead
&lt;/h2&gt;

&lt;p&gt;If you only ever need a couple of parts, just open the product page. If you need&lt;br&gt;
&lt;strong&gt;keyword → part codes&lt;/strong&gt;, resolve codes first (from your BOM or on lcsc.com), because this&lt;br&gt;
approach is deliberately code-first to stay on stable pages. And if you need account-&lt;br&gt;
specific contract pricing, that lives behind your LCSC login and no public method will&lt;br&gt;
reach it.&lt;/p&gt;

&lt;p&gt;For the common case, turning a list of LCSC codes into clean, structured price/stock/spec&lt;br&gt;
data for BOM costing or price monitoring, reading the public product pages is the reliable&lt;br&gt;
path, and a hosted actor removes the maintenance entirely.&lt;/p&gt;

</description>
      <category>electronics</category>
      <category>api</category>
      <category>webscraping</category>
      <category>javascript</category>
    </item>
  </channel>
</rss>
