<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: AI Viewz</title>
    <description>The latest articles on DEV Community by AI Viewz (@aiviewz_team).</description>
    <link>https://dev.to/aiviewz_team</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3389193%2F55749656-22da-4b63-a82c-510d6ca99628.png</url>
      <title>DEV Community: AI Viewz</title>
      <link>https://dev.to/aiviewz_team</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/aiviewz_team"/>
    <language>en</language>
    <item>
      <title>How I Built a Multi-Core Client-Side Video Clipper &amp; YouTube-to-Shorts Studio in Pure WebAssembly (Zero Server Cost)</title>
      <dc:creator>AI Viewz</dc:creator>
      <pubDate>Sun, 01 Nov 2026 06:32:45 +0000</pubDate>
      <link>https://dev.to/aiviewz_team/how-i-built-a-multi-core-client-side-video-clipper-youtube-to-shorts-studio-in-pure-webassembly-1h9k</link>
      <guid>https://dev.to/aiviewz_team/how-i-built-a-multi-core-client-side-video-clipper-youtube-to-shorts-studio-in-pure-webassembly-1h9k</guid>
      <description>&lt;p&gt;Most online video clippers and social media downloaders follow the same playbook: force users to create an account, queue their video on an expensive cloud GPU instance, slap a watermark on the output, and charge $19/month once they cut more than three clips.&lt;/p&gt;

&lt;p&gt;I wanted to see how far we could push modern browser APIs—specifically &lt;strong&gt;WebAssembly (&lt;code&gt;ffmpeg.wasm&lt;/code&gt;)&lt;/strong&gt;, &lt;strong&gt;Web Worker Thread Pools&lt;/strong&gt;, &lt;strong&gt;HTTP Range Chunking&lt;/strong&gt;, and &lt;strong&gt;HTML5 Canvas Compositing&lt;/strong&gt;—to build a 100% free, zero-server-cost alternative that runs entirely on the user's device.&lt;/p&gt;

&lt;p&gt;The result is &lt;strong&gt;&lt;a href="https://download-reels.github.io/" rel="noopener noreferrer"&gt;Download Reels &amp;amp; Multi-Reel Clipper Studio&lt;/a&gt;&lt;/strong&gt; — a privacy-first web app hosted on GitHub Pages that lets creators download videos and slice long videos into multiple vertical Shorts/Reels simultaneously across their CPU cores.&lt;/p&gt;

&lt;p&gt;Here is how the architecture works under the hood.&lt;/p&gt;




&lt;h2&gt;
  
  
  🏗️ 1. Solving the &lt;code&gt;SharedArrayBuffer&lt;/code&gt; Problem on GitHub Pages
&lt;/h2&gt;

&lt;p&gt;If you have ever tried running &lt;code&gt;@ffmpeg/ffmpeg&lt;/code&gt; in a static browser environment like GitHub Pages, you have likely hit this infamous error:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;ReferenceError: SharedArrayBuffer is not defined
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;Multi-threaded WebAssembly (&lt;code&gt;@ffmpeg/core-mt&lt;/code&gt;) requires strict Cross-Origin Isolation headers (&lt;code&gt;Cross-Origin-Opener-Policy: same-origin&lt;/code&gt; and &lt;code&gt;Cross-Origin-Embedder-Policy: require-corp&lt;/code&gt;). However, turning on strict &lt;code&gt;COEP&lt;/code&gt; often breaks cross-origin &lt;code&gt;&amp;lt;video&amp;gt;&lt;/code&gt; previews and CDN media streams that don't send CORP headers.&lt;/p&gt;
&lt;h3&gt;
  
  
  Our Solution: Multi-Worker Single-Threaded Pool (&lt;code&gt;@ffmpeg/core-st&lt;/code&gt;)
&lt;/h3&gt;

&lt;p&gt;Instead of relying on a single multi-threaded WASM instance that requires &lt;code&gt;SharedArrayBuffer&lt;/code&gt;, we built a &lt;strong&gt;Web Worker Pool&lt;/strong&gt; (&lt;a href="https://github.com/download-reels/download-reels.github.io/blob/main/js/clipper-worker.js" rel="noopener noreferrer"&gt;&lt;code&gt;js/clipper-worker.js&lt;/code&gt;&lt;/a&gt;) where each Web Worker boots an isolated single-threaded &lt;code&gt;@ffmpeg/core-st@0.11.1&lt;/code&gt; instance:&lt;br&gt;
&lt;/p&gt;
&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="c1"&gt;// Size the pool based on hardware CPU cores (up to 6 parallel workers)&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;hwCores&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;navigator&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;hardwareConcurrency&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;desiredWorkers&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;Math&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;min&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;6&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nb"&gt;Math&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;max&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;hwCores&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;));&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;When a creator queues &lt;strong&gt;5 timestamp clips&lt;/strong&gt; in the &lt;strong&gt;&lt;a href="https://download-reels.github.io/youtube-video-cutter.html" rel="noopener noreferrer"&gt;YouTube Video Cutter &amp;amp; Multi-Reel Splitter&lt;/a&gt;&lt;/strong&gt;, the main thread dispatches all 5 segments across the worker pool in parallel. Each CPU core slices its own clip simultaneously without needing &lt;code&gt;SharedArrayBuffer&lt;/code&gt;!&lt;/p&gt;


&lt;h2&gt;
  
  
  ⚡ 2. 4× Faster Downloads with Parallel HTTP &lt;code&gt;Range&lt;/code&gt; Chunking &amp;amp; &lt;code&gt;moov&lt;/code&gt; Prefix Truncation
&lt;/h2&gt;

&lt;p&gt;Before FFmpeg can cut a clip in the browser, it needs the source video buffer in memory. Downloading a 45-minute podcast over a single HTTP stream just to cut a 30-second clip at &lt;code&gt;01:15 – 01:45&lt;/code&gt; is a massive waste of bandwidth.&lt;/p&gt;

&lt;p&gt;We solved this in two ways inside &lt;a href="https://github.com/download-reels/download-reels.github.io/blob/main/js/converter.js" rel="noopener noreferrer"&gt;&lt;code&gt;js/converter.js&lt;/code&gt;&lt;/a&gt;:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Parallel &lt;code&gt;Range&lt;/code&gt; Requests&lt;/strong&gt;: We send a &lt;code&gt;HEAD&lt;/code&gt; request to inspect &lt;code&gt;Accept-Ranges: bytes&lt;/code&gt; and &lt;code&gt;Content-Length&lt;/code&gt;, then split the download into &lt;strong&gt;4 concurrent &lt;code&gt;Range: bytes=start-end&lt;/code&gt; streams&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Smart &lt;code&gt;moov&lt;/code&gt; Prefix Truncation&lt;/strong&gt;: For web-optimized MP4s (&lt;code&gt;faststart&lt;/code&gt;, where the &lt;code&gt;moov&lt;/code&gt; atom sits at the start of the file), if the user's latest clip ends at &lt;code&gt;02:00&lt;/code&gt; of a &lt;code&gt;30:00&lt;/code&gt; video, we estimate the byte offset with a safety margin and &lt;strong&gt;only fetch the first ~12% of the file&lt;/strong&gt;!
&lt;/li&gt;
&lt;/ol&gt;
&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="c1"&gt;// Estimate prefix byte ceiling when cutting early segments of a long MP4&lt;/span&gt;
&lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;maxEndTimeSec&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="nx"&gt;durationSec&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="nx"&gt;totalBytes&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;5&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;1024&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;1024&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;ratio&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;Math&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;min&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;maxEndTimeSec&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mf"&gt;1.35&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="mi"&gt;15&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;/&lt;/span&gt; &lt;span class="nx"&gt;durationSec&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;ratio&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="mf"&gt;0.85&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nx"&gt;targetBytes&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;Math&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;max&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;1024&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;1024&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nb"&gt;Math&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;ceil&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;totalBytes&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="nx"&gt;ratio&lt;/span&gt;&lt;span class="p"&gt;));&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  🎯 3. Instant Keyframe Seeking (&lt;code&gt;-ss&lt;/code&gt; Before &lt;code&gt;-i&lt;/code&gt;)
&lt;/h2&gt;

&lt;p&gt;A common mistake when using FFmpeg to trim videos is placing &lt;code&gt;-ss&lt;/code&gt; after &lt;code&gt;-i&lt;/code&gt;, which forces FFmpeg to decode every frame from &lt;code&gt;00:00:00&lt;/code&gt; up to your start timestamp.&lt;/p&gt;

&lt;p&gt;By placing &lt;code&gt;-ss &amp;lt;start&amp;gt;&lt;/code&gt; &lt;strong&gt;before&lt;/strong&gt; &lt;code&gt;-i&lt;/code&gt; and using &lt;code&gt;-t &amp;lt;duration&amp;gt;&lt;/code&gt; with stream copy (&lt;code&gt;-c copy&lt;/code&gt;), cutting a 30-second clip takes &lt;strong&gt;less than 200ms&lt;/strong&gt; in WebAssembly:&lt;br&gt;
&lt;/p&gt;
&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;ffmpeg &lt;span class="nt"&gt;-ss&lt;/span&gt; 45.00 &lt;span class="nt"&gt;-i&lt;/span&gt; source.mp4 &lt;span class="nt"&gt;-t&lt;/span&gt; 30.00 &lt;span class="nt"&gt;-c&lt;/span&gt; copy &lt;span class="nt"&gt;-avoid_negative_ts&lt;/span&gt; make_zero reel_1.mp4
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;If the user enables our &lt;strong&gt;1st-Second Viral Hook Banner&lt;/strong&gt; or &lt;strong&gt;Custom Logo Watermark&lt;/strong&gt;, we composite the overlay onto a high-DPI HTML5 &lt;code&gt;&amp;lt;canvas&amp;gt;&lt;/code&gt;, export a transparent PNG into WASM virtual memory (&lt;code&gt;FS.writeFile&lt;/code&gt;), and apply FFmpeg's &lt;code&gt;overlay&lt;/code&gt; filter only when requested.&lt;/p&gt;


&lt;h2&gt;
  
  
  🔗 Try the Live Tools (100% Free, No Signup, Open Source)
&lt;/h2&gt;

&lt;p&gt;We broke the platform into dedicated, fast-loading tools for each workflow:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;🎬 &lt;strong&gt;&lt;a href="https://download-reels.github.io/" rel="noopener noreferrer"&gt;Download Reels &amp;amp; Multi-Reel Clipper Studio (Main App)&lt;/a&gt;&lt;/strong&gt; — All-in-one video saver, batch queue, and multi-reel splitter.&lt;/li&gt;
&lt;li&gt;✂️ &lt;strong&gt;&lt;a href="https://download-reels.github.io/youtube-video-cutter.html" rel="noopener noreferrer"&gt;YouTube Video Cutter &amp;amp; Convert YouTube to Shorts&lt;/a&gt;&lt;/strong&gt; — Cut YouTube videos by timestamp into multiple Shorts &amp;amp; Reels simultaneously with 1st-second hook banners and ZIP export.&lt;/li&gt;
&lt;li&gt;🎵 &lt;strong&gt;&lt;a href="https://download-reels.github.io/youtube-to-mp3.html" rel="noopener noreferrer"&gt;YouTube to MP3 320kbps Audio Converter&lt;/a&gt;&lt;/strong&gt; — Extract studio-grade 320kbps / 192kbps MP3 audio or lossless WAV directly in your browser via WebAssembly.&lt;/li&gt;
&lt;li&gt;📸 &lt;strong&gt;&lt;a href="https://download-reels.github.io/instagram-reel-downloader.html" rel="noopener noreferrer"&gt;Instagram Reel Downloader (1080p HD MP4)&lt;/a&gt;&lt;/strong&gt; — Save public Instagram Reels and audio tracks in original 1080p MP4 quality.&lt;/li&gt;
&lt;li&gt;📱 &lt;strong&gt;&lt;a href="https://download-reels.github.io/tiktok-video-downloader.html" rel="noopener noreferrer"&gt;TikTok Video Downloader Without Watermark&lt;/a&gt;&lt;/strong&gt; — Download clean, unwatermarked TikTok MP4 videos and original sounds.&lt;/li&gt;
&lt;/ul&gt;
&lt;h3&gt;
  
  
  ⭐ Check Out the Code on GitHub
&lt;/h3&gt;

&lt;p&gt;The entire frontend, Web Worker pool, and Cloudflare CORS proxy worker are open source. Feel free to star the repo, open feature requests, or fork it for your own media tools:&lt;/p&gt;


&lt;div class="ltag-github-readme-tag"&gt;
  &lt;div class="readme-overview"&gt;
    &lt;h2&gt;
      &lt;img src="https://assets.dev.to/assets/github-logo-5a155e1f9a670af7944dd5e12375bc76ed542ea80224905ecaf878b9157cdefc.svg" alt="GitHub logo"&gt;
      &lt;a href="https://github.com/download-reels" rel="noopener noreferrer"&gt;
        download-reels
      &lt;/a&gt; / &lt;a href="https://github.com/download-reels/download-reels.github.io" rel="noopener noreferrer"&gt;
        download-reels.github.io
      &lt;/a&gt;
    &lt;/h2&gt;
    &lt;h3&gt;
      Download Your Favourite Videos and Musics From Instagram, Youtube, Facebook , Tiktok and Other Platforms
    &lt;/h3&gt;
  &lt;/div&gt;
  &lt;div class="ltag-github-body"&gt;
    
&lt;div id="readme" class="md"&gt;&lt;div class="markdown-heading"&gt;
&lt;h1 class="heading-element"&gt;Download Reels — Universal Video Downloader &amp;amp; YouTube to MP3&lt;/h1&gt;
&lt;/div&gt;

&lt;blockquote&gt;
&lt;p&gt;The clean, ad-free, high-speed open-source alternative to SaveFrom.net. Download Instagram Reels, TikTok without watermarks, YouTube Shorts, X/Twitter, and convert to 320kbps MP3 directly in your browser.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;🌐 &lt;strong&gt;Live Web Utility&lt;/strong&gt;: &lt;a href="https://download-reels.github.io/" rel="nofollow noopener noreferrer"&gt;&lt;strong&gt;https://download-reels.github.io/&lt;/strong&gt;&lt;/a&gt;&lt;/p&gt;




&lt;div class="markdown-heading"&gt;
&lt;h2 class="heading-element"&gt;💡 Why This Project Was Created&lt;/h2&gt;
&lt;/div&gt;

&lt;p&gt;For over fifteen years, &lt;strong&gt;SaveFrom.net&lt;/strong&gt; was one of the most visited websites on the internet, relied on daily by hundreds of millions of students, editors, educators, journalists, and everyday internet users to save educational clips, archive memories, and download audio.&lt;/p&gt;

&lt;p&gt;With &lt;strong&gt;SaveFrom.net shutting down&lt;/strong&gt;, a massive void was left in the web ecosystem. In its place, the search results have been overrun by predatory download portals filled with:&lt;/p&gt;


&lt;ul&gt;

&lt;li&gt;

&lt;strong&gt;Intrusive and deceptive advertising&lt;/strong&gt;: Aggressive pop-unders, fake "virus detected" alerts, and unwanted notifications.&lt;/li&gt;

&lt;li&gt;

&lt;strong&gt;Paywalls &amp;amp; Artificial Throttling&lt;/strong&gt;: Demanding subscription fees or forcing users to wait 30 seconds in artificial queues to…&lt;/li&gt;

&lt;/ul&gt;&lt;/div&gt;
&lt;br&gt;
  &lt;/div&gt;
&lt;br&gt;
  &lt;div class="gh-btn-container"&gt;&lt;a class="gh-btn" href="https://github.com/download-reels/download-reels.github.io" rel="noopener noreferrer"&gt;View on GitHub&lt;/a&gt;&lt;/div&gt;
&lt;br&gt;
&lt;/div&gt;
&lt;br&gt;


&lt;p&gt;What browser-side media features would you like to see next (e.g., WebGPU Whisper auto-captions or 9:16 face-tracking crop)? Let me know in the comments!&lt;/p&gt;

</description>
      <category>webdev</category>
      <category>javascript</category>
      <category>opensource</category>
      <category>video</category>
    </item>
    <item>
      <title>Tesseract OCR 5.5 Installation</title>
      <dc:creator>AI Viewz</dc:creator>
      <pubDate>Fri, 26 Sep 2025 21:43:31 +0000</pubDate>
      <link>https://dev.to/aiviewz_team/tesseract-ocr-55-installation-384g</link>
      <guid>https://dev.to/aiviewz_team/tesseract-ocr-55-installation-384g</guid>
      <description>&lt;h2&gt;
  
  
  How To Install Latest Version of Tesseract OCR 5.5
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;sudo add-apt-repository ppa:alex-p/tesseract-ocr5.4
sudo apt update

sudo add-apt-repository ppa:alex-p/tesseract-ocr-devel
sudo apt update

sudo apt install tesseract-ocr-eng   # For English
sudo apt install tesseract-ocr-ara   # For Arabic
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;We Have a Very Detailed Explanation on &lt;a href="https://www.aiviewz.com/posts/how-to-install-latest-version-5-4-of-tesseract-ocr" rel="noopener noreferrer"&gt;Tesseract Installation &lt;/a&gt;&lt;/p&gt;

</description>
      <category>programming</category>
      <category>ai</category>
      <category>ocr</category>
      <category>tutorial</category>
    </item>
    <item>
      <title>Generating Synthetic RTL OCR Data for Donut with SynthDoG-RTL</title>
      <dc:creator>AI Viewz</dc:creator>
      <pubDate>Tue, 23 Sep 2025 20:11:56 +0000</pubDate>
      <link>https://dev.to/aiviewz_team/generating-synthetic-rtl-ocr-data-for-donut-with-synthdog-rtl-3ghi</link>
      <guid>https://dev.to/aiviewz_team/generating-synthetic-rtl-ocr-data-for-donut-with-synthdog-rtl-3ghi</guid>
      <description>&lt;h2&gt;
  
  
  Introduction
&lt;/h2&gt;

&lt;p&gt;Building OCR models for right-to-left (RTL) languages like Arabic, Urdu, Persian, or Hebrew often suffers from a lack of annotated training data. &lt;strong&gt;SynthDoG-RTL&lt;/strong&gt; is a synthetic document generator adapted from Donut’s SynthDoG, extended to handle RTL text rendering correctly. In this post, we’ll walk through how advanced developers can generate large-scale synthetic datasets compatible with &lt;a href="https://github.com/clovaai/donut" rel="noopener noreferrer"&gt;Donut&lt;/a&gt;.&lt;/p&gt;




&lt;h2&gt;
  
  
  What is SynthDoG-RTL?
&lt;/h2&gt;

&lt;p&gt;SynthDoG (Synthetic Document Generator) was introduced with Donut to create training data on the fly for document understanding. &lt;a href="https://github.com/aiviewz/Synthdog-RTL" rel="noopener noreferrer"&gt;SynthDoG-RTL&lt;/a&gt; extends it by:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Supporting RTL text direction and contextual script shaping.&lt;/li&gt;
&lt;li&gt;Including sample corpora, fonts, and templates for Arabic, Urdu, Persian, Hebrew, and others.&lt;/li&gt;
&lt;li&gt;Allowing custom YAML configuration for layouts, distortions, and effects.&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  Installation and Setup
&lt;/h2&gt;

&lt;p&gt;Clone the repository and install dependencies:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;git clone https://github.com/aiviewz/Synthdog-RTL.git
&lt;span class="nb"&gt;cd &lt;/span&gt;Synthdog-RTL

conda create &lt;span class="nt"&gt;-n&lt;/span&gt; synthdog &lt;span class="nv"&gt;python&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;3.8 &lt;span class="nt"&gt;-y&lt;/span&gt;
conda activate synthdog
pip &lt;span class="nb"&gt;install &lt;/span&gt;synthtiger
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Make sure to install &lt;a href="https://github.com/HOST-Oman/libraqm" rel="noopener noreferrer"&gt;libraqm&lt;/a&gt; for proper Arabic/RTL shaping:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nb"&gt;sudo &lt;/span&gt;apt-get &lt;span class="nb"&gt;install &lt;/span&gt;libfreetype6-dev libharfbuzz-dev
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;On macOS, set:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nb"&gt;export &lt;/span&gt;&lt;span class="nv"&gt;OBJC_DISABLE_INITIALIZE_FORK_SAFETY&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;YES
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  Preparing Resources
&lt;/h2&gt;

&lt;p&gt;Each language needs:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Corpus&lt;/strong&gt;: UTF-8 text file under &lt;code&gt;resources/corpus/&lt;/code&gt; (e.g., &lt;code&gt;urdu.txt&lt;/code&gt;, &lt;code&gt;arabic.txt&lt;/code&gt;).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Fonts&lt;/strong&gt;: Place &lt;code&gt;.ttf/.otf&lt;/code&gt; fonts in &lt;code&gt;resources/font/&amp;lt;lang_code&amp;gt;/&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Backgrounds&lt;/strong&gt;: Optional textures under &lt;code&gt;resources/backgrounds/&lt;/code&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Example structure:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;resources/
 ├─ corpus/
 │   ├─ urdu.txt
 │   └─ arabic.txt
 └─ font/
     ├─ ur/
     │   └─ NotoNastaliq.ttf
     └─ ar/
         └─ NotoNaskh.ttf
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  Configuring Generation
&lt;/h2&gt;

&lt;p&gt;YAML config files (e.g., &lt;code&gt;config_ur.yaml&lt;/code&gt;) define page size, font range, distortions, and paths.&lt;/p&gt;

&lt;p&gt;Example Urdu config:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;corpus_path&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;resources/corpus/urdu.txt"&lt;/span&gt;
&lt;span class="na"&gt;font_dir&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;resources/font/ur/"&lt;/span&gt;
&lt;span class="na"&gt;page_width&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;1240&lt;/span&gt;
&lt;span class="na"&gt;page_height&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;1754&lt;/span&gt;
&lt;span class="na"&gt;min_font_size&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;20&lt;/span&gt;
&lt;span class="na"&gt;max_font_size&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="m"&gt;40&lt;/span&gt;
&lt;span class="na"&gt;rotate_angle&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="pi"&gt;[&lt;/span&gt;&lt;span class="nv"&gt;-2&lt;/span&gt;&lt;span class="pi"&gt;,&lt;/span&gt; &lt;span class="nv"&gt;2&lt;/span&gt;&lt;span class="pi"&gt;]&lt;/span&gt;
&lt;span class="na"&gt;background_dir&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;resources/backgrounds/paper/"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  Generating Synthetic Data
&lt;/h2&gt;

&lt;p&gt;Run the CLI:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;synthtiger &lt;span class="nt"&gt;-o&lt;/span&gt; ./outputs/synthdog_ur &lt;span class="nt"&gt;-c&lt;/span&gt; 1000 &lt;span class="nt"&gt;-w&lt;/span&gt; 8 &lt;span class="nt"&gt;-v&lt;/span&gt; template.py SynthDoG config_ur.yaml
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This generates 1000 samples with 8 workers, outputting images and text into &lt;code&gt;./outputs/synthdog_ur/&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Repeat with &lt;code&gt;config_ar.yaml&lt;/code&gt;, &lt;code&gt;config_fa.yaml&lt;/code&gt;, etc. for multiple languages.&lt;/p&gt;




&lt;h2&gt;
  
  
  Formatting for Donut
&lt;/h2&gt;

&lt;p&gt;Donut expects an &lt;strong&gt;image + JSON&lt;/strong&gt; pair. Structure your dataset like:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;my_dataset/
 ├─ train/
 │   ├─ metadata.jsonl
 │   ├─ 00000001.png
 │   └─ ...
 ├─ validation/
 │   └─ ...
 └─ test/
     └─ ...
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Each line in &lt;code&gt;metadata.jsonl&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"file_name"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"00000001.png"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"ground_truth"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"{&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;gt_parse&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;:{&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;text_sequence&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;:&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;یہ اردو کا متن ہے&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;}}"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Donut will tokenize this internally. Ensure that &lt;code&gt;file_name&lt;/code&gt; matches your image and &lt;code&gt;text_sequence&lt;/code&gt; contains the RTL ground truth text.&lt;/p&gt;




&lt;h2&gt;
  
  
  Advanced Tips
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Layouts&lt;/strong&gt;: Customize &lt;code&gt;template.py&lt;/code&gt; for multi-column, headers, or tables.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Effects&lt;/strong&gt;: Add noise, blur, or perspective distortion in YAML for realism.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Fonts&lt;/strong&gt;: Use multiple fonts per language to avoid overfitting.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Mixed Scripts&lt;/strong&gt;: Include English corpora to simulate bilingual documents.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Scaling&lt;/strong&gt;: Generate 10k–100k samples to pre-train Donut effectively.&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  Conclusion
&lt;/h2&gt;

&lt;p&gt;With &lt;a href="https://github.com/aiviewz/Synthdog-RTL" rel="noopener noreferrer"&gt;SynthDog-RTL&lt;/a&gt; you can rapidly bootstrap synthetic OCR datasets for all major RTL languages. The generated data integrates seamlessly with Donut, enabling you to train or fine-tune robust document understanding models even in low-resource settings.&lt;/p&gt;




&lt;p&gt;&lt;strong&gt;References&lt;/strong&gt;:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://github.com/clovaai/donut" rel="noopener noreferrer"&gt;Donut GitHub&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/aiviewz/Synthdog-RTL" rel="noopener noreferrer"&gt;SynthDoG-RTL&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/clovaai/synthtiger" rel="noopener noreferrer"&gt;SynthTIGER&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://www.aiviewz.com/posts/how-to-create-synthetic-dataset-for-donut-ocr-for-your-custom-language" rel="noopener noreferrer"&gt;Synthdog-RTL Tutorial Blog Post&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>python</category>
      <category>ocr</category>
      <category>computervision</category>
      <category>programming</category>
    </item>
  </channel>
</rss>
