<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Stephen BJ</title>
    <description>The latest articles on DEV Community by Stephen BJ (@jerin_stephen).</description>
    <link>https://dev.to/jerin_stephen</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F494691%2Fcd94f085-9aa3-4107-9307-e65431f7c80c.jpg</url>
      <title>DEV Community: Stephen BJ</title>
      <link>https://dev.to/jerin_stephen</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/jerin_stephen"/>
    <language>en</language>
    <item>
      <title>ASGI and WSGI</title>
      <dc:creator>Stephen BJ</dc:creator>
      <pubDate>Thu, 05 Feb 2026 05:44:03 +0000</pubDate>
      <link>https://dev.to/jerin_stephen/asgi-and-wsgi-2hg</link>
      <guid>https://dev.to/jerin_stephen/asgi-and-wsgi-2hg</guid>
      <description>&lt;p&gt;HI all,&lt;br&gt;
welcome to an other interesting topic about the servers in python&lt;/p&gt;

&lt;p&gt;lore: while i was interviewed interviewer ask me a question why did you chose fastapi instead of django and flask. i was stuck for few min because what i did before i pick the framework just google + GPT's and get the best framework for my project. (actually they told me why it was the best option also). i didn't pick up django because it has very big learning curve but it provide lots of functionalities like ORM, middleware , auth etc., out of the box.&lt;/p&gt;

&lt;p&gt;Now, will see why exactly the major and important thing on selection of Fast vs Flask is &lt;strong&gt;ASGI ** and **WSGI&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;ASGI provide the asynchronous feature which WSGI is not offers (best of my knowledge ) my be it is debatable&lt;/p&gt;

&lt;p&gt;lets take a process like Make coffee  and lemon juice:&lt;/p&gt;

&lt;p&gt;steps like&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;get the milk , sugar , coffee powder
&lt;/li&gt;
&lt;li&gt;turn on the stove&lt;/li&gt;
&lt;li&gt;warm the milk &lt;/li&gt;
&lt;li&gt;add sugar and coffee powder &lt;/li&gt;
&lt;li&gt;&lt;p&gt;pore it in coup&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;get the lemon, sugar, water, ice cubes and salt&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;cut the lemon and squeeze into the coup&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;add sugar, water, ice cubes and salt&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;pore it in the coup&lt;/p&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;this is what the step will follow by the wsgi but in the case of asgi&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;get the milk , sugar , coffee powder
&lt;/li&gt;
&lt;li&gt;turn on the stove&lt;/li&gt;
&lt;li&gt;&lt;p&gt;warm the milk &lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;get the lemon, sugar, water, ice cubes and salt&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;cut the lemon and squeeze into the coup&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;add sugar, water, ice cubes and salt&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;pore it in the coup&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;add sugar and coffee powder &lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;pore it in coup&lt;/p&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;this is how the asgi will do &lt;br&gt;
now we may have question like ok it do only one request can handle at the time direct answer is no. in this place service worker play's an major role for example we may have multiple person doing this . but the problem is kitten size if the kitten fit only for one person we can add multiple people in it &lt;/p&gt;

&lt;p&gt;in similar way WSGI can have multiple workers but the workers limit based on the CPU size like Number of core X 2 that is the max&lt;/p&gt;

&lt;p&gt;Conclusion&lt;/p&gt;

&lt;p&gt;WSGI is not bad. it's just done evolving.&lt;br&gt;
ASGI is what the modern internet expects.&lt;/p&gt;

&lt;p&gt;In platform terms:&lt;br&gt;
WSGI = stable, predictable, capped&lt;br&gt;
ASGI = scalable, concurrent, future-proof&lt;/p&gt;

&lt;p&gt;in my term based on the context it is mater if the process has move cpu related one and not request intensive both will give the same result &lt;/p&gt;

</description>
      <category>python</category>
      <category>flask</category>
      <category>fastapi</category>
    </item>
    <item>
      <title>Paddle OCR-VL &amp; DeepSeek-OCR</title>
      <dc:creator>Stephen BJ</dc:creator>
      <pubDate>Tue, 21 Oct 2025 09:58:47 +0000</pubDate>
      <link>https://dev.to/jerin_stephen/paddle-ocr-vl-deepseek-ocr-31m2</link>
      <guid>https://dev.to/jerin_stephen/paddle-ocr-vl-deepseek-ocr-31m2</guid>
      <description>&lt;h2&gt;
  
  
  PaddleOCR-VL
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;What it is&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;PaddleOCR-VL is a vision-language model (VLM) built for document parsing-meaning not just "read the text" &lt;br&gt;
but "understand the layout, tables, formulas, charts, multilingual text."  it is supports 109 languages. &lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Why it stands out&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Multilingual: covers a wide spectrum of scripts (English, French, german, etc.). &lt;/li&gt;
&lt;li&gt;Document-layout aware: recognizes text blocks, tables, formulas, charts so it is designed for complex documents beyond simple printed text. &lt;/li&gt;
&lt;li&gt;Resource efficiency: the documentation claims faster inference, lower memory consumption compared with many generic large VLMs. &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Where to use it&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Parsing scanned PDF documents where you need structure and multiple languages.&lt;/li&gt;
&lt;li&gt;layout understanding is required (not just OCR, but "what is in the table?").&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Limitations&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;although it supports many languages, real-world accuracy might vary wildly for less common scripts or poor quality scans.&lt;/li&gt;
&lt;li&gt;as with any model: OCR + layout parsing is never "perfect" - expect error rates, need post-processing.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  DeepSeek-OCR
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;What it is&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Deepseek-OCR is a newer open-source model that targets OCR/document parsing—but with a twist: &lt;br&gt;
    they emphasize "vision-text compression" (how many vision tokens are needed to decode text tokens) &lt;br&gt;
    document throughput/efficiency. &lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Why it stands out&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Code and weights appear publicly available (huggingFace). &lt;/li&gt;
&lt;li&gt;Easy to set up. &lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Where it could excel&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;If you care about throughput—running large volumes of pages/documents, wanting to minimize compute per page.&lt;/li&gt;
&lt;li&gt;Use-cases where you have relatively "standard" scanned documents and you want a streamlined/inexpensive pipeline.&lt;/li&gt;
&lt;li&gt;Potentially situations where compressing document representations matters (archive, storage, indexing).&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Limitations&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Since it is newer: less maturity, fewer benchmarks publicly compared to paddleOCR-vl.&lt;/li&gt;
&lt;li&gt;Compression focus may trade off accuracy in edge cases (complex layout, unusual fonts, low quality scans).&lt;/li&gt;
&lt;li&gt;Documentation &amp;amp; ecosystem may be thinner compared to more established toolkits.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  where did i find this details?
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://huggingface.co/PaddlePaddle/PaddleOCR-VL" rel="noopener noreferrer"&gt;https://huggingface.co/PaddlePaddle/PaddleOCR-VL&lt;/a&gt; "PaddlePaddle/PaddleOCR-VL"&lt;br&gt;
&lt;a href="https://github.com/deepseek-ai/DeepSeek-OCR" rel="noopener noreferrer"&gt;https://github.com/deepseek-ai/DeepSeek-OCR&lt;/a&gt; "deepseek-ai/DeepSeek-OCR: Contexts Optical Compression"&lt;/p&gt;

</description>
      <category>deepseek</category>
      <category>ocr</category>
      <category>llm</category>
    </item>
    <item>
      <title>Found better Markdown editor</title>
      <dc:creator>Stephen BJ</dc:creator>
      <pubDate>Thu, 10 Jul 2025 07:14:25 +0000</pubDate>
      <link>https://dev.to/jerin_stephen/found-better-markdown-editor-59oj</link>
      <guid>https://dev.to/jerin_stephen/found-better-markdown-editor-59oj</guid>
      <description>&lt;p&gt;Today, I had the pleasure of working with &lt;a href="https://mdxeditor.dev" rel="noopener noreferrer"&gt;&lt;code&gt;mdxeditor.dev&lt;/code&gt;&lt;/a&gt;, and I must say — it's an &lt;em&gt;impressively modern&lt;/em&gt; and &lt;em&gt;developer-focused&lt;/em&gt; rich text editor designed specifically for working with MDX (Markdown + JSX). If you are building content management systems, documentation platforms, or &lt;strong&gt;LLM prompt builder&lt;/strong&gt; this library is definitely worth your attention.&lt;/p&gt;




&lt;h2&gt;
  
  
  &lt;code&gt;mdxeditor.dev&lt;/code&gt;?
&lt;/h2&gt;

&lt;p&gt;it blends the simplicity of markdown with the flexibility of reactjs components, offering a great authoring experience for developers.&lt;/p&gt;




&lt;h2&gt;
  
  
  Features i like
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Clear and Modular API&lt;/strong&gt; — Easy to extend and customize the editor experience.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Image and Media support&lt;/strong&gt; — Built-in components for handling images, code blocks, etc.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Custom Component Previews&lt;/strong&gt; — Define how your custom MDX components render in the editor. &lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Plugin-based Architecture&lt;/strong&gt; — Add or remove features like headings, tables, or frontmatter with ease.&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  Getting started (I copied this from their API document )
&lt;/h2&gt;

&lt;p&gt;Installation is a breeze:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npm &lt;span class="nb"&gt;install&lt;/span&gt; @mdxeditor/editor
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Then, just drop it into your React component:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight tsx"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nx"&gt;MDXEditor&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;headingsPlugin&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;listsPlugin&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;@mdxeditor/editor&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="p"&gt;&amp;lt;&lt;/span&gt;&lt;span class="nc"&gt;MDXEditor&lt;/span&gt;
  &lt;span class="na"&gt;markdown&lt;/span&gt;&lt;span class="p"&gt;=&lt;/span&gt;&lt;span class="s"&gt;"# Hello MDX"&lt;/span&gt;
  &lt;span class="na"&gt;plugins&lt;/span&gt;&lt;span class="p"&gt;=&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nf"&gt;headingsPlugin&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt; &lt;span class="nf"&gt;listsPlugin&lt;/span&gt;&lt;span class="p"&gt;()]&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;/&amp;gt;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;the api is clean and intuitive. it does not take much to get a full-fledged editor up and running.&lt;/p&gt;




&lt;h2&gt;
  
  
  developer experience
&lt;/h2&gt;

&lt;p&gt;What stood out the most during my usage today:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;The documentation is incredibly clear and well-structured — with example, props, and plugin guides.&lt;/li&gt;
&lt;li&gt;the plugin architecture  makes it very easy to compose editor feature.&lt;/li&gt;
&lt;li&gt;everything was 'typed with TypeScript', making development smoother and but little anoying for me (because i am not good at typescript).&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  Thing i loved
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Simplicity + Power&lt;/strong&gt; - easy enough to get started in minutes, yet powerful enough for complex customization.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Plugin Ecosystem&lt;/strong&gt; - Everything from code blocks to markdown text can be plugged in or out.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Live Preview Support&lt;/strong&gt; - you can define how MDX components render in-editor.&lt;/li&gt;
&lt;/ul&gt;




&lt;p&gt;&lt;a href="https://mdxeditor.dev" rel="noopener noreferrer"&gt;Visit mdxeditor.dev →&lt;/a&gt;&lt;br&gt;
&lt;a href="https://buymeacoffee.com/stephenbj" rel="noopener noreferrer"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Ftam27klrepvq97jvits1.png" alt=" " width="700" height="700"&gt;&lt;/a&gt; &lt;br&gt;
&lt;a href="https://buymeacoffee.com/stephenbj" rel="noopener noreferrer"&gt;buymeacoffee  &lt;/a&gt; &lt;/p&gt;

</description>
      <category>react</category>
      <category>frontend</category>
      <category>llm</category>
      <category>uidesign</category>
    </item>
    <item>
      <title>From vision to reality: My journey building with code</title>
      <dc:creator>Stephen BJ</dc:creator>
      <pubDate>Tue, 01 Jul 2025 04:40:38 +0000</pubDate>
      <link>https://dev.to/jerin_stephen/from-vision-to-reality-my-journey-building-with-code-4fdc</link>
      <guid>https://dev.to/jerin_stephen/from-vision-to-reality-my-journey-building-with-code-4fdc</guid>
      <description>&lt;p&gt;Hi, I’m Jerin Stephen  a full-stack developer, creative thinker, and someone who believes in turning ideas into working systems. I wanted to share a story, not just about coding, but about making things real. If you're someone who loves to build, experiment, or even just imagine this is for you.&lt;/p&gt;

&lt;p&gt;It all started with a thought...&lt;/p&gt;

&lt;p&gt;I didn’t just want to write code, I wanted to create.&lt;/p&gt;

&lt;p&gt;For me, building software has always been about more than just syntax or solving tickets. It is about crafting meaningful experiences, whether that is a dynamic dashboard, a smart automation, or a tool that actually helps someone. I’ve worked with Python, FastAPI, react.js, PostgreSQL — not because they’re trendy, but because they help me build what i envision.&lt;/p&gt;

&lt;p&gt;I Love building things Like:&lt;/p&gt;

&lt;p&gt;✅ smart user interfaces&lt;/p&gt;

&lt;p&gt;The kind that respond fast, feel intuitive, and adapt to the user. &lt;/p&gt;

&lt;p&gt;✅ Seamless integrations&lt;/p&gt;

&lt;p&gt;Linking systems together so they feel like one — from frontend to APIs.&lt;/p&gt;

&lt;p&gt;✅ Clean, scalable backends&lt;/p&gt;

&lt;p&gt;Codebases that make senses months later, with logs, logic, and structure that scale.&lt;/p&gt;

&lt;p&gt;✅ Tools That Help People&lt;/p&gt;

&lt;p&gt;Whether it’s a teammate, a client, or a complete stranger when someone says, "This made my job easier," that is the win.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Why I’m Sharing This&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Because I started just like most people - Googling how to make a button work or doing vibe coding&lt;/p&gt;

&lt;p&gt;Today, I build things I once thought were too complex. And I have learned something powerful:&lt;/p&gt;

&lt;p&gt;You don’t need permission to start. Just curiosity and consistency.&lt;/p&gt;

&lt;p&gt;You don’t need a big team. You do not need all the answers. You just need the courage to build, one step at a time.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Lessons I Live By&lt;/strong&gt;&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt; Start small but start. A simple tool can lead to something big.&lt;/li&gt;
&lt;li&gt; Keep things human. design for people, not just for machines.&lt;/li&gt;
&lt;li&gt; Be okay not knowing. That is where learning lives.&lt;/li&gt;
&lt;li&gt; Use tools. But also shape them. Do not settle for default.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;&lt;strong&gt;Final Words&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;I used to chase certifications. Now I chase ideas.&lt;/p&gt;

&lt;p&gt;If you're a student, a beginner, or even a tired senior dev:&lt;/p&gt;

&lt;p&gt;The things you imagine can become real.&lt;/p&gt;

&lt;p&gt;Whether it’s an app, a game, a plugin, or something no one’s seen before bring it to life.&lt;/p&gt;

&lt;p&gt;Take your passion. Add some code. And create.&lt;/p&gt;

&lt;p&gt;We’re not just developers. &lt;strong&gt;We’re makers.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt; &lt;/p&gt;

</description>
    </item>
    <item>
      <title>career update</title>
      <dc:creator>Stephen BJ</dc:creator>
      <pubDate>Mon, 30 Jun 2025 06:41:21 +0000</pubDate>
      <link>https://dev.to/jerin_stephen/career-update-5749</link>
      <guid>https://dev.to/jerin_stephen/career-update-5749</guid>
      <description>&lt;div class="ltag__link--embedded"&gt;
  &lt;div class="crayons-story "&gt;
  &lt;a href="https://dev.to/jerin_stephen/the-future-of-software-engineering-from-my-experience-12g3" class="crayons-story__hidden-navigation-link"&gt;The Future of Software Engineering — From My Experience&lt;/a&gt;


  &lt;div class="crayons-story__body crayons-story__body-full_post"&gt;
    &lt;div class="crayons-story__top"&gt;
      &lt;div class="crayons-story__meta"&gt;
        &lt;div class="crayons-story__author-pic"&gt;

          &lt;a href="/jerin_stephen" class="crayons-avatar  crayons-avatar--l  "&gt;
            &lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F494691%2Fcd94f085-9aa3-4107-9307-e65431f7c80c.jpg" alt="jerin_stephen profile" class="crayons-avatar__image"&gt;
          &lt;/a&gt;
        &lt;/div&gt;
        &lt;div&gt;
          &lt;div&gt;
            &lt;a href="/jerin_stephen" class="crayons-story__secondary fw-medium m:hidden"&gt;
              Stephen BJ
            &lt;/a&gt;
            &lt;div class="profile-preview-card relative mb-4 s:mb-0 fw-medium hidden m:inline-block"&gt;
              
                Stephen BJ
                
              
              &lt;div id="story-author-preview-content-2627428" class="profile-preview-card__content crayons-dropdown branded-7 p-4 pt-0"&gt;
                &lt;div class="gap-4 grid"&gt;
                  &lt;div class="-mt-4"&gt;
                    &lt;a href="/jerin_stephen" class="flex"&gt;
                      &lt;span class="crayons-avatar crayons-avatar--xl mr-2 shrink-0"&gt;
                        &lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F494691%2Fcd94f085-9aa3-4107-9307-e65431f7c80c.jpg" class="crayons-avatar__image" alt=""&gt;
                      &lt;/span&gt;
                      &lt;span class="crayons-link crayons-subtitle-2 mt-5"&gt;Stephen BJ&lt;/span&gt;
                    &lt;/a&gt;
                  &lt;/div&gt;
                  &lt;div class="print-hidden"&gt;
                    
                      Follow
                    
                  &lt;/div&gt;
                  &lt;div class="author-preview-metadata-container"&gt;&lt;/div&gt;
                &lt;/div&gt;
              &lt;/div&gt;
            &lt;/div&gt;

          &lt;/div&gt;
          &lt;a href="https://dev.to/jerin_stephen/the-future-of-software-engineering-from-my-experience-12g3" class="crayons-story__tertiary fs-xs"&gt;&lt;time&gt;Jun 27 '25&lt;/time&gt;&lt;span class="time-ago-indicator-initial-placeholder"&gt;&lt;/span&gt;&lt;/a&gt;
        &lt;/div&gt;
      &lt;/div&gt;

    &lt;/div&gt;

    &lt;div class="crayons-story__indention"&gt;
      &lt;h2 class="crayons-story__title crayons-story__title-full_post"&gt;
        &lt;a href="https://dev.to/jerin_stephen/the-future-of-software-engineering-from-my-experience-12g3" id="article-link-2627428"&gt;
          The Future of Software Engineering — From My Experience
        &lt;/a&gt;
      &lt;/h2&gt;
        &lt;div class="crayons-story__tags"&gt;
        &lt;/div&gt;
      &lt;div class="crayons-story__bottom"&gt;
        &lt;div class="crayons-story__details"&gt;
            &lt;a href="https://dev.to/jerin_stephen/the-future-of-software-engineering-from-my-experience-12g3#comments" class="crayons-btn crayons-btn--s crayons-btn--ghost crayons-btn--icon-left flex items-center"&gt;
              Comments


              &lt;span class="hidden s:inline"&gt;Add Comment&lt;/span&gt;
            &lt;/a&gt;
        &lt;/div&gt;
        &lt;div class="crayons-story__save"&gt;
          &lt;small class="crayons-story__tertiary fs-xs mr-2"&gt;
            3 min read
          &lt;/small&gt;
            
              &lt;span class="bm-initial"&gt;
                

              &lt;/span&gt;
              &lt;span class="bm-success"&gt;
                

              &lt;/span&gt;
            
        &lt;/div&gt;
      &lt;/div&gt;
    &lt;/div&gt;
  &lt;/div&gt;
&lt;/div&gt;

&lt;/div&gt;


</description>
      <category>career</category>
      <category>softwareengineering</category>
      <category>productivity</category>
    </item>
    <item>
      <title>PDF extractor alternatives for Marker PDF</title>
      <dc:creator>Stephen BJ</dc:creator>
      <pubDate>Mon, 30 Jun 2025 04:50:44 +0000</pubDate>
      <link>https://dev.to/jerin_stephen/pdf-extractor-alternatives-for-marker-pdf-ld5</link>
      <guid>https://dev.to/jerin_stephen/pdf-extractor-alternatives-for-marker-pdf-ld5</guid>
      <description>&lt;p&gt;When working on PDF extraction tasks for large-scale document processing and knowledge workflows, especially involving llm, i needed an efficient alternative to &lt;strong&gt;Marker PDF&lt;/strong&gt;. i’ve evaluated and tested multiple open-source libraries, checking for their speed, accuracy, layout retention, image extraction, and ease of integration with LLM pipelines. Below is my hands-on review of different tools i explored and how they stack up.&lt;/p&gt;




&lt;h2&gt;
  
  
  My Evaluation Criteria
&lt;/h2&gt;

&lt;p&gt;i focus the following aspects:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;license&lt;/strong&gt; – open source friendly for commercial projects.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;page limitations&lt;/strong&gt; – whether there's a hard limit. &lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;image extraction&lt;/strong&gt; – support for embedded or scanned images. &lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;output types&lt;/strong&gt; – markdown, json and plain text .&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;layout undarstanding&lt;/strong&gt; – whether it preserves layout elements like tables, columns, and headers.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;ease of use&lt;/strong&gt; – setup, community, and speed.&lt;/li&gt;
&lt;/ul&gt;




&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;sn&lt;/th&gt;
&lt;th&gt;library name&lt;/th&gt;
&lt;th&gt;license&lt;/th&gt;
&lt;th&gt;image extraction&lt;/th&gt;
&lt;th&gt;output types&lt;/th&gt;
&lt;th&gt;layout details&lt;/th&gt;
&lt;th&gt;pypi link&lt;/th&gt;
&lt;th&gt;comments&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;PyMuPDF4LLM&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;agpl-3.0&lt;/td&gt;
&lt;td&gt;yes&lt;/td&gt;
&lt;td&gt;Markdown, Llamaindex Docs&lt;/td&gt;
&lt;td&gt;Limited&lt;/td&gt;
&lt;td&gt;&lt;a href="https://pypi.org/project/pymupdf4llm/" rel="noopener noreferrer"&gt;link&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;very fast and effective. minimal setup.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;pdfplumber&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;mit&lt;/td&gt;
&lt;td&gt;limited&lt;/td&gt;
&lt;td&gt;JSON, Text&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;&lt;a href="https://pypi.org/project/pdfplumber/" rel="noopener noreferrer"&gt;link&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;good for table heavy pdfs.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;pdfminer.six&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;mit&lt;/td&gt;
&lt;td&gt;no&lt;/td&gt;
&lt;td&gt;Text&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;&lt;a href="https://pypi.org/project/pdfminer.six/" rel="noopener noreferrer"&gt;link&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;works but average accuracy.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;markitdown&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;mit&lt;/td&gt;
&lt;td&gt;no&lt;/td&gt;
&lt;td&gt;Markdown&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;td&gt;&lt;a href="https://pypi.org/project/markitdown/" rel="noopener noreferrer"&gt;link&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;basic converter, nothing special.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;5&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;nougat-ocr&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;apache 2.0&lt;/td&gt;
&lt;td&gt;yes&lt;/td&gt;
&lt;td&gt;MultiMarkdown&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;td&gt;&lt;a href="https://github.com/facebookresearch/nougat" rel="noopener noreferrer"&gt;link&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;good results, but painfully slow.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;6&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;pdf-to-markdown&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;mit&lt;/td&gt;
&lt;td&gt;yes&lt;/td&gt;
&lt;td&gt;Markdown&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;td&gt;-&lt;/td&gt;
&lt;td&gt;decent, but loses some structured data.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;7&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;olmocr&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;apache 2.0&lt;/td&gt;
&lt;td&gt;yes&lt;/td&gt;
&lt;td&gt;Markdown&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;&lt;a href="https://pypi.org/project/olmocr" rel="noopener noreferrer"&gt;link&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;excellent results with structure retained.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;8&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;docling&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;mit&lt;/td&gt;
&lt;td&gt;yes&lt;/td&gt;
&lt;td&gt;Markdown, JSON and HTML&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;&lt;a href="https://ds4sd.github.io/docling/examples/minimal/" rel="noopener noreferrer"&gt;Demo&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;tested via colab, very effective.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;9&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;EasyOCR&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;apache 2.0&lt;/td&gt;
&lt;td&gt;images only&lt;/td&gt;
&lt;td&gt;List, JSON&lt;/td&gt;
&lt;td&gt;Yes (image)&lt;/td&gt;
&lt;td&gt;&lt;a href="https://pypi.org/project/easyocr/" rel="noopener noreferrer"&gt;link&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;only for ocr, doesn't retain layout.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;10&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;OCRmyPDF&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;mpl 2.0&lt;/td&gt;
&lt;td&gt;yes&lt;/td&gt;
&lt;td&gt;PDF/A&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;&lt;a href="https://pypi.org/project/ocrmypdf/" rel="noopener noreferrer"&gt;link&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;complicated setup, good ocr layer addition.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;11&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;LayoutParser&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;apache 2.0&lt;/td&gt;
&lt;td&gt;no&lt;/td&gt;
&lt;td&gt;JSON&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;&lt;a href="https://pypi.org/project/layoutparser/" rel="noopener noreferrer"&gt;link&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;use with other ocr tools for layout parsing.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;12&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;pandoc&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;gpl 2.0&lt;/td&gt;
&lt;td&gt;no&lt;/td&gt;
&lt;td&gt;Markdown&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;td&gt;&lt;a href="https://github.com/jgm/pandoc" rel="noopener noreferrer"&gt;link&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;did not work well in pdf to md conversion.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;13&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;pypdfDirectoryLoader&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;open source&lt;/td&gt;
&lt;td&gt;no&lt;/td&gt;
&lt;td&gt;Text, Markdown and HTML&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;td&gt;-&lt;/td&gt;
&lt;td&gt;works but lacks layout awareness.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;14&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;pdfium&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;bsd-3&lt;/td&gt;
&lt;td&gt;no&lt;/td&gt;
&lt;td&gt;CSV, Markdown, Text&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;td&gt;-&lt;/td&gt;
&lt;td&gt;good for low-level pdf rendering.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;15&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;tableau&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;mit&lt;/td&gt;
&lt;td&gt;no&lt;/td&gt;
&lt;td&gt;CSV&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;td&gt;-&lt;/td&gt;
&lt;td&gt;only useful for table extraction.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;16&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;pypdf&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;open source&lt;/td&gt;
&lt;td&gt;not clear&lt;/td&gt;
&lt;td&gt;Text&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;td&gt;&lt;a href="https://pypi.org/project/pypdf/" rel="noopener noreferrer"&gt;link&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;does not handle layout or images well.&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;




&lt;h2&gt;
  
  
  My top picks (as of now)
&lt;/h2&gt;

&lt;blockquote&gt;
&lt;h3&gt;
  
  
  &lt;strong&gt;1. PyMuPDF4LLM&lt;/strong&gt;
&lt;/h3&gt;

&lt;p&gt;super fast and integrates well with llm pipelines like llamaindex. excellent markdown output. limited layout parsing, but still very usable&lt;/p&gt;
&lt;h3&gt;
  
  
  &lt;strong&gt;2. OlmoCR&lt;/strong&gt;
&lt;/h3&gt;

&lt;p&gt;surprisingly accurate with layout and structure.it even preserved indentation and titles nicely. great for fine-tuning document-based llms.&lt;/p&gt;
&lt;h3&gt;
  
  
  &lt;strong&gt;3. Docling&lt;/strong&gt;
&lt;/h3&gt;

&lt;p&gt;A great all-rounder, especailly with visual layout parsing.useful for html or json-based workflows. &lt;/p&gt;
&lt;h3&gt;
  
  
  &lt;strong&gt;4. pdfplumber&lt;/strong&gt;
&lt;/h3&gt;

&lt;p&gt;if you are dealing with &lt;strong&gt;tabular PDFs&lt;/strong&gt;, this is your goto. Fast and battle-tested.&lt;/p&gt;
&lt;/blockquote&gt;




&lt;h2&gt;
  
  
  What did not Work for Me
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;pandoc&lt;/strong&gt; failed miserably with most PDF inputs for markdown conversion.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;MarkerDown&lt;/strong&gt; and &lt;strong&gt;pdfminer.six&lt;/strong&gt; struggled with layouts and formatting.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;OCR-only tools&lt;/strong&gt; like EasyOCR and OCRmyPDF were limited to images or required post-processing.&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  Final thought
&lt;/h2&gt;

&lt;p&gt;Marker PDF is solid, but not always the right fit, especially when you need &lt;strong&gt;control&lt;/strong&gt;, &lt;strong&gt;open-source licensing&lt;/strong&gt;, and &lt;strong&gt;llm-friendly formatting&lt;/strong&gt;.tools like &lt;code&gt;PyMuPDF4LLM&lt;/code&gt;, &lt;code&gt;Olmocr&lt;/code&gt;, and &lt;code&gt;Docling&lt;/code&gt; provide a powerful alternative with minimal trade-offs.&lt;/p&gt;

&lt;p&gt;if you want something &lt;strong&gt;ready for fine-tuning datasets&lt;/strong&gt;, these three would be my top choices to build on.&lt;/p&gt;

&lt;p&gt;Please share your ideas in comment it will be very helpful for me to learn something &lt;/p&gt;

&lt;p&gt;&lt;a href="https://github.com/GiftMungmeeprued/document-parsers-list" rel="noopener noreferrer"&gt;document-parsers-list&lt;/a&gt;&lt;br&gt;
&lt;a href="https://buymeacoffee.com/stephenbj" rel="noopener noreferrer"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Ftam27klrepvq97jvits1.png" alt=" " width="700" height="700"&gt;&lt;/a&gt;&lt;/p&gt;

</description>
      <category>ocr</category>
      <category>python</category>
      <category>pdf</category>
      <category>llm</category>
    </item>
    <item>
      <title>The Future of Software Engineering — From My Experience</title>
      <dc:creator>Stephen BJ</dc:creator>
      <pubDate>Fri, 27 Jun 2025 04:02:20 +0000</pubDate>
      <link>https://dev.to/jerin_stephen/the-future-of-software-engineering-from-my-experience-12g3</link>
      <guid>https://dev.to/jerin_stephen/the-future-of-software-engineering-from-my-experience-12g3</guid>
      <description>&lt;p&gt;As a developer who's used Chatgpt, claude, gemini, versal and cursor I've seen firsthand how AI is reshaping how we build software. I wanted to share my point of view on where this is all headed not from a theoretical standpoint, but based on actual hands-on experience with these tools.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;What I Noticed with Each Tool&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;*&lt;em&gt;ChatGPT *&lt;/em&gt;
ChatGPT is my go-to for brainstorming ideas, debugging, and getting explanations. It's like having a 24/7 assistant who can break things down or generate alternatives when I'm stuck.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Claude&lt;/strong&gt;
When it comes to code, Claude really stands out. The way it generates clean, readable, and logically sound code is better than most tools I’ve used. I find myself using it when I need reliable and thoughtful solutions.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Gemini&lt;/strong&gt;
Gemini integrates well with Google’s ecosystem. I didn’t use it as much as others, but for Android development and anything Google-related, it’s a solid choice.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Versal&lt;/strong&gt;
Versal blew me away with its UI-building capability. It’s the best tool I’ve come across for visually designing interfaces. It feels like Figma but with AI working behind the scenes.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Cursor&lt;/strong&gt;
Cursor is amazing for code. It’s like a coding partner inside the IDE. You get suggestions, refactoring help, and contextual responses without leaving your environment.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;What This Means for the Future of Software Engineering&lt;/strong&gt;&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;From developer to designer : With tools like Versal, I don’t have to manually code every layout. I focus more on the structure, flow, and usability. The AI takes care of the repetitive stuff.&lt;/li&gt;
&lt;li&gt;From writing code to building systems : ai handles most of the boilerplate and even logic-heavy parts. I’ve started spending more time designing system behavior, not just writing lines of code.&lt;/li&gt;
&lt;li&gt;QA and debugging are changing : claude and Cursor are so good at catching bugs or writing test cases that I’m relying on them for quality checks. It saves a lot of time.&lt;/li&gt;
&lt;li&gt;Prompting is the new skill :being able to give the right prompt is now more important than memorizing syntax. I’ve realized that a good prompt leads to great results.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;*&lt;em&gt;Where i think this is going *&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;AI-native stacks&lt;/strong&gt;: We’ll see frameworks and tools built to work hand-in-hand with AI.&lt;/p&gt;

&lt;p&gt;More non-coders building apps: Tools like Versal + AI let anyone design interfaces devs will be the ones building smart components behind them.&lt;/p&gt;

&lt;p&gt;Multimodal input: In the future, I think we’ll build apps using sketches, voice, or even images. Text won’t be the only way.&lt;/p&gt;

&lt;p&gt;Will AI Replace Developers Like Me?&lt;/p&gt;

&lt;p&gt;Not really. But it’s clear to me that:&lt;/p&gt;

&lt;p&gt;Developers who don’t use AI will fall behind.&lt;/p&gt;

&lt;p&gt;I now see my role evolving less coding from scratch, more system thinking and smart prompting. The people who grow with AI will always stay relevant.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Final Words&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;If you are a developer, start using these tools claude for coding, versal for UI, chatgpt for planning, Cursor for in-editor help. Learn how to guide AI, not just compete with it.&lt;/p&gt;

&lt;p&gt;That’s how I see the future, a developer who can work with AI is not just more efficient they’re unstoppable.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://buymeacoffee.com/stephenbj" rel="noopener noreferrer"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Ftam27klrepvq97jvits1.png" alt=" " width="700" height="700"&gt;&lt;/a&gt;&lt;br&gt;
 &lt;a href="https://buymeacoffee.com/stephenbj" rel="noopener noreferrer"&gt;buymeacoffee&lt;/a&gt;&lt;/p&gt;

</description>
    </item>
    <item>
      <title>RAG vs Fine-Tuning vs Alternatives</title>
      <dc:creator>Stephen BJ</dc:creator>
      <pubDate>Thu, 26 Jun 2025 12:41:28 +0000</pubDate>
      <link>https://dev.to/jerin_stephen/rag-vs-fine-tuning-vs-alternatives-3lba</link>
      <guid>https://dev.to/jerin_stephen/rag-vs-fine-tuning-vs-alternatives-3lba</guid>
      <description>&lt;h2&gt;
  
  
  1. RAG (Retrieval-augmented generation)
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Overview:
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;Combines retrieval and generation. &lt;/li&gt;
&lt;li&gt;Retrieves relevant documents or chunks from a knowledge base.&lt;/li&gt;
&lt;li&gt;The model then generates a response based on both the prompt and the retrieved content.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Cons:
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;Depends on retrieval quality.&lt;/li&gt;
&lt;li&gt;Slightly higher latency.&lt;/li&gt;
&lt;li&gt;Requires vector DB setup.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Use Cases:
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;Chatbots with knowledge bases.&lt;/li&gt;
&lt;li&gt;Real-time support systems .&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  * Assistants with dynamic information.
&lt;/h2&gt;

&lt;h2&gt;
  
  
  2. Fine-Tuning
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Overveiw:
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;Update model weights with domain or task-specific data. &lt;/li&gt;
&lt;li&gt;Trains the model to behave specifically for your use case.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Types:
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;Full fine-tuning&lt;/li&gt;
&lt;li&gt;Parameter-efficient (LoRA, QLoRA)&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Cons:
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;Expensive in terms of compute.&lt;/li&gt;
&lt;li&gt;Harder to update (  Need more GPU and datasets ).&lt;/li&gt;
&lt;li&gt;Risk of overfitting.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Use Cases:
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;Legal or medical assistants&lt;/li&gt;
&lt;li&gt;Summarization of company documents&lt;/li&gt;
&lt;li&gt;Enterprise chat models&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  3. Alternatives to RAG and Fine-Tuning
&lt;/h2&gt;

&lt;h3&gt;
  
  
  3.1 Prompt Engineering
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;Design prompts to guide model behavior.&lt;/li&gt;
&lt;li&gt;Use zero-shot or few-shot learning.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Pros:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Fast and low-cost&lt;/li&gt;
&lt;li&gt;No retraining needed&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Cons:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Limited in complexity and flexibility&lt;/li&gt;
&lt;/ul&gt;




&lt;h3&gt;
  
  
  3.2 Instruction-Tuned Models
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;Use LLMs already fine-tuned on instructions.&lt;/li&gt;
&lt;li&gt;E.g., GPT-Instruct, Mistral-Instruct, Zephyr, LLaMA-2-chat&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Pros:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;High generalization&lt;/li&gt;
&lt;li&gt;Works well with structured prompts&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Cons:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Cannot customize deeply&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  Summary Table
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Methods&lt;/th&gt;
&lt;th&gt;cost&lt;/th&gt;
&lt;th&gt;Setup complexity&lt;/th&gt;
&lt;th&gt;customization&lt;/th&gt;
&lt;th&gt;fresh knowledge&lt;/th&gt;
&lt;th&gt;Ideal For&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;RAG&lt;/td&gt;
&lt;td&gt;medium&lt;/td&gt;
&lt;td&gt;medium&lt;/td&gt;
&lt;td&gt;medium&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;Assistants, dynamic knowledge&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Finetuning&lt;/td&gt;
&lt;td&gt;high&lt;/td&gt;
&lt;td&gt;high&lt;/td&gt;
&lt;td&gt;high&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;td&gt;Specialized domain behavior&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Prompt engineering&lt;/td&gt;
&lt;td&gt;Low&lt;/td&gt;
&lt;td&gt;very Low&lt;/td&gt;
&lt;td&gt;low&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;Lightweight domain tasks&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;a href="https://buymeacoffee.com/stephenbj" rel="noopener noreferrer"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Ftam27klrepvq97jvits1.png" alt=" " width="700" height="700"&gt;&lt;/a&gt;&lt;br&gt;
 &lt;a href="https://buymeacoffee.com/stephenbj" rel="noopener noreferrer"&gt;buymeacoffee&lt;/a&gt;&lt;/p&gt;

</description>
      <category>llm</category>
      <category>rag</category>
      <category>ai</category>
      <category>python</category>
    </item>
    <item>
      <title>hey how to create plugin using react like dataTable.js?</title>
      <dc:creator>Stephen BJ</dc:creator>
      <pubDate>Thu, 08 Apr 2021 09:53:45 +0000</pubDate>
      <link>https://dev.to/jerin_stephen/hey-how-to-create-plugin-using-react-like-datatable-js-83o</link>
      <guid>https://dev.to/jerin_stephen/hey-how-to-create-plugin-using-react-like-datatable-js-83o</guid>
      <description>&lt;p&gt;hey guys i wounder is it possible to create plugin or library using react js or any other front end framework ? Please add comment or post related to this &lt;/p&gt;

</description>
      <category>javascript</category>
      <category>react</category>
      <category>vue</category>
    </item>
  </channel>
</rss>
