<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Joseph Zhang</title>
    <description>The latest articles on DEV Community by Joseph Zhang (@joseph_zhang_c2dc324f9062).</description>
    <link>https://dev.to/joseph_zhang_c2dc324f9062</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4238573%2F3a63ce35-97ab-430c-a5c7-ad7bdfdab0b2.png</url>
      <title>DEV Community: Joseph Zhang</title>
      <link>https://dev.to/joseph_zhang_c2dc324f9062</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/joseph_zhang_c2dc324f9062"/>
    <language>en</language>
    <item>
      <title>How I Built an AI Pipeline to Detect Misleading Job Listings</title>
      <dc:creator>Joseph Zhang</dc:creator>
      <pubDate>Wed, 11 Nov 2026 14:32:10 +0000</pubDate>
      <link>https://dev.to/joseph_zhang_c2dc324f9062/how-i-built-an-ai-pipeline-to-detect-misleading-job-listings-4n7g</link>
      <guid>https://dev.to/joseph_zhang_c2dc324f9062/how-i-built-an-ai-pipeline-to-detect-misleading-job-listings-4n7g</guid>
      <description>&lt;p&gt;Have you ever clicked on a job labeled &lt;strong&gt;"Remote"&lt;/strong&gt;, only to discover that you're expected in the office three days a week?&lt;/p&gt;

&lt;p&gt;I've run into this constantly.&lt;/p&gt;

&lt;p&gt;Some jobs are remote but only accept applicants from one country. Others are actually hybrid. And the same position often appears across multiple boards with conflicting information.&lt;/p&gt;

&lt;p&gt;While building &lt;a href="https://jobs.codemind.site" rel="noopener noreferrer"&gt;CodeMind Jobs&lt;/a&gt;, I realized that collecting job listings wasn't the hardest part.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The real challenge was figuring out what those listings actually meant.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;So I built an AI pipeline to analyze, validate, and clean up job data.&lt;/p&gt;

&lt;p&gt;Here's how it works.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why job board data is messy
&lt;/h2&gt;

&lt;p&gt;Job platforms collect listings from company career pages, ATS platforms like Greenhouse and Lever, and other sources.&lt;/p&gt;

&lt;p&gt;But there's no consistent format.&lt;/p&gt;

&lt;p&gt;Consider two job descriptions:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Candidates must work from our New York office three days per week.&lt;/p&gt;

&lt;p&gt;Work remotely from anywhere in the United States.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Both might arrive labeled "Remote."&lt;/p&gt;

&lt;p&gt;But they mean completely different things. The first is hybrid. The second is fully remote, but geographically restricted.&lt;/p&gt;

&lt;p&gt;I wanted CodeMind Jobs to understand that distinction rather than blindly trust the original labels.&lt;/p&gt;

&lt;h2&gt;
  
  
  How the pipeline works
&lt;/h2&gt;

&lt;p&gt;Every job goes through four main steps:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Collect:&lt;/strong&gt; Gather listings from career pages, ATS platforms, and job boards.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Analyze:&lt;/strong&gt; Use an LLM to extract structured information.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Validate:&lt;/strong&gt; Check the extracted data against the original description.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Deduplicate:&lt;/strong&gt; Identify repeated listings across different sources.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The interesting part is what happens during analysis and validation.&lt;/p&gt;

&lt;h3&gt;
  
  
  1. Extract structured data with AI
&lt;/h3&gt;

&lt;p&gt;I use OpenAI's structured outputs with &lt;code&gt;strict: true&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Instead of asking the model to summarize a job, I give it a JSON schema to fill in.&lt;/p&gt;

&lt;p&gt;Here's a simplified example:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nl"&gt;"workplace"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nl"&gt;"type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"remote | hybrid | onsite | unknown"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nl"&gt;"evidence"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"exact quote or null"&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nl"&gt;"location"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nl"&gt;"remoteEligibleCountries"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nl"&gt;"remoteScope"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"global | region | null"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nl"&gt;"mustResideIn"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nl"&gt;"timezone"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;null&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The actual schema also extracts salary, seniority, technical skills, experience requirements, and visa sponsorship.&lt;/p&gt;

&lt;p&gt;Two things matter here.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Remote work and location eligibility are separate.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;A job can be fully remote while requiring applicants to live in Germany.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;And &lt;code&gt;unknown&lt;/code&gt; is a valid answer.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;If the description doesn't clearly explain the work arrangement, I'd rather show "unknown" than make something up.&lt;/p&gt;

&lt;h3&gt;
  
  
  2. Validate the AI output
&lt;/h3&gt;

&lt;p&gt;This was the biggest lesson.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Valid JSON doesn't mean accurate information.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Even with structured outputs, models can confidently misinterpret descriptions.&lt;/p&gt;

&lt;p&gt;So I added a validation layer:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Evidence required:&lt;/strong&gt; The model must provide an exact quote supporting its classification.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Quote verification:&lt;/strong&gt; The evidence must actually appear in the original description.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Safe fallback:&lt;/strong&gt; Unsupported classifications become &lt;code&gt;unknown&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Independent validation:&lt;/strong&gt; A bad salary extraction doesn't invalidate the entire job.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Model escalation:&lt;/strong&gt; If &lt;code&gt;gpt-5-nano&lt;/code&gt; fails validation twice, I retry with &lt;code&gt;gpt-5-mini&lt;/code&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Evidence verification doesn't guarantee the interpretation is correct, but it prevents unsupported claims from silently entering the database.&lt;/p&gt;

&lt;p&gt;I also use OpenAI's Batch API to reduce costs.&lt;/p&gt;

&lt;p&gt;So far, enrichment costs are &lt;strong&gt;well under $0.001 per job&lt;/strong&gt;.&lt;/p&gt;

&lt;h3&gt;
  
  
  3. Remove duplicate listings
&lt;/h3&gt;

&lt;p&gt;Another problem: the same job appearing five times.&lt;/p&gt;

&lt;p&gt;I compare company names, titles, and descriptions.&lt;/p&gt;

&lt;p&gt;When the company and title match and descriptions are at least 80% similar, the listings become candidates for merging.&lt;/p&gt;

&lt;p&gt;I also prioritize direct company application links over third-party aggregators.&lt;/p&gt;

&lt;p&gt;The goal isn't simply to collect more jobs. It's to make the results more useful.&lt;/p&gt;

&lt;h2&gt;
  
  
  A real-world example
&lt;/h2&gt;

&lt;p&gt;Imagine receiving this job:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Source data:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Title: Senior Backend Engineer&lt;/li&gt;
&lt;li&gt;Workplace: Remote&lt;/li&gt;
&lt;li&gt;Location: United States&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;But the description says:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Employees must work from our Chicago office three days per week.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;The pipeline should classify it as &lt;strong&gt;hybrid&lt;/strong&gt;, preserve the Chicago requirement, and attach the supporting evidence.&lt;/p&gt;

&lt;p&gt;That's a small correction, but it can save someone from wasting time on an unsuitable job.&lt;/p&gt;

&lt;h2&gt;
  
  
  Three lessons I learned
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;1. Never assume source labels are correct.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;I considered skipping AI enrichment for jobs already marked on-site to save money.&lt;/p&gt;

&lt;p&gt;But in a test sample, AI classifications disagreed with the original labels roughly two-thirds of the time.&lt;/p&gt;

&lt;p&gt;That didn't prove the AI was right, but it showed the shortcut was risky. I dropped it.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. Let the model say "I don't know."&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Forcing a classification creates false confidence.&lt;/p&gt;

&lt;p&gt;Sometimes &lt;code&gt;unknown&lt;/code&gt; is the most accurate result.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;3. Validation can matter more than model size.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;A smaller model with evidence checks, field-level validation, and fallback logic can be more practical than always using a more expensive model.&lt;/p&gt;

&lt;h2&gt;
  
  
  What's next?
&lt;/h2&gt;

&lt;p&gt;I'm still working through tricky cases:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;"Remote (US preferred)"&lt;/li&gt;
&lt;li&gt;Jobs requiring Pacific Time overlap&lt;/li&gt;
&lt;li&gt;Contradictory location requirements&lt;/li&gt;
&lt;li&gt;Remote positions with unclear country restrictions&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;I'm building this into &lt;a href="https://jobs.codemind.site" rel="noopener noreferrer"&gt;CodeMind Jobs&lt;/a&gt;, where developers can search tech opportunities using enriched job information instead of relying only on source labels.&lt;/p&gt;

&lt;p&gt;One thing I've learned from building this:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Better job data is often more valuable than more job listings.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;I'm curious how other developers handle this.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;When using LLMs to classify messy real-world data, how do you decide whether to trust the result or fall back to &lt;code&gt;unknown&lt;/code&gt;?&lt;/strong&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>webdev</category>
      <category>showdev</category>
      <category>career</category>
    </item>
  </channel>
</rss>
