DEV Community: Webequipe

AI Agents Can Read Your Website — But Can They Read Your PDF

Webequipe — Thu, 09 Jul 2026 07:02:31 +0000

Websites are changing.

A few years ago, we mainly cared about human visitors and Google search. Now we also need to think about AI search tools, chatbots, RAG systems, and AI agents that read website content to answer questions.

But there is one problem many WordPress sites still ignore:

A lot of important content is locked inside PDFs.

And if that PDF content is not searchable or properly indexed, users may never find it.

PDFs Are Not Just Attachments

Many websites upload PDFs for things like:

manuals
reports
policy documents
forms
product guides
public notices
documentation
brochures

The problem is that most site owners treat these PDFs like simple downloadable files.

But if the PDF contains useful information, then it is not just a file.

It is part of your website content.

For example, a visitor may search: refund policy for international orders

Maybe the answer exists inside a PDF. But if your WordPress search cannot read inside that PDF, the visitor gets no result.

From the user’s point of view, the content does not exist.

AI Agents Have the Same Problem

AI tools are becoming better at reading web pages. But a random PDF link is not always enough.

An AI agent or chatbot needs clear, accessible, searchable content.

If your important information is inside a PDF but not extracted or indexed, then your site creates a gap:

Website pages = readable
PDF files = often hidden
Search result = incomplete
AI answer = missing context

This is a bigger issue for websites that depend on documents, such as universities, government websites, nonprofits, healthcare sites, legal sites, SaaS documentation, and product manual libraries.

Text PDFs vs Scanned PDFs

Not all PDFs are the same.

There are usually three types:

Text-based PDFs
These PDFs contain selectable text. The text can usually be extracted and indexed.
Scanned PDFs
These are basically images inside a PDF. You cannot properly search them unless OCR is used.
Mixed PDFs
Some pages have text, and some pages are scanned images.

This matters because your search system must understand what kind of PDF it is handling.

A simple text PDF may only need text extraction.
A scanned PDF needs OCR.

A Quick WordPress Check

If you are a developer, you can quickly check how many PDFs exist on a WordPress site:

$pdfs = get_posts([
    'post_type'      => 'attachment',
    'post_mime_type' => 'application/pdf',
    'post_status'    => 'inherit',
    'posts_per_page' => -1,
]);

echo 'Total PDFs found: ' . count($pdfs);

If the number is high, then your site probably has a lot of hidden content.

The next question is:

Can users actually search inside those PDFs?

Make PDFs More Useful

A better document structure can help both users and AI systems.

Instead of only adding a plain PDF link like this:

<a href="/files/safety-manual.pdf">Download PDF</a>

Add context around it:

<article class="document-card">
  <h2>Product Safety Manual</h2>
  <p>
    Includes installation steps, safety warnings,
    and maintenance instructions.
  </p>
  <a href="/files/safety-manual.pdf">View PDF</a>
</article>

This helps users understand the document before opening it.
It also gives search engines and AI tools better context.

A Simple PDF Search Workflow

A useful PDF search system should work like this:

PDF uploaded ↓ Text extracted or OCR processed ↓ Content indexed ↓ User searches from WordPress ↓ Relevant PDF result appears

That is the missing layer in many WordPress websites.

Where WebEquipe PDF Search Fits

For WordPress sites, a plugin like WebEquipe PDF Search can help make PDF content searchable.

You can add a PDF search form using a shortcode like:

[webequipe_pdf_search_form placeholder="Search PDFs..." button_text="Search" results_per_page="10"]

The free version is useful for text-based PDFs.
For scanned PDFs, OCR support is needed, which is where a Pro/OCR workflow becomes important.

The goal is not only to “search files.”

The real goal is to make PDF content part of the website experience.

Final Thought

AI agents, chatbots, and modern search tools are changing how people find information.

But if your most valuable content is hidden inside PDFs, your website may look complete while still being difficult to search.

So before adding another AI tool to your WordPress site, ask one simple question:

Can users — and AI systems — actually read what is inside your PDFs?

Why PDF Indexing in WordPress Needs Background Processing

Webequipe — Fri, 03 Jul 2026 06:20:24 +0000

Uploading a PDF to WordPress feels simple.

You add the file to the Media Library, WordPress stores it, and the file is ready to use.

But if you want that PDF to become searchable, the work does not stop at upload.

The plugin has to read the PDF, extract the text, clean the content, store it in a searchable index, and later match that content with user search queries.

That sounds fine for one small PDF.

But what happens when the file has 100 pages?

Or when the website has hundreds of PDFs?

Or when the PDF is scanned and needs OCR?

That is where background processing becomes important.

PDF Indexing Is Heavier Than Normal WordPress Tasks

Most WordPress content is already stored in the database.

When you publish a post, the title and body content are saved in fields like post_title and post_content. Search can query that data directly.

PDFs are different.

A PDF is a file. WordPress stores the file information, but it does not automatically read the actual text inside that file.

So a PDF search plugin needs to do extra work:

For a small text-based PDF, this may happen quickly.

But large files can take more time. Some PDFs have complex layouts, repeated headers, tables, images, or broken text structure. Scanned PDFs are even heavier because they need OCR before any text can be indexed.

Trying to do all of this instantly during upload is risky.

The Problem With Processing During Upload

When a user uploads a PDF, WordPress is handling that request in real time.

If your plugin starts parsing a large PDF immediately, several things can go wrong:

The upload request may become slow
PHP execution time may run out
Memory usage may increase
The admin screen may freeze
OCR processing may take too long
The user may not know whether indexing succeeded or failed

This creates a bad experience.

The user only wanted to upload a document. They should not have to wait while the system tries to fully process a 200-page PDF in the same request.

That is why PDF indexing should be treated as a background job, not an upload-time task.

A Better Flow: Queue the Work

A more reliable approach is to separate file upload from file processing.

The upload should stay fast. After upload, the plugin creates an indexing job and processes it later.

A simple flow looks like this:

This makes the system much easier to manage.

Instead of forcing everything to happen immediately, the plugin can process PDFs one by one in the background.

For example, the plugin can store a job like this:

[
    'attachment_id' => 245,
    'status'        => 'pending',
    'type'          => 'text_extraction',
    'attempts'      => 0,
    'created_at'    => current_time('mysql'),
]

Then a background worker can pick up pending jobs and process them safely.

Status Tracking Matters

Background processing also makes status tracking possible.

A PDF can move through states like:

pending
processing
indexed
ocr_required
failed

This is useful for admins.

Without status tracking, users have no idea what happened. A PDF may be uploaded, but not searchable. Or OCR may fail silently. Or the file may be too large to process.

A clear status helps users understand the situation.

For example:

“Pending” means the file is waiting to be processed
“Processing” means indexing is running
“Indexed” means the PDF is searchable
“OCR required” means normal text extraction found little or no text
“Failed” means the job needs attention or retry

This makes the plugin feel more predictable.

OCR Makes Background Processing Even More Important

Text-based PDFs can often be processed with a parser.

Scanned PDFs are different.

A scanned PDF is basically an image inside a PDF file. A normal parser may return empty content because there is no text layer.

In that case, OCR is needed.

OCR usually means:

Convert PDF page to image
↓
Send image to OCR engine
↓
Receive detected text
↓
Store text in search index

This is much heavier than normal text extraction.

If the plugin uses an external OCR service like Google Vision API, the processing time also depends on API response time, page count, network conditions, and retry handling.

Running this during upload is not a good idea.

A queue-based system gives the plugin room to process OCR safely without breaking the admin experience.

Retry Logic Helps With Real-World Failures

Background jobs can fail for many reasons.

The file may be corrupted.
The server may run out of memory.
The OCR API may timeout.
The PDF parser may fail on a strange file structure.

Instead of failing permanently, the plugin can retry the job.

For example:

if ($job['attempts'] < 3) {
    retry_indexing_job($job['id']);
} else {
    mark_job_as_failed($job['id']);
}

This makes the system more resilient.

Not every failure needs to become a support issue. Some failures can be retried automatically.

Final Thoughts

PDF search looks simple from the outside.

A visitor searches, and a PDF appears in the result.

But behind that simple experience, the plugin has to extract content, detect scanned files, run OCR when needed, store searchable text, and handle failures.

That is why background processing matters.

It keeps uploads fast.
It avoids PHP timeout issues.
It makes OCR manageable.
It allows status tracking.
It gives the system a way to retry failed jobs.

For WebEquipe PDF Search, this kind of thinking is important because PDF indexing is not just a search feature. It is a processing pipeline.

And when a WordPress plugin handles heavy file processing, the best experience is usually the one users do not have to wait for.

Why WordPress Search Ignores Your PDFs — and How to Fix It

Webequipe — Thu, 25 Jun 2026 11:41:45 +0000

WordPress search works well when your content lives inside posts and pages.

But if important information is inside PDF files, WordPress often misses it completely.

A visitor may search for a phrase that clearly exists inside a PDF manual, report, form, or policy document. But WordPress returns no result.

The PDF is uploaded.
The content exists.
But search still cannot find it.

So why does this happen?

WordPress Search Does Not Read PDF Content

Default WordPress search mainly checks database fields like:

post_title
post_content
post_excerpt

When you publish a post, the content is stored in post_content, so WordPress can search it.

But when you upload a PDF, WordPress treats it as a media attachment. It stores the file title, URL, MIME type, upload date, and some metadata.

What it does not do by default is open the PDF, read the text inside it, and save that text into post_content.

So this kind of situation is common:

PDF file: employee-handbook.pdf
Text inside PDF: "Remote work requests must be approved by HR"
WordPress searchable content: filename and attachment data only
Search query: "remote work requests"
Result: no match

The text is inside the file, but not inside the database fields WordPress normally searches.

That is the main reason PDF search fails.

Not All PDFs Are the Same

Before fixing PDF search, you need to understand one important difference.

There are two common types of PDFs:

Text-based PDFs
Scanned PDFs

They may look the same in the browser, but technically they behave very differently.

Text-Based PDFs

A text-based PDF contains real selectable text.

If you can open a PDF, select a sentence, copy it, and paste it into a text editor, it probably has a text layer.

These PDFs are usually exported from tools like Word, Google Docs, InDesign, or reporting software.

For these files, a plugin can use a PDF parser to extract the text.
In PHP, one common library is smalot/pdfparser.

use Smalot\PdfParser\Parser; 
$parser = new Parser(); 
$pdf = $parser->parseFile($file_path); 

$text = $pdf->getText(); 
if (! empty($text)) { 
// Store extracted text for search 
}

After extraction, the text needs to be stored somewhere searchable.

For small use cases, post meta may work. But for better control and performance, a custom database table is usually better.

Example:

CREATE TABLE wp_pdf_search_index ( 
  id BIGINT UNSIGNED AUTO_INCREMENT PRIMARY KEY, 
  attachment_id BIGINT UNSIGNED NOT NULL, 
  extracted_text LONGTEXT NOT NULL, 
  indexed_at DATETIME NOT NULL, 
  FULLTEXT KEY pdf_text_index (extracted_text) 
);

Then the plugin can search the extracted PDF content:

SELECT attachment_id 
FROM wp_pdf_search_index 
WHERE MATCH(extracted_text) 
AGAINST ('remote work requests' IN NATURAL LANGUAGE MODE);

So the PDF itself is not searched directly every time.

Instead, the plugin extracts the PDF text once, stores it, and searches the stored index.

Showing PDF Results in WordPress Search

Once PDF content is indexed, the next step is showing matching PDF files in the search results.

A plugin can hook into WordPress search using pre_get_posts, or it can provide a separate PDF search form.

For example, a shortcode-based PDF search form can be placed on a resource page:

[webequipe_pdf_search_form]

[webequipe_pdf_search_form placeholder="Search PDFs…" button_text="Search" results_per_page="10"]

This is useful for documentation pages, school notice sections, report libraries, member portals, or product manual pages.

The Scanned PDF Problem

Text extraction works only when the PDF contains real text.

But many PDFs are scanned documents.

Examples:

Old reports
Signed forms
Scanned notices
Paper manuals
Archive documents
Government or legal documents

A scanned PDF may look readable to a human, but technically each page is just an image.
So when a parser runs this:

$text = $pdf->getText();

It may return:

There are visible words on the page, but no machine-readable text layer.

This is where normal PDF parsing stops.

OCR Is Needed for Scanned PDFs

For scanned PDFs, the solution is OCR.

OCR means Optical Character Recognition. It reads text from images and converts it into machine-readable text.

The flow looks like this:

Google Vision API is one common OCR option. It can read pixel-based document images and return detected text.

A simplified flow:

$image_path = '/tmp/page-1.png'; 
$ocr_text = run_google_vision_ocr($image_path); 
if (! empty($ocr_text)) { 
  index_pdf_text($attachment_id, $ocr_text); 
}

The key idea is simple:

Text-based PDFs need parsing.
Scanned PDFs need OCR first.

If a search plugin treats both the same way, scanned PDFs will usually fail.

A Simple Detection Strategy

Most users do not know whether their PDF is text-based or scanned.

So the plugin should decide automatically.
A practical approach is:

$text = extract_pdf_text($file_path); 
if (strlen(trim($text)) > 100) { 
  index_pdf_text($attachment_id, $text);
} else { 
  queue_pdf_for_ocr($attachment_id); 
}

First, try normal extraction.

If enough text is found, index it.

If the result is empty or too short, send the file to OCR.

For large PDFs or OCR processing, this should run in the background. Otherwise, uploads can become slow and PHP timeouts can happen.

A better flow is:

In WordPress, this can be handled with WP-Cron, Action Scheduler, or a custom queue system.

Final Thoughts

WordPress search ignores PDFs because PDF content is not stored in the database fields WordPress normally searches.

To fix it, you need a separate indexing pipeline.

For text-based PDFs, extract the text and store it in a searchable index.

For scanned PDFs, run OCR first, then store the detected text.

That is the technical foundation of PDF search in WordPress.

We built this into WebEquipe PDF Search. The free version helps index text-based PDFs, and the Pro version adds OCR support for scanned PDFs.

But the bigger idea is simple:

Uploading a PDF does not automatically make its content searchable.

If your WordPress site depends on PDFs, those files need to be treated as part of your search index.