DEV Community: Julia

Is PDF a problem for AI?

Julia — Thu, 10 Sep 2026 06:10:18 +0000

Boris Doubrov, CEO of Dual Lab sprl, has written the article exploring the relationship between PDF and AI for the PDF Association.

Every organization relies on documents: contracts, financial reports, research papers, compliance filings, insurance policies, technical manuals, and government records. Many of these documents are PDFs.

But what happens when these documents become input for AI?

An AI system may have access to a PDF, but it does not necessarily receive all the information contained in it. During ingestion, a document can be reduced to plain text, processed with OCR, analyzed page by page, or converted to Markdown. Tables can lose their relationships, reading order can change, annotations can disappear, and metadata or semantic structure can be ignored.

The result is a gap between what the PDF contains and what the AI system actually understands.

This is the real PDF problem for AI.

But there is an important distinction: PDF itself is not inherently a bad format for AI. The problem is often the way PDF is created, interpreted, and ingested.

Why PDF can be challenging for AI

PDF is a page-oriented document format designed to preserve content and appearance across systems. But a standard PDF does not necessarily contain the semantic information that tells a machine what every piece of content means.

This creates several challenges:

Visual structure does not always equal semantic structure: PDF can describe text and graphics as positioned objects, but a visually obvious heading, list, or table is not necessarily represented with its logical meaning. Tagged PDF can provide additional information about logical reading order, headings, lists, tables, language, and alternative text, but not all PDF files are tagged.
Scanned documents may contain no usable text layer.

OCR may be necessary, but it introduces another recognition step and can produce errors.
Complex layouts require interpretation.

Multi-column pages, tables, footnotes, sidebars, figures, and content spanning page boundaries can be difficult to reconstruct from raw text.
Text encoding can create problems.

Poor Unicode mappings or malformed PDFs can make otherwise visible text difficult to extract reliably.
More than visual rendering.

PDF also contains metadata, bookmarks, digital signatures, attachments and other supplementary information which may be crucial for a user workflow, but is often ignored by AI.

So PDF is not “unreadable by machines.”

Rather, PDF can contain several layers of information, and AI systems do not always extract all of them.

The PDF AI actually receives may not be the PDF you created

This is one of the most important points in the PDF Association FAQ.

Different AI systems use different PDF ingestion strategies. Some systems may ignore bookmarks, layers, annotations, metadata, or Structure Tree. Others may apply OCR even when the PDF already contains extractable text. Some reduce the entire document to a stream of plain text.

If the ingestion pipeline extracts only visible characters, the AI receives only a part of the document. That means two AI systems can receive the same PDF but effectively process different documents.

This can explain why one AI tool answers a question correctly while another misses a table, misunderstands reading order, or produces an incorrect answer.

Tagged PDF can give AI a structural advantage

Tagged PDF gives AI important structural information, including headings, paragraphs, lists, tables, and figures. However, a structure tree does not guarantee perfect semantics; tags can be incomplete or incorrect.

This is similar to HTML, where semantic structure may also require additional technologies and guidance such as WAI-ARIA and WCAG. In PDF, standards such as WTPDF and PDF/UA provide additional requirements for the quality of structured and accessible PDF documents.

Still, the fact that PDF creators added Tagged structure is a step forward and provides a potential source of valuable information for AI.

OCR is not the universal solution

A common PDF-to-AI pipeline looks like this:

PDF → OCR → text → LLM

But, as the PDF Association says in its FAQ, not every PDF needs OCR.

Most born-digital PDFs already contain extractable text. For these documents, native text extraction is generally preferable to OCR. OCR is slower, more costly, and can introduce recognition errors. It also recovers only visible text and does not replace other PDF information such as annotations, metadata, semantic structure, or embedded content.

Native PDF extraction can also access information that OCR cannot reliably recover, including intended Unicode mappings, intended textual representations of images (ActualText), and text contained in annotations or other non-page content.

OCR remains important for scanned documents and situations where PDF text cannot be reliably extracted.

A better workflow is therefore:

Detect → Extract native content → Analyze → OCR only when necessary

OCR should be a fallback, not an automatic first step.

Converting PDF to Markdown is not always better

Another popular workflow is:

PDF → Markdown → LLM

Markdown is easy for developers and language models, but converting every PDF into Markdown can be counterproductive.

The PDF Association describes conversion to simpler formats as potentially lossy because PDF contains features that do not have direct equivalents in Markdown or HTML. Complex tables, merged cells, digital signatures, layers, annotations, and other semantics can be simplified or lost during conversion.

For a simple document, Markdown may be an excellent intermediate representation. But for a complex PDF, less structure does not necessarily mean better AI understanding.

The goal should not be to make the document as simple as possible. The goal should be to create a machine-readable representation while preserving meaningful information and relationships.

Page-by-Page processing can lose context

PDF is paginated, but the logical document is not necessarily organized around pages.

A sentence can continue onto the next page, or even several pages ahead. A table can span several pages, both vertically and horizontally. A heading can appear at the bottom of one page while its content begins on another.

The PDF Association therefore advises against treating each PDF page as an independent AI input. Page isolation can reduce context and increase the risk of incorrect interpretation.

However, page information remains important for attribution and verification. A good AI system should preserve both logical document structure and physical page references.

This is especially important in RAG systems, where users need to verify where an answer came from.

Metadata, Annotations and Attachments Matter

A PDF ingestion system should not automatically discard metadata.

The PDF Association specifically recommends that AI systems make use of PDF metadata because it can contribute context and potentially reduce processing requirements.

Annotations are also important.They can represent comments, markup, links, digital signatures, multimedia, attachments, or proposed changes to document content. Ignoring them can mean ignoring information relevant to the document's context.

Attachments need not be referenced by annotations but are nonetheless part of the document; these must also be considered.

Redaction is a special AI risk

There is also an important security issue.

A correctly redacted PDF has the sensitive information removed. But a PDF containing redaction annotations may represent an incomplete redaction workflow; the underlying information may still exist in the file. A PDF may also contain other types of annotations that mask (but do not remove) text; a very common type of redaction failure.

An AI system processing such a file could potentially ingest information that the author believed had already been removed.

Visual appearance is not the same as document state.

Very large, old, or invalid PDFs create additional challenges

PDF is designed to be backward-compatible, so even very old PDF documents can potentially be processed by modern systems. But malformed, truncated, or corrupted PDFs can behave differently depending on the software used to recover and interpret them. Different AI systems may therefore recover different information from the same damaged file.

AI processing can also be slower for PDFs than for HTML because PDFs are frequently long, multi-page documents. Unnecessary OCR adds additional processing overhead.

This is another reason that PDF-aware preprocessing matters.

So, is PDF actually a problem for AI?

Yes but !! not in the way it is usually described.

PDF itself is not inherently unsuitable for AI.

In fact, PDF is valuable precisely because it often contains high-density, long-form, persistent information that organizations need AI to understand.

The real challenge lies in PDF ingestion processes.

If an AI system converts a complex PDF into an incomplete stream of text, it may discard the very information needed to interpret the document correctly.

Conclusion

PDF is not the enemy of AI. PDF should not be automatically OCR-processed or converted into a simpler format just because image recognition is available or Markdown and plain text are easier to process. Such conversion can discard semantics, structure, and context.

The better approach is PDF-aware AI ingestion: preserve native text where available, use structural information such as Tagged PDF, retain relevant metadata and annotations, maintain page references, and use OCR only when necessary.

The goal is not simply to extract text from a PDF but to preserve all information within the document and all relationships that make the document meaningful.

Contact:
info@pdf4wcag.com

pdf4wcag.com

Dual Lab releases PDF4WCAG Accessibility Checker 1.12

Julia — Fri, 04 Sep 2026 05:42:05 +0000

Dual Lab announces the release of PDF4WCAG Accessibility Checker 1.12. The release introduces improvements to accessibility and keyboard navigation, error visualization, validation accuracy, localization to Korean language and application stability.

PDF4WCAG Accessibility Checker is designed to help organizations evaluate PDF documents against PDF/UA, WCAG, and WTPDF accessibility requirements. The solution is powered by the veraPDF validation architecture and provides machine-verifiable checks based on the PDF/UA and WTPDF validation profiles.

What’s new in PDF4WCAG 1.12

Improved accessibility and user experience

Version 1.12 improves the PDF4WCAG interface and makes the application easier to navigate and use.

Key improvements include:

Enhanced accessibility and keyboard navigation.

Better error visualization

PDF4WCAG 1.12 improves the way validation errors are presented and connected to the relevant elements in a PDF.

The release includes:

Improved handling of annotation-related errors, including missing highlights and corresponding properties in the annotation panel.
Support for displaying metadata errors.
Improved highlighting of errors related to missing language identification.

Validation improvements

The new release includes several updates designed to improve the accuracy and consistency of validation results.

Version 1.12:

Fixes reporting for ISO 14289-2:2024 8.2.5.20 (Link annotations targeting different locations)
Fixed LaTeX specific errors in human checks.
Removes selected WCAG rules that are no longer applicable to the validation workflow.

These changes help provide more precise and actionable validation results for PDF accessibility professionals.

Localization updates

PDF4WCAG 1.12 expands localization with the addition of the Korean language.

The release includes:

Korean interface translations.
Korean translations for validation error messages.
Support of CJK fonts in PDF reports.
Translated documentation.

The localization updates make PDF accessibility testing more accessible to Korean-speaking users and organizations.

Application improvements

Additional application improvements include:

Improved veraPDF log and warning reporting to users.
Updated and clarified error messages.
Improvements to the verapdf-js-viewer documentation.
Improved session management.

Supporting PDF accessibility and compliance

The combination of machine-verifiable validation, detailed error reporting, and an interactive interface helps accessibility teams, developers, document creators, and compliance professionals identify and address PDF accessibility problems more efficiently.

PDF4WCAG 1.12 continues Dual Lab’s focus on making PDF accessibility validation more accurate, transparent, and practical for organizations working toward PDF/UA, WCAG, and WTPDF compliance.

Contact us:

pdf4wcag.com

info@duallab.com

info@pdf4wcag.com

Dual Lab releases PDF4WCAG Accessibility Checker 1.12

Julia — Wed, 02 Sep 2026 11:15:55 +0000

What’s new in PDF4WCAG 1.12

Improved accessibility and user experience

Version 1.12 improves the PDF4WCAG interface and makes the application easier to navigate and use.

Key improvements include:

Enhanced accessibility and keyboard navigation.

Better error visualization

PDF4WCAG 1.12 improves the way validation errors are presented and connected to the relevant elements in a PDF.

The release includes:

Improved handling of annotation-related errors, including missing highlights and corresponding properties in the annotation panel.
Support for displaying metadata errors.
Improved highlighting of errors related to missing language identification.

Validation improvements

The new release includes several updates designed to improve the accuracy and consistency of validation results.

Version 1.12:

Fixes reporting for ISO 14289-2:2024 8.2.5.20 (Link annotations targeting different locations)
Fixed LaTeX specific errors in human checks.
Removes selected WCAG rules that are no longer applicable to the validation workflow.

These changes help provide more precise and actionable validation results for PDF accessibility professionals.

Localization updates

PDF4WCAG 1.12 expands localization with the addition of the Korean language.

The release includes:

Korean interface translations.
Korean translations for validation error messages.
Support of CJK fonts in PDF reports.
Translated documentation.

The localization updates make PDF accessibility testing more accessible to Korean-speaking users and organizations.

Application improvements

Additional application improvements include:

Improved veraPDF log and warning reporting to users.
Updated and clarified error messages.
Improvements to the verapdf-js-viewer documentation.
Improved session management.

Supporting PDF accessibility and compliance

Contact us:

pdf4wcag.com

info@duallab.com

info@pdf4wcag.com

Analysis of PDF file size for 2006-2025 by Dual Lab

Julia — Fri, 21 Aug 2026 07:55:59 +0000

Analysis of PDF file size for 2006-2025 by Dual Lab

Dual Lab analyzed the complete June 2026 Common Crawl dataset (CC-MAIN-2026-25), comprising 20,578,394 PDF documents. In the second part of the analysis we analyze the evolution of the median PDF file size over the past 20 years.

Executive Summary

One of the reasons for the ongoing development of the Brotli compression in PDF format is the growing size of modern documents. It would be reasonable to assume, therefore, that PDF files are growing substantially larger. However, our analysis indicates otherwise.

Brotli compression has been introduced into the PDF specification to improve compression efficiency and reduce file sizes, particularly for modern workflows that embed large amounts of data and resources.

Despite these technological advances, our analysis tells a different story about the actual size of PDF documents published on the web.

This publication in no way diminishes the importance of Brotli. We fully recognize its value, while noting that PDF archive sizes remain an important consideration for storage, transfer, and processing efficiency.

Use median instead of arithmetic mean

Because PDF file sizes span several orders of magnitude, the arithmetic mean can be strongly affected by a relatively small number of exceptionally large files.

Figure 1 below compares the mean with the median file size per each month for the past 20 years. It shows that the monthly mean is roughly 3-5 times the median for much of the period. More importantly, the mean has large isolated excursions, most dramatically around 2012-2013, where it jumps to almost 5 MiB, while the median hardly reacts.

Figure 1. Arithmetic mean versus median

Figure 2 quantifies this particularly well: the largest 1% of PDFs account for roughly 25-35% of all bytes in a typical month, with occasional values around 40%. One percent of observations contributing around one third of the quantity being averaged means the arithmetic mean is inherently highly sensitive to that 1%.

Figure 2. Share of largest 1% of PDFs

To account for the skewed distribution and the influence of outliers, we complement the mean with the median, 25th–75th percentiles (interquartile range), and 10th–90th percentiles. This combination distinguishes changes in the overall average file size from changes in the size of a typical PDF.

Use Logarithmic scale for PDF File sizes

PDF document sizes vary by several orders of magnitude, from small text documents to publications containing thousands of pages and high-resolution images. To understand this distribution, Figure 3 presents PDF file sizes using exponentially increasing bins (powers of two).

Figure 3. Size of PDF files

Using powers of 2 for the bins reveals an approximately log-normal distribution, making logarithmic scales more suitable than linear ones for analyzing PDF file sizes. Since file sizes grow multiplicatively, this approach provides a more accurate view of the distribution across the full range of document sizes.

This again reconfirms the reason why we use the mean to examine overall changes in average file size, and the median and quartiles to assess long-term changes in typical PDF size.

Evolution of monthly median and quartiles

Based on the above reasons, we perform long term analysis of the median file size together with the 25th–75th percentiles (interquartile range).

Figure 4. Monthly PDF file size distribution

Figure 4 shows the monthly evolution of PDF file sizes over the past 20 years using the median and the 25-75 quartiles on a logarithmic scale. The median and interquartile range provide a more robust representation of typical PDF sizes and their variation over time.

Use quadratic regression instead of linear

Figure 4 suggests that the growth in median PDF file size is nonlinear. The increase was most pronounced between 2006 and 2016, after which the trend gradually flattened and even showed a slight decline in recent years. To model this behavior more accurately, we fitted a quadratic regression using the formula

log_median ~ years + years^2

The standard regression analysis shows that both the linear and quadratic terms are highly statistically significant (p < 0.001), confirming that the evolution of PDF file sizes cannot be adequately described by a simple linear trend. The regression results are presented in Figure 5.

Figure 5. Regression results

Figure 6 illustrates the monthly median PDF file size together with the fitted quadratic trend. The gray line shows the observed monthly median values, while the blue curve represents the fitted regression model.

Despite noticeable month-to-month variation, the long-term trend is clear. The monthly median PDF file size increased steadily from approximately 0.20 MB in 2006 to just over 0.40 MB around 2021–2022, after which the trend leveled off and began to decline slightly. This suggests that the continuous growth in PDF file sizes observed over the previous decade has slowed, indicating that the typical size of PDF documents published on the web has stabilized in recent years.

Figure 6. Trends in PDF file-size median

Annual growth of median file size in values (percentage)

The quadratic regression model also reveals a clear slowdown in the annual growth of median PDF file size. The median increased by approximately 9.2% between 2006 and 2007, but the annual growth rate gradually declined over time, falling below 1% by 2020–2021. Since 2022, the trend has become slightly negative, indicating that the median PDF file size has stabilized and is beginning to decrease modestly. These results suggest that the long-term growth in PDF file sizes observed during the 2000s and 2010s has largely reached a plateau.

Annual increase of median file size (percentage):

2006-2007: 9.21%
2007-2008: 8.59%
2008-2009: 7.97%
2009-2010: 7.36%
2010-2011: 6.75%
2011-2012: 6.14%
2012-2013: 5.54%
2013-2014: 4.94%
2014-2015: 4.34%
2015-2016: 3.75%
2016-2017: 3.16%
2017-2018: 2.57%
2018-2019: 1.99%
2019-2020: 1.41%
2020-2021: 0.83%
2021-2022: 0.26%
2022-2023: -0.31%
2023-2024: -0.88%
2024-2025: -1.44%
2025-2026: -2.00%

Median file size of Tagged vs. Untagged PDFs

During the reserch we have analyzed the median file size of Tagged versus Untagged PDFs. Initial guess was that Tagged PDFs should be larger in average, as PDF structure tree adds more objects to the document. However, this turned completely wrong! Figure 7 shows that Tagged PDFs are systematically smaller than Untagged ones.

Figure 7. Tagged vs Untagged PDFs

To be absolutely sure, we applied the Wilcoxon signed-rank test, a non‑parametric statistical test, which shows statistically significant difference in median file sizes between tagged and untagged PDFs per month (P-value = 1.65875e-13).

It is not clear why Tagged PDFs turn out to be smaller in average. One of the conjectures is that Tagged PDFs are mostly digitally born and do not include scanned files, which tend to be larger. We shall analyze this conjecture in the next parts of our analysis report.

Conclusions

Analysis of more than 20.5 million PDF documents from June 2026 Common Crawl collection leads to several key findings.

PDF file sizes follow an approximately log-normal distribution, making logarithmic visualization and median-based statistics more appropriate than arithmetic averages.
Median file size increased steadily until approximately 2021, after which growth flattened and became slightly negative.
Quadratic regression confirms that this slowdown is highly statistically significant.
The evidence suggests that the typical PDF published on the web has not become substantially larger over time.
Despite initial guesses, Tagged PDFs turn out to be smaller in average than Untagged ones.

These findings demonstrate the value of large-scale empirical analysis. Although technologies such as Brotli address the needs of increasingly sophisticated PDF documents, the overall characteristics of publicly available PDFs have remained remarkably stable.

In the next parts of our PDF analysis we shall discuss distribution of PDFs by Producer and the statistics on the use of structure elements and the validity of the structure tree against schemas defined in PDF standards.
Contact us : info@duallab.com

Website:

https://pdf4wcag.com/company/

https://duallab.com/

Your LLM is not a PDF parser: use OpenDataLoader first

Julia — Tue, 11 Aug 2026 11:29:53 +0000

Large Language Models (LLMs) have become remarkably powerful at understanding documents. Many modern AI platforms can accept PDF files directly, creating the impression that PDFs are ready-to-use inputs for AI workflows.

A PDF is not a plain text document. It is a complex format that contains layout information, text objects, images, tables, fonts, annotations, metadata, and sometimes a logical structure tree. The visual appearance of a PDF page does not always represent the correct reading order or semantic relationships between elements.
If PDF content is extracted incorrectly before reaching the LLM, the model receives incomplete or disorganized information. Problems such as broken reading order, corrupted tables, missing hierarchy, and lost relationships between elements directly affect the quality of AI-generated answers.

Even the best prompt cannot fix incorrect document parsing.

The solution is simple:

Parse the PDF first, then send structured content to the LLM.

Parse first, prompt second

A common mistake in AI workflows is sending a raw PDF directly into an LLM or RAG pipeline.

A better approach is:
PDF ⇒ OpenDataLoader ⇒ Structured Data ⇒ LLM

OpenDataLoader PDF converts PDF documents into AI-ready formats while preserving the original semantics of the document.

Supported output formats include:

Markdown, JSON, HTML, plain text.

Instead of forcing an LLM to interpret a complex PDF file, developers may provide clean, structured information optimized for AI processing.

Example: Convert a PDF for LLM processing

We provide a Python Installation guide

**Requires: **Java 11+ and Python 3.10+

Before you start: run java -version. If not found, install JDK 11+ from Adoptium.

Installing OpenDataLoader:

pip install -U opendataloader-pdf

Python script to convert multiple PDFs into AI-friendly formats:

import opendataloader_pdf opendataloader_pdf.convert( input_path=["document.pdf"], output_dir="output/", format="json,html,pdf,markdown" )

Code from OpenDataLoader.com
https://github.com/opendataloader-project/opendataloader-pdf
The user can run it from a Python shell or can create a Python script file first and then run it from the shell.
Instructions for Node.js | Java is also available on OpenDataloader official website.

The generated Markdown can be used directly for LLM conversations and summarization, while the JSON output is suitable for RAG pipelines, vector databases, and AI agents that require structured document information.

Figure 1. Results with PDF

Figure 2. Results with Markdown

In the first Figure, the LLM had to interpret the 1.4 MB, 16-page PDF file directly, relying on its vision capabilities. In the second example, the same file was provided as structured Markdown, allowing the model to immediately understand the document hierarchy and data relationships. By separating document parsing from LLM reasoning, OpenDataLoader produces more reliable, consistent, and efficient AI workflows.

Conclusion

Using OpenDataLoader to convert the 1.4 MB, 16-page PDF file into Markdown before sending it to an LLM significantly reduces both processing time and cost.

Compared with processing the PDF directly:

API processing was approximately 2.4× faster (38 s - 16 s).
Inference cost was approximately **2.9× lower ($0.35 - $0.12), a 66% cost reduction.
Input token usage decreased **by approximately 64% (29.2k - 10.6k tokens).
The LLM received structured Markdown instead of having to reconstruct the document layout itself, allowing it to focus on reasoning rather than PDF parsing.

While the exact savings depend on the document and the LLM, this example demonstrates that preprocessing PDFs with OpenDataLoader can substantially improve the efficiency of AI workflows while reducing both latency and inference costs. To perform this operation, users should have basic scripting skills.

Why Raw PDF Parsing Breaks AI Applications

PDF files are designed primarily for visual presentation, not direct machine understanding. A document can appear perfect to a human reader while still being difficult for an AI system to interpret correctly.

This is especially important for RAG systems, where incorrect extraction can lead to incomplete or misleading context. OpenDataLoader preserves document structure and converts PDFs into structured outputs optimized for AI workflows, including LLM applications, Retrieval-Augmented Generation (RAG), semantic search, knowledge bases, and document automation.

The key difference is that OpenDataLoader provides structured understanding of documents, not just extracted text.

Tell your LLM to use OpenDataLoader

For AI assistants, agents, and custom GPT workflows, OpenDataLoader can become the default PDF preprocessing step.

Instead of:
Analyze this PDF.

Use instructions such as:
Whenever a PDF is provided, first process it with OpenDataLoader. Use the generated Markdown or JSON output as the source for all analysis, retrieval, and reasoning. Do not rely on built-in PDF parsing unless OpenDataLoader output is unavailable.

This creates a consistent workflow where every PDF is processed before the LLM starts generating answers.

Clean Markdown for Chat, JSON for RAG

Different AI applications require different output formats.

Markdown

Markdown is ideal for: AI assistants; document summarization; question answering; conversational workflows.
It keeps headings, paragraphs, and lists structured while remaining easy for LLMs to process.

JSON

JSON is recommended for: RAG pipelines; vector databases; AI agents; structured extraction; document search.

OpenDataLoader JSON includes structured elements together with bounding box information. This allows applications to connect retrieved information back to its original location in the PDF, improving transparency and citation workflows.

Local, Deterministic Processing for AI Pipelines

One of the important advantages of OpenDataLoader is that it can run locally.

This provides:

deterministic results : the same PDF produces the same output;
improved privacy : documents do not need to be uploaded to external services;
predictable processing pipelines;
no dependency on external APIs for basic parsing.

For organizations processing confidential documents such as contracts, financial reports, technical documentation, or research papers, local processing is often an important requirement.

Conclusion

The quality of an LLM response depends heavily on the quality of the information provided to it. Feeding raw PDFs directly into an LLM often transfers the hardest part of the problem document understanding to the model.

A more reliable workflow is:
PDF → OpenDataLoader → Markdown / JSON → LLM

By using OpenDataLoader as the PDF parsing layer, developers can provide LLMs with structured, layout-aware, and machine-readable content. This improves retrieval accuracy, reduces parsing errors, and creates more reliable AI applications built on PDF documents.

Contact us

Website: https://opendataloader.org/docs
GitHub: https://github.com/opendataloader-project/opendataloader-pdf

PDF trends 2026Q2 by Dual Lab company

Julia — Mon, 03 Aug 2026 14:38:16 +0000

PDF trends 2026Q2 by Dual Lab company

Analysis of 20.6 Million PDF Documents from the June 2026 Common Crawl Dataset

Executive Summary

PDF remains one of the most widely used formats for publishing digital information, yet accessibility continues to be a major challenge. To better understand the current state of PDF accessibility, Dual Lab analyzed the complete June 2026 Common Crawl dataset (CC-MAIN-2026-25), containing 20,578,394 PDF documents.

This report extends our previous study of approximately 15 million PDFs from CC-MAIN-2026-04 and presents new data on encryption, permission flags, document size, page counts, annotations, PDF versions, and document age.

The analysis provides a large-scale view of how PDF technology is used across the public web and establishes a foundation for future reports on PDF in general with focus on Tagged PDF, PDF/UA adoption, and accessibility trends.

Research Scope and Methodology

The study analyzed every PDF referenced in the June 2026 Common Crawl (CC-MAIN-2026-25) dataset.

Because Common Crawl stores only the first 5 MB of each PDF, documents exceeding this size were downloaded directly from their original URLs to enable complete analysis.

The final dataset contains:

20,578,394 PDF documents
approximately 38 TB of source data

For each PDF we extracted:

basic metadata:
- page count
- file size
- creation and modification dates
- PDF version (including Version entry in the document catalog)
- producer and creator
encryption information and permissions
annotations
presence of interactive forms
presence of optional content layers
presence of digital signatures
image only (scanned) pages
Tagged PDF information:
- stats on the use of structure element types
- logical structure tree validation against ISO 32005

The dataset contains:

1,905,490 PDFs (9.26%) with interactive forms
419,069 PDFs (2.04%) with digital signatures
888,237 PDFs (4.32%) with optional content (layers)

Distribution of PDF documents by date

To understand the distribution of documents by timeline we analyzed document dates using ModDate when available; otherwise, CreationDate was used.

Because PDF metadata is not always reliable, the analysis was limited to documents dated between 1990 and June 2026. Approximately 974,000 files (about 5%) were excluded because their dates were missing, had invalid syntax, or were outside this range.

Figure 1. Distribution of PDF modification date, 1990–2026

Most publicly available PDFs from June 2026 Common Crawl dataset were created or modified within the last several years.

Distribution of page counts in PDF Files

Figure 2. Number of pages in PDFs

Most PDFs published on the web are relatively short.

Single-page documents represent the largest group (5.8 million files), followed by:

2–3 pages (4.9 million), 4–7 pages (3.4 million), 8–15 pages (2.7 million).

Document frequency decreases steadily as page count increases.

Methodological note. Around 8000 PDFs had malformed page trees resulting in missing page information. They were excluded from the page-count analysis.

Distribution of PDF Files by PDF Version

The reported PDF version was determined using both the document header and the optional /Version entry in the Catalog, as defined in PDF 2.0 ( ISO 32000-2).

Only valid PDF versions were included. During processing, 81 documents with invalid version numbers (for example, 1.8, 1.9, 2.3, 7.0, 112.0, and 990.0) were excluded.

PDF 1.7 remains the dominant version with more than 6 million documents, followed by: PDF 1.4, PDF 1.5, PDF 1.6. Together these four versions account for the majority of PDFs on today's web.

Figure 3. PDF files by by header+catalog version (1.0-2.0)

Total Number of Annotations by Type

Annotations are one of the most widely used interactive features of the PDF format. Across the 20.6 million PDF documents analyzed, we identified hundreds of millions of annotations of different types.

The three most common annotation types are:

Link — 265.3 million
Widget — 20.4 million
Square — 6.4 million

Other frequently used annotation types include Popup, FreeText, Stamp, Ink, Highlight, Watermark, and Text.

Figure 4. Top 20 Annotation types total count

Figure 4 presents the total number of annotations of each type across all analyzed PDF documents. The Figure is displayed on a logarithmic scale, allowing less frequent annotation types to remain visible and enabling meaningful comparison across the full distribution.

Figure 5. Top 20 Annotation Types by Document Count

Figure 5 shows Top 20 Annotation Types by the number of documents in which they appear. The vertical axis is plotted on a logarithmic scale.

Besides the annotation types defined by the PDF specification (such as Link, Text, Highlight, or Stamp), the dataset contains dozens of proprietary subtypes created by specific PDF applications and workflows, such as BatesN, InstaSign, MultiSig, SILANIS_SIGNATURE, GoldGrid:AddSeal, TrapNet, and numerous specific annotations generated by products such as GdPicture, BJCA, FICL, and others.

Figure 6. Full list of annotation types by document count

PDF Encryption and Permission Flags

Figure 7. Percentage of Permission Flags

Only 513,342 documents (2.5%) were encrypted with an empty open password. The page-count analysis excludes malformed PDFs with missing page information and 80,062 password-protected PDFs with unknown passwords. For encrypted documents with empty open passwords we analyzed the permission flags stored in the PDF encryption dictionary.

The majority of encrypted PDFs permit normal document use.

Printing — 91.8%
High-resolution printing — 84.1%
Accessibility text extraction — 83.9%

The high percentage of documents allowing accessibility extraction is encouraging because the PDF specification defines this permission independently of general content copying, allowing assistive technologies to access document text even when copying is prohibited.

However, approximately 16% of encrypted PDFs, or 0.4% of the total analyzed document count, disable accessibility extraction (this permission flag was deprecated in PDF 2.0), potentially creating unnecessary barriers for users of screen readers and other assistive technologies.

Overall, encrypted PDFs on the public web are primarily configured to prevent document modification rather than document access.

Implications of analysis

This first part of the June 2026 Common Crawl PDFs analysis reveals several long-term characteristics of PDF usage on the public web:

Most PDFs remain relatively small, short documents.
Encryption is uncommon and generally does not prevent document access.
Accessibility text extraction is enabled in most encrypted documents, although a significant minority still disables it.
PDF 1.7 continues to dominate document production.
Proprietary extensions remain common, particularly in annotation workflows.
Link annotations dominate all other annotation types combined.

Conclusion

This first part of the report provides an initial statistical overview of more than 20.5 million PDF documents collected from the June 2026 Common Crawl dataset.

The findings establish a baseline for understanding how PDFs are created, distributed, and protected on today's web.

Stay tuned. In the next parts we analyse the evolution of a median size of PDF documents for the past 20 years, top producers of PDFs, and Tagged PDF trends.

Structure Tree in PDF4WCAG Accessibility Checker

Julia — Fri, 24 Jul 2026 05:38:50 +0000

What is a Structure Tree?

The Structure Tree represents the logical structure of a tagged PDF document. It consists of structure elements such as headings, paragraphs, lists, tables, and figures, organized in a hierarchical tree that is interpreted by assistive technologies. It also defines the reading order of the document content.

Accessibility validation is performed against these logical structure elements rather than the document's visual appearance, making the Structure Tree an essential component of PDF accessibility analysis.

The Structure Tree defines how assistive technologies interpret and navigate a tagged PDF. In PDF4WCAG Accessibility Checker, users can inspect this hierarchy to verify that headings, paragraphs, lists, and other structure elements are organized correctly and follow a logical reading order.

Accessibility Checker and the Structure Tree

Accessibility checkers can detect an empty paragraph (<P>), but without a Structure Tree it is often impossible to determine which paragraph caused the error. Unlike many visual accessibility issues, an empty structure element usually has no visible representation on the page and therefore cannot be highlighted in the document view. As a result, users are often left searching through the document to locate the offending element.

PDF4WCAG Accessibility Checker 1.10 addresses this problem by introducing an interactive Structure Tree. When a validation error is selected, the corresponding structural element is highlighted in the Structure Tree panel, allowing users to quickly locate the issue within the document hierarchy. An empty paragraph is just one example; the same approach can be used to investigate other structural accessibility problems.

So, when an empty paragraph is detected, users can navigate directly to the corresponding <P> structure element in the tree. This makes it immediately clear where the error occurs and allows users to inspect the element's parent and child nodes, understand its context within the document hierarchy, and resolve the issue more efficiently.

Structure Tree and Roadmap Navigation

PDF4WCAG Accessibility Checker 1.10 also enhances navigation through both the Structure Tree and the Roadmap.

The Structure Tree provides a hierarchical view of the document's logical organization, while the Roadmap presents the logical reading sequence of the document. Together, these complementary views help users understand both the document hierarchy and its reading order, making it easier to investigate and remediate accessibility issues in complex PDF documents.

Conclusion

The Structure Tree is one of the most valuable tools for PDF accessibility remediation. While validation reports identify what is wrong, the Structure Tree shows where the problem exists within the document's logical structure.

By combining synchronized navigation between the validation results, Structure Tree, Roadmap, and document view, PDF4WCAG Accessibility Checker 1.10 enables accessibility specialists to locate and understand structural issues such as empty paragraphs much more quickly than with traditional validation reports alone. For large and complex tagged PDFs, this significantly reduces remediation time and improves the efficiency and accuracy of accessibility corrections.

The Fonts Panel in PDF4WCAG: supporting PDF accessibility and compliance

Julia — Thu, 16 Jul 2026 13:07:31 +0000

When it comes to PDF accessibility, fonts are far more than a design choice. They are an important technical component that affects how text is represented and interpreted by assistive technologies. One of the key additions in PDF4WCAG Accessibility Checker 1.10 is the new Fonts inspection panel, which provides a detailed analysis of embedded fonts, font types and subsets, and encoding information.

For textual content, PDF/UA and Well-Tagged PDF (WTPDF) require text to be represented in a way that supports reliable Unicode extraction and interpretation by assistive technologies.

Proper font implementation helps ensure:

Reliable text extraction
Searchable and selectable text
Accurate Unicode mapping
Reliable interpretation by assistive technologies

If font encoding or accurate Unicode character mapping is incorrect, text may appear correctly on screen while being interpreted incorrectly by assistive technologies or accessibility validation tools.

What the Fonts panel shows

The new Fonts panel in PDF4WCAG 1.10 provides detailed technical information about every font used in the document, including:

Embedded fonts – displays information about fonts embedded in the document
Font type and subset information – displays the font type and whether a font is embedded as a subset or in full
Encoding information – provides information about font encoding to assist in diagnosing Unicode mapping issues

Users can immediately inspect all font resources from a single location. This makes troubleshooting much faster, especially in complex documents containing multiple embedded fonts.

Screen readers rely primarily on correctly encoded text, Unicode mappings, and the tagged PDF structure. Incorrect font encoding or missing ToUnicode mappings can prevent assistive technologies from interpreting text correctly, even when the document appears visually correct. This results in unreadable or skipped content for users with visual disabilities.

The new Fonts panel in PDF4WCAG Accessibility Checker 1.10 gives users direct access to essential font information that previously required specialized PDF inspection tools. By exposing embedded fonts, font types, subset status, and encoding information, it helps accessibility professionals diagnose problems more quickly and improve the technical quality of accessible PDF documents.

Combined with PDF4WCAG's validation engine, powered by the veraPDF architecture, the Fonts panel makes version 1.10 a more comprehensive accessibility validation solution.

Together with the new Metadata and Annotations panels introduced in version 1.10 PDF4WCAG, the Fonts panel provides deeper insight into the technical structure of PDF documents and supports more efficient accessibility analysis.

Contact us:

email: info@pdf4wcag.com

website: https://pdf4wcag.com/

New Annotations Panel in PDF4WCAG

Julia — Thu, 09 Jul 2026 13:00:29 +0000

Annotations are a general mechanism for adding an interactive layer to PDF documents. They include elements such as links, comments, interactive form fields, multimedia, and more. Like all other content, annotations may or may not be accessible. PDF4WCAG checks also cover a number of PDF/UA and WCAG requirements on annotations.

PDF4WCAG Accessibility Checker 1.10 introduces a dedicated Annotations panel that gives users deeper insight into interactive elements critical for accessibility compliance.

The panel inspects all types of PDF annotations relevant to usability evaluation, including:

Comments – user notes and markup
Hyperlinks – navigation and reference links
Form controls – interactive form fields
Other interactive elements – additional dynamic content

The importance of annotation inspection

The Annotations panel provides visibility into the most common accessibility failures related to PDF annotations, including:

Untagged links: users can identify untagged links annotations, which lead to accessibility issues: screen readers treat it as plain text or ignore it entirely.
Missing form labels: users can identify forms with missing labels.
Incorrect inclusion of annotations into the structure tree: users can identify annotations whose parent tags are missing or not in the correct position within the document structure.
Alt text: users can quickly see which annotations have missing or empty alt text.
Forbidden annotation types: users can identify annotation types that are not allowed in the accessible PDF documents.

The new annotations panel helps users quickly identify these issues, supporting compliance with WCAG and PDF/UA requirements.

Persistent preferences

Configuration settings are now persisted between sessions, meaning any custom filtering or view states the user applies to his annotation checks will be remembered the next time the user opens the tool.

email: info@pdf4wcag.com

website: https://pdf4wcag.com/

accessibility #pdf #duallab #dev

OpenDataLoader PDF: one tool and so many options!

Julia — Tue, 30 Jun 2026 07:05:53 +0000

TL;DR: OpenDataLoader PDF is the first open-source tool to auto-tag untagged PDFs into screen-reader-ready Tagged PDFs and the most performant open-source PDF parser for RAG pipelines. But it offers many options because not all PDFs are the same. The heuristic engine processes 60+ pages per second on CPU with 0.91 reading order accuracy; hybrid AI mode boosts accuracy to 0.934 for complex documents. Outputs include JSON with bounding boxes for RAG pipelines or Markdown for human readability. Auto-tagging is free (Apache 2.0); full PDF/UA-1 & PDF/UA-2 export is an enterprise add-on. You choose what fits your documents, compliance needs, and infrastructure.

Core technical options & their meanings
OpenDataLoader PDF gives many choices not to complicate things, but because different use cases and different document types need different approaches. Here's what each option does and why it matters.

Output Format: JSON Markdown HTML Annotated PDF Text

When you run OpenDataLoader, you choose between these output formats.

JSON gives structured, machine-readable data. Every element heading, paragraph, table, list, caption is tagged with a semantic type and a bounding box. Users get exact coordinates for every piece of content. This is the foundation for RAG pipelines, because users can map extracted text back to its exact location on the page.

Markdown offers human-readable text. It's cleaner, simpler, and works well when you just need to read or preview the extracted content.

Advice: Choose JSON when you need precision and structure. Choose Markdown when you need readability.

HTML output transforms your PDF content into a styled, web-ready document. The structure is preserved headings, paragraphs, lists, and tables are rendered with appropriate HTML tags and inline styling.
Annotated PDF output generates a visual overlay on the original document. Every detected element: heading, paragraph, table, list, image is highlighted with a colored bounding box and labeled with its semantic type.

Annotated PDF gives confidence to the users visually, instantly, and without reading a single line of raw JSON.

Text output format strips away everything except the raw text content. No bounding boxes. No semantic types. No formatting. Just the extracted text in the correct reading order.
Comparison of output formats

Layout Analysis: The XY-Cut++ Algorithm

Reading order is one of the hardest problems in PDF extraction. A page may look perfect to a human, but a machine, for example, can easily confuse multi-column page layout with a table or mix up footnotes with body text.

OpenDataLoader solves this with the XY-Cut++ algorithm. It analyzes the page geometry, finds the gaps between columns and blocks, recursively splits the page until every element is in the right order. The result is a logical reading order that mimics how a human would read the page.

This matters because incorrect reading order breaks information retrieval. If the RAG pipeline gets the order wrong, the answers it generates will be wrong too.

In OpenDataLoader this algorithm is enabled by default, and there is an option to disable it.

Processing engine: Heuristic vs. Hybrid

OpenDataLoader's default engine is heuristic, a fast, deterministic, rule-based system that runs entirely on CPU. It processes 60+ pages per second, requires no GPU, and is 100% local. No data ever leaves your machine.

The heuristic engine is ideal for most text-based PDFs. It's private, fast, and predictable.

For complex documents: scanned pages, borderless tables, mathematical formulas, charts OpenDataLoader offers a hybrid AI mode. This routes difficult pages to a local AI backend that handles what the heuristic engine cannot.

The result: table accuracy jumps from 0.49 to 0.93, and reading order accuracy improves from 0.91 to 0.934.

Users choose the engine based on their documents and their performance needs. Also the choices are designed to balance speed (CPU-only, 60+ pages/sec), privacy (100% local), and accuracy (bounding boxes, correct reading order). You select the output and rely on the engine's built-in intelligence for layout and structure, making it a powerful tool for high-throughput, local RAG pipelines.

Two algorithms for table detection: border and cluster

In the process of table extraction in heuristic mode, OpenDataLoader uses two different methods. By default, only the 'border' algorithm is used, which focuses only on table borders. Users can also enable a second algorithm, 'cluster', which divides content into clusters to identify tables (including tables without borders).

Noise filtering in OpenDataLoader

PDFs are full of small text, invisible text, hidden layers, and text outside the page. If users pass all of this to their LLM, they pollute the context with irrelevant information.

OpenDataLoader automatically filters out small text, invisible text, hidden layers, and text outside the page. Only the main body content is extracted and passed to the user’s pipeline. Cleaner input means better outputs.

Filters are also customizable. By default, they're all enabled, removing all content: small text, invisible text, hidden layers, and text outside the page. However, the user can disable these filters in any combination.

Tagged PDF Support: using native structure

When a PDF is "Tagged" it already contains native structural information: headings, paragraphs, lists, reading order. This is often the case with accessible PDFs that comply with PDF/UA or WCAG standards.

ODL can use the existing document structure instead of re-analyzing the layout. This is faster and more accurate, as it relies on the document's existing tags. We recommend using this option only if the PDF is properly tagged.

OpenDataLoader is one tool. Multiple workflows. You decide.

hancom #opendataloader #pdf

Website: https://opendataloader.org/

GitHub: https://github.com/opendataloader-project/opendataloader-pdf

Privacy Policy differences between the Web and Desktop versions of PDF4WCAG

Julia — Tue, 23 Jun 2026 11:41:19 +0000

TL;DR: This article explains the Privacy Policy of PDF4WCAG. How PDF4WCAG collects, uses, and protects information when a user performs the validation on the website or works with the Desktop version.

Organizations that process PDF documents often face strict requirements for data privacy, confidentiality, and regulatory compliance. To meet different operational needs, PDF4WCAG is available in both Web and Desktop versions giving users the opportunity to choose the deployment that best fits security and workflow requirements.

Privacy is a major concern

PDF accessibility validation involves sensitive content, including corporate reports, legal documents, financial statements, educational materials and government publications. Before selecting PDF accessibility checker, organizations should understand where their documents are processed and what information may be transmitted outside their environment.

PDF4WCAG Web version and privacy policy

The Web version of PDF4WCAG is designed for convenience and accessibility. Users can access the service through a web browser without installing any software. Users access Web versions instantly, regardless of their operating system, making onboarding fast and effortless. Automatic updates mean there is no need to manage versions or worry about outdated functionality.

Document processing in the Web version

Files are saved in the browser and then sent to the PDF4WCAG server, where they are deleted immediately after the end of the session. PDF4WCAG doesn’t send files anywhere else. PDF4WCAG uses files just for analysis in case of problems when a user requests.

Web version integrates with veraPDF validation engine. PDF4WCAG doesn’t store its own cookies in the browser. However, it does utilize Google Analytics and collects cookies required by the Google Agent itself. PDF4WCAG also stores basic application settings in the browser (language, selected profile, document zoom, and whether to open the right-hand panel by default).

Use cases of web version

Individual accessibility specialists.
Small and medium-sized organizations.
Small PDF remediation projects.
Remote teams requiring browser-based access.

PDF4WCAG Desktop version and privacy policy

PDF4WCAG provides Desktop version for all major platforms, offering an identical user experience across operating systems (Windows, Linux, macOS). PDF4WCAG Desktop transfers the functionality of the web-based PDF4WCAG Accessibility Checker into a local environment keeping the same visual experience. It represents a desktop wrapper for the web application, enabling users to perform PDF accessibility validation directly on their computers without relying on an internet connection.

Document processing in Desktop version

Desktop version of PDF4WCAG operates offline. It does not send or collect any data to the Internet or outside. As Web version, the desktop version also integrates with veraPDF validation engine, providing the same error previews, compliance reports, and interactive issue visualization as the online tool.

This approach reduces exposure to third-party infrastructure and supports environments with strict confidentiality requirements.

Use cases of Desktop version

Government agencies.
Financial institutions.
Healthcare organizations.
Legal firms.
Enterprises handling confidential or regulated information.

Comparing the two versions

🌐 Web Version

Installation required: No
Browser access: Yes
Document storage location: PDF4WCAG server
Local processing: No
Sensitive docs: Depends on policies
Files auto-delete after session: Yes

💻 Desktop Version

Installation required: Yes
Browser access: No
Document storage location: Local directory
Local processing: Yes
Sensitive docs: Highly suitable
Files auto-delete after session: Yes

Conclusion

Both PDF4WCAG Web and Desktop editions deliver powerful PDF accessibility capabilities. The key difference lies in where document processing takes place. Organizations handling confidential, proprietary, or regulated information may prefer the Desktop version for its local-processing architecture, while users seeking flexibility and ease of deployment may find the Web version the more practical choice.

Understanding these privacy distinctions helps organizations select the deployment model that best aligns with their security, compliance, and operational requirements.

Contact us:

email: info@pdf4wcag.com
website: https://pdf4wcag.com/

How tags are saved in the initial PDF. OpenDataLoader experience

Julia — Mon, 15 Jun 2026 07:15:03 +0000

TL;DR: OpenDataLoader’s auto-tagging engine analyzes the document’s layout, detecting headings by visual text properties, identifying tables by grid patterns, recognizing lists by bullet positions and then writes this structural information directly into the PDF’s internal structure tree.

PDF accessibility begins with mapping document content (headings, paragraphs, tables, lists) into a logical structure tree that assistive technologies can navigate. Manual tagging is slow, error-prone, and impractical for large document volumes.

⁉️ How OpenDataLoader Implements Tag Writing
OpenDataLoader is the first open-source tool which adds tags directly into the initial PDF file without altering the visual appearance of the document. The AI analyzes document structure, distinguishes components such as titles, tables, lists, and images, and inserts the corresponding tags into the source PDF.

Key characteristics of OpenDataLoader’s approach:

No proprietary SDK dependency: most existing tools rely on commercial SDKs for the tag-writing step;
#OpenDataLoader does it all under Apache 2.0 license.
On-premise processing : sensitive documents never leave your network
No page caps or watermarks unlimited use without document quantity restrictions

OpenDataLoader’s auto-tagging was built in collaboration with the
Dual Lab (Member of PDF Association, supports veraPDF, developers of
PDF4WCAG Accessibility checker.

OpenDataLoader’s auto-tagging preserves visual integrity by design. The technology adds semantic structure without touching the presentation layer, follows industry specifications validated by PDF accessibility experts, and has been built specifically to solve the accessibility problem without creating new ones.

GitHub:
https://github.com/opendataloader-project/opendataloader-pdf