Merge PDFs Recursively Using Python

Introduction

Merging multiple PDF files into a single document can be a tedious task, especially if the files are spread across multiple directories. With Python, this task becomes seamless and automated. In this tutorial, we’ll create a command-line interface (CLI) tool using PyPDF2 and click to merge all PDF files in a directory (including its subdirectories), while excluding specific directories like .venv and .git.

---

Prerequisites

Before getting started, ensure you have the following:

1. Python: Version 3.7 or above.
2. pip: Python's package manager.

3. Required Libraries:

   • Install PyPDF2 for PDF manipulation:

    pip install PyPDF2
   

• Install click for creating the CLI:
  

   pip install click
  

---

Code Walkthrough

Here’s the complete code for our CLI tool:

import click
from pathlib import Path
from PyPDF2 import PdfMerger
import os

EXCLUDED_DIRS = {".venv", ".git"}

@click.command()
@click.argument("directory", type=click.Path(exists=True, file_okay=False, path_type=Path))
@click.argument("output_file", type=click.Path(dir_okay=False, writable=True, path_type=Path))
def merge_pdfs(directory: Path, output_file: Path):
    """
    Merge all PDF files from DIRECTORY and its subdirectories into OUTPUT_FILE,
    excluding specified directories like .venv and .git.
    """
    # Initialize the PdfMerger
    merger = PdfMerger()

    # Walk through the directory tree, including the base directory
    for root, dirs, files in os.walk(directory):
        # Exclude specific directories
        dirs[:] = [d for d in dirs if d not in EXCLUDED_DIRS]

        # Convert the root to a Path object
        current_dir = Path(root)

        click.echo(f"Processing directory: {current_dir}")

        # Collect PDF files in the current directory
        pdf_files = sorted(current_dir.glob("*.pdf"))

        if not pdf_files:
            click.echo(f"No PDF files found in {current_dir}")
            continue

        # Add PDF files from the current directory
        for pdf in pdf_files:
            click.echo(f"Adding {pdf}...")
            merger.append(str(pdf))

    # Write the merged output file
    output_file.parent.mkdir(parents=True, exist_ok=True)
    merger.write(str(output_file))
    merger.close()

    click.echo(f"All PDFs merged into {output_file}")

if __name__ == "__main__":
    merge_pdfs()

---

How It Works

1. Directory Traversal:

   • The os.walk() function recursively traverses the specified directory.
   • Specific directories (e.g., .venv, .git) are excluded using a filter on dirs.

2. PDF File Collection:

   • The current_dir.glob("*.pdf") collects all PDF files in the current directory.

3. Merging PDFs:

   • PdfMerger from PyPDF2 is used to append all PDFs.
   • The merged output is written to the specified file.

4. CLI Integration:

   • The click library makes it easy to provide directory and output file paths as arguments.

---

Running the Tool

Save the code to a file, e.g., merge_pdfs.py. Run it from the terminal as follows:

python merge_pdfs.py /path/to/directory /path/to/output.pdf

Example

Suppose you have the following directory structure:

/documents
├── file1.pdf
├── subdir1
│   ├── file2.pdf
├── subdir2
│   ├── file3.pdf
├── .git
│   ├── ignored_file.pdf

Run the tool as follows:

python merge_pdfs.py /documents /merged.pdf

This will merge file1.pdf, file2.pdf, and file3.pdf into merged.pdf, skipping .git.

---

Features

1. Recursive Merging:

   • The tool automatically includes PDFs from all subdirectories.

2. Directory Exclusions:

   • Excludes directories like .venv and .git to avoid irrelevant files.

3. Sorted Merging:

   • Ensures PDFs are added in sorted order for consistent results.

4. CLI Simplicity:

   • Provides an intuitive interface for users to specify input and output paths.

---

Notes and Limitations

1. Large Files:

   • Merging a large number of PDFs might consume significant memory. Test with smaller datasets first.

2. PDF Compatibility:

   • Ensure all input PDFs are valid and not corrupted.

3. Custom Exclusions:

   • Modify the EXCLUDED_DIRS set to exclude additional directories as needed.

---

Conclusion

This tutorial demonstrates how to automate the merging of PDFs from a directory structure using Python. The provided CLI tool is flexible and can be adapted for more complex workflows. Try it out and let us know how it works for you!

Happy coding! 🚀