Mariano Gobea Alcoba

Posted on Mar 27 • Originally published at mgatc.com

HandyMKV for MakeMKV and HandBrake Automation!

#automation #videoprocessing #makemkv #handbrake

The proliferation of digital media consumption has significantly shifted paradigms from physical media ownership to digital libraries. Despite this trend, a substantial installed base of physical media, particularly optical discs such as DVDs and Blu-rays, persists. Users often seek to migrate their personal collections to digital formats for convenience, archival purposes, space efficiency, and playback compatibility across a myriad of devices. This migration process, however, presents several technical challenges, primarily stemming from the manual, multi-stage nature of converting proprietary disc formats into universally accessible digital files.

The initial stage involves extracting content from encrypted optical discs. This typically yields large, uncompressed, or minimally compressed files that are direct representations of the disc's primary content. Subsequent stages require re-encoding these large files into more compact and device-friendly formats, often involving complex parameter adjustments to balance file size, visual fidelity, and processing time. Manually executing these steps for an entire collection is a labor-intensive and repetitive task, prone to inconsistencies and errors. This context establishes a clear requirement for automation, seeking to streamline the workflow from disc insertion to a fully encoded digital file, thereby reducing user intervention and enhancing efficiency.

Core Utilities: MakeMKV and HandBrake

The digital media community has converged on two primary open-source and cross-platform tools for facilitating this physical-to-digital migration: MakeMKV and HandBrake. Each serves a distinct, critical role in the overall workflow.

MakeMKV

MakeMKV is an application designed to convert proprietary optical disc formats, such as DVD and Blu-ray, into the Matroska (MKV) container format. Its primary function is to "remux" the audio and video streams directly from the disc, bypassing re-encoding. This process is inherently lossless in terms of media data, preserving the original quality of the audio and video tracks without alteration. MakeMKV is notable for its ability to decrypt AACS and BD+ protections commonly found on Blu-ray discs, a capability that positions it as an indispensable first step in the archival process.

The output of MakeMKV is typically a large MKV file, often mirroring the size of the original disc content. For a standard Blu-ray, this can range from 20GB to 50GB or more, depending on the disc's content and duration. While ideal for archival purity, these large files are often impractical for routine playback on constrained devices or for storage in large quantities due to their significant storage footprint.

HandBrake

Following the lossless extraction by MakeMKV, HandBrake enters the workflow as a powerful, multi-platform video transcoder. Its purpose is to convert video from virtually any format into a selection of modern, widely supported codecs (e.g., H.264, H.265, VP9, AV1) within various container formats (e.g., MP4, MKV, WebM). HandBrake is renowned for its extensive configurability, allowing users fine-grained control over encoding parameters such as resolution, bitrate, frame rate, audio tracks, subtitle tracks, deinterlacing, denoising, and more.

The primary benefit of HandBrake is its ability to significantly reduce file sizes while maintaining a user-defined level of visual and auditory quality. This compression is crucial for managing large digital libraries, optimizing files for streaming, and ensuring compatibility across a broad spectrum of playback devices. The complexity, however, arises from the sheer number of available settings and the iterative nature of finding an optimal balance between quality and file size for different types of content.

The Automation Gap and HandyMKV

While MakeMKV and HandBrake are individually powerful, their sequential execution and the manual intervention required at each stage present a significant workflow bottleneck. A typical manual process involves:

Inserting a disc.
Launching MakeMKV, scanning the disc, selecting the primary title (and potentially specific audio/subtitle tracks), and initiating the rip.
Waiting for MakeMKV to complete.
Launching HandBrake, loading the newly created MKV file.
Selecting an appropriate preset or configuring custom encoding settings.
Choosing desired audio and subtitle tracks, potentially applying filters.
Setting an output file path and initiating the encode.
Waiting for HandBrake to complete.
Renaming the output file according to a consistent scheme.
Deleting the large intermediate MKV file.
Repeating this for the next disc.

This manual process is repetitive, error-prone, and inefficient, especially for users with extensive physical media collections. The need for an automated solution that orchestrates these steps, applies consistent settings, and manages file system operations becomes apparent. This is precisely the problem domain addressed by HandyMKV.

HandyMKV emerges as a Python-based utility designed to bridge this automation gap. It acts as an orchestrator, integrating MakeMKV and HandBrake into a cohesive, hands-off workflow. Its core objective is to automate the ripping of optical discs and their subsequent encoding into more manageable formats, transforming a multi-hour, multi-step manual process per disc into a largely unattended batch operation. The tool leverages the command-line interfaces (CLIs) of MakeMKV and HandBrake, making it suitable for server environments, scheduled tasks, and integration into broader media management ecosystems.

Architectural Deep Dive

HandyMKV's architecture is centered around a sequential processing pipeline, driven by configuration and external tool interaction. Its primary components include disc detection, title selection logic, MakeMKV invocation and output parsing, HandBrake command generation, and robust file system management.

Core Components

Configuration Management: HandyMKV relies on a configuration file (often in INI or similar format) to define user preferences, paths to executables, default encoding settings, and specific rules for title and track selection. This external configuration allows for flexible customization without modifying the core script.

[Paths]
MakeMKV_Path = /usr/bin/makemkvcon
HandBrake_Path = /usr/bin/HandBrakeCLI
Output_Directory = /mnt/media/movies

[MakeMKV]
Minimum_Title_Length_Minutes = 60
Enable_Rip_Log = True

[HandBrake]
Preset = "General/Fast 1080p30"
Audio_Languages = eng,jpn
Subtitle_Languages = eng
Forced_Subtitle_Behavior = only
Desired_Audio_Tracks = 2

Disc Detection and Scanning: The utility initiates by polling for inserted optical discs. Once a disc is detected, it invokes MakeMKV's information command to retrieve a structured JSON output detailing the disc's contents, including titles, chapters, audio tracks, and subtitle tracks.
Title Selection Logic: This is a critical component for automation. MakeMKV often reports numerous titles, including main features, extras, menus, and sometimes spurious short clips. HandyMKV employs logic, configurable by the user, to identify the primary movie or episode titles. Common heuristics include:
- Minimum title duration.
- Presence of specific audio/subtitle tracks.
- Title with the most chapters.
- Title with the largest file size (pre-encoding).

MakeMKV Invocation and Output Parsing: HandyMKV utilizes the subprocess module in Python to execute makemkvcon. The -r --json info disc:0 command is used to query disc information, and the --minlength parameter can filter out short titles at the MakeMKV level.

import subprocess
import json

def get_makemkv_info(disc_id=0):
    try:
        command = ['makemkvcon', '-r', '--json', 'info', f'disc:{disc_id}']
        process = subprocess.run(command, capture_output=True, text=True, check=True)
        return json.loads(process.stdout)
    except subprocess.CalledProcessError as e:
        print(f"MakeMKV error: {e}")
        print(f"Stderr: {e.stderr}")
        return None
    except json.JSONDecodeError as e:
        print(f"JSON parsing error: {e}")
        print(f"Output: {process.stdout}")
        return None

# Example of parsing titles
# makemkv_data = get_makemkv_info()
# if makemkv_data and 'titles' in makemkv_data:
#     for title in makemkv_data['titles']:
#         print(f"Title ID: {title['id']}, Length: {title['duration']}, Streams: {len(title['streams'])}")

The JSON output from makemkvcon provides a rich data structure that HandyMKV parses to extract relevant information about each title, including its duration, stream details (video, audio, subtitle codecs, languages), and chapter markers. This information is crucial for applying selection criteria and constructing HandBrake commands.

HandBrake Command Generation: After a title is ripped by MakeMKV, HandyMKV dynamically constructs a HandBrakeCLI command. This command incorporates the user's chosen preset, selects specific audio and subtitle tracks based on language preferences and track type (e.g., forced subtitles, descriptive audio), and defines the output file path.

def generate_handbrake_command(input_file, output_file, config, title_info):
    base_command = [
        config['Paths']['HandBrake_Path'],
        '-i', input_file,
        '-o', output_file,
        '--preset', config['HandBrake']['Preset']
    ]

    # Audio track selection logic
    audio_tracks = []
    desired_languages = config['HandBrake']['Audio_Languages'].split(',')
    selected_audio_count = 0
    for stream in title_info['streams']:
        if stream['codec_id'] == 'A_AC3' and stream['langcode'] in desired_languages and selected_audio_count < int(config['HandBrake']['Desired_Audio_Tracks']):
            audio_tracks.append(str(stream['id']))
            selected_audio_count += 1
    if audio_tracks:
        base_command.extend(['-a', ','.join(audio_tracks)])
        base_command.extend(['-E', 'copy:ac3']) # Example: Passthrough AC3

    # Subtitle track selection logic
    subtitle_tracks = []
    desired_sub_languages = config['HandBrake']['Subtitle_Languages'].split(',')
    for stream in title_info['streams']:
        if stream['codec_id'] == 'S_HDMV/PGS' and stream['langcode'] in desired_sub_languages:
            if config['HandBrake']['Forced_Subtitle_Behavior'] == 'only' and stream.get('forced', False):
                subtitle_tracks.append(str(stream['id']))
            elif config['HandBrake']['Forced_Subtitle_Behavior'] == 'all':
                subtitle_tracks.append(str(stream['id']))
    if subtitle_tracks:
        base_command.extend(['-s', ','.join(subtitle_tracks)])
        # Optional: burn-in forced subtitles
        if config['HandBrake']['Forced_Subtitle_Behavior'] == 'only' and subtitle_tracks:
            base_command.extend(['--subtitle-burned', '1']) # Assuming the first selected is forced

    return base_command

File System Management: This component handles input/output paths, temporary files, and consistent naming. After a successful rip and encode, HandyMKV renames the final output file according to a defined convention (e.g., Movie Title (Year).mkv) and optionally deletes the large intermediate MKV file produced by MakeMKV. This systematic approach ensures a clean and organized digital library.

import os
import re

def sanitize_filename(name):
    # Remove invalid characters for filenames
    name = re.sub(r'[\\/:*?"<>|]', '', name)
    # Replace spaces with underscores or hyphens, or just trim
    name = name.strip()
    return name

def generate_output_filename(title_name, year, output_dir, extension=".mp4"):
    sanitized_title = sanitize_filename(title_name)
    if year:
        filename = f"{sanitized_title} ({year}){extension}"
    else:
        filename = f"{sanitized_title}{extension}"
    return os.path.join(output_dir, filename)

# Example usage
# output_path = generate_output_filename("The Great Movie", "2023", config['Paths']['Output_Directory'])
# print(f"Output will be: {output_path}")

Interaction with External Tools

HandyMKV's robustness relies heavily on its ability to reliably interact with MakeMKV and HandBrake. This involves:

Process Execution: Using subprocess.run() for blocking calls to ensure sequential execution. This is critical as HandBrake cannot encode a file before MakeMKV has finished ripping it.
Output Capture and Parsing: Capturing stdout and stderr from both utilities is essential for logging, progress monitoring, and error detection. MakeMKV's JSON output is particularly valuable for programmatic decision-making regarding titles and tracks. HandBrake's verbose output can indicate encoding progress and potential issues.
Error Handling: Implementing try-except blocks around subprocess.run calls to catch CalledProcessError indicates that an external command failed. This allows HandyMKV to gracefully handle situations where a disc is unreadable, MakeMKV encounters decryption issues, or HandBrake fails during encoding, preventing the entire automation pipeline from crashing.

Configuration and Customization

HandyMKV's utility is significantly enhanced by its configurable nature. Users can define various parameters to tailor the automation to their specific needs and preferences. Key areas of configuration include:

Paths to Executables and Directories

MakeMKV_Path: Absolute path to the makemkvcon executable.
HandBrake_Path: Absolute path to the HandBrakeCLI executable.
Output_Directory: The root directory where final encoded files will be stored.
Temporary_Directory: A directory for intermediate MKV files before encoding and deletion.

MakeMKV Specific Settings

Minimum_Title_Length_Minutes: Filters out short titles (e.g., warnings, studio intros) by setting a minimum duration for ripped content. This is a direct mapping to MakeMKV's --minlength parameter.
Enable_Rip_Log: Whether to generate a log file for each MakeMKV rip.

HandBrake Specific Settings

Preset: The name of a HandBrake preset to use (e.g., "Fast 1080p30", "HQ 720p30 Surround"). This simplifies complex HandBrake configurations.
Audio_Languages: A comma-separated list of preferred audio languages (e.g., eng,jpn,fre). HandyMKV will attempt to select tracks in these languages in order of preference.
Desired_Audio_Tracks: The maximum number of audio tracks to include in the final encode, prioritizing specified languages. This prevents excessive audio tracks from being included unnecessarily.
Subtitle_Languages: A comma-separated list of preferred subtitle languages.
Forced_Subtitle_Behavior: Defines how forced subtitles are handled. Options could include only (selects only forced subtitles), all (selects all subtitles in preferred languages), or none.
Burn_In_Forced_Subtitles: A boolean flag to determine if forced subtitles should be hardcoded into the video stream (burned in) or kept as selectable soft subtitles.
Chapter_Markers: Whether to include chapter markers in the output file, inherited from the source.
Remove_Original_MKV: A boolean flag to control whether the large intermediate MKV file is deleted after successful encoding.

These parameters allow a user to set up a "rip-and-encode" profile that can be applied consistently across their entire collection, ensuring uniformity in naming, quality, and included tracks.

Implementation Details (Python Specific)

HandyMKV, being written in Python, leverages several standard library modules for its functionality.

The os and shutil modules are fundamental for file system interactions:

os.path.join() for platform-independent path construction.
os.remove() for deleting temporary files.
shutil.move() for renaming and moving files.
os.listdir() and os.stat() for monitoring directories and file properties.

The subprocess module is the backbone for interacting with external executables. Its run function, particularly with capture_output=True, text=True, and check=True, provides a robust way to execute CLI tools, capture their output, and detect execution failures.

import subprocess
import os

def execute_command(command_list, description="Executing command"):
    print(f"{description}: {' '.join(command_list)}")
    try:
        result = subprocess.run(
            command_list,
            capture_output=True,
            text=True,
            check=True,
            encoding='utf-8'
        )
        if result.stdout:
            print(f"STDOUT:\n{result.stdout}")
        if result.stderr:
            print(f"STDERR:\n{result.stderr}")
        return result.stdout
    except subprocess.CalledProcessError as e:
        print(f"Error during {description}: {e}")
        print(f"Command: {e.cmd}")
        print(f"Return Code: {e.returncode}")
        print(f"STDOUT: {e.stdout}")
        print(f"STDERR: {e.stderr}")
        raise # Re-raise the exception for upstream error handling

For parsing MakeMKV's output, the json module is indispensable. MakeMKV's --json flag provides a machine-readable, structured representation of disc contents, which simplifies the task of programmatically selecting titles and tracks.

import json

def parse_makemkv_json(json_output):
    try:
        data = json.loads(json_output)
        return data
    except json.JSONDecodeError as e:
        print(f"Failed to parse MakeMKV JSON output: {e}")
        return None

Configuration parsing is often handled by the configparser module for INI-style files, or json or yaml for more complex structures, allowing users to define their settings externally.

Robust logging, typically via the logging module, is crucial for tracking the automation process, diagnosing issues, and providing detailed information on disc processing, title selection, and encoding parameters. This includes timestamps, message levels (INFO, WARNING, ERROR), and destination (console, file).

The overall flow involves a main loop that detects discs, then for each disc:

Calls get_makemkv_info.
Filters titles based on Minimum_Title_Length_Minutes and other heuristics.
For each selected title: a. Generates a temporary MakeMKV output path. b. Constructs and executes a makemkvcon mkv command. c. On success, generates a HandBrake output path and constructs a HandBrakeCLI command with selected audio/subtitle tracks and preset. d. Executes the HandBrakeCLI command. e. On success, renames the final file and optionally deletes the intermediate MKV. f. Handles errors at each stage, potentially skipping to the next title or disc.

Challenges and Considerations

Developing an automation tool like HandyMKV necessitates addressing several inherent complexities and potential pitfalls.

Disc Complexity

Optical discs, particularly Blu-rays, can exhibit significant structural complexity.

Multi-angle content: Some discs offer multiple camera angles, which MakeMKV may present as distinct titles or streams. HandyMKV's title selection logic must be sophisticated enough to differentiate main content from alternate angles or extras.
Seamless branching: Often used for director's cuts or regional variations, seamless branching presents a single logical title composed of multiple physical stream segments. MakeMKV handles this transparently, but it impacts accurate duration reporting and title identification.
TV series discs: A single disc might contain multiple episodes. HandyMKV needs to correctly identify each episode as a separate title and process them individually, often requiring more nuanced naming conventions (e.g., Show Name - S01E01 - Episode Title.mp4). This can be challenging if titles are not clearly differentiated by duration or stream count.
Foreign language discs: Discs from different regions may have varying default audio/subtitle tracks. Robust language selection logic is critical to ensure the desired tracks are always chosen.

Performance

Video encoding is computationally intensive. HandBrake, especially with high-quality presets or H.265 encoding, can consume significant CPU resources for extended periods.

Sequential processing: Given the resource demands, HandyMKV typically processes discs and titles sequentially. Parallel ripping and encoding from multiple drives, while technically possible, would require complex resource management and could easily saturate system resources, leading to instability or slowdowns.
Hardware acceleration: Modern HandBrakeCLI versions support hardware-accelerated encoding (e.g., Intel Quick Sync Video, NVIDIA NVENC, AMD VCE/VCN). HandyMKV can integrate this by allowing users to specify HandBrake presets that leverage these capabilities, significantly reducing encoding times. However, this depends on the user's hardware and HandBrakeCLI build.

Error Handling and Resumption

Robust error handling is paramount for an unattended automation system.

Unreadable discs: Physical defects or severe DRM can cause MakeMKV to fail. HandyMKV should log these failures, potentially eject the disc, and proceed to the next, rather than halting the entire process.
Encoding failures: HandBrake might crash due to corrupted intermediate files, insufficient disk space, or unexpected video stream properties. HandyMKV needs to detect these failures, log the details, and clean up partially encoded files.
Partial progress: In a multi-title disc, if one title fails, HandyMKV should ideally complete other successful titles before reporting failures. The system might also consider mechanisms to resume processing a disc from a specific title if an interruption occurs.

Dependency Management

HandyMKV relies on external binaries (makemkvcon and HandBrakeCLI) being installed and accessible in the system's PATH or specified directly in the configuration. The script must validate the presence and executability of these dependencies at startup. Version compatibility can also be an issue if new features are introduced or old ones deprecated in the underlying tools.

Future Enhancements

Potential areas for future development include:

GUI: While a CLI tool is powerful for automation, a graphical user interface could improve accessibility for less technical users.
Metadata tagging: Automatically fetching metadata (movie title, year, cast, synopsis, poster art) from online databases (e.g., TMDB, IMDb) and embedding it into the output files for better media library organization.
Cloud integration: Storing output files directly to cloud storage services.
Advanced content recognition: Using AI/ML models to improve title selection, automatically identify TV series seasons/episodes, and apply optimal encoding settings based on content characteristics.

HandyMKV represents a practical solution to a common and laborious problem. By systematically orchestrating MakeMKV and HandBrake, it significantly reduces the friction associated with digitizing physical media collections, providing a streamlined, configurable, and robust workflow for media enthusiasts.

For professional consulting services related to media automation, data pipeline optimization, and custom software development, please visit https://www.mgatc.com.

Originally published in Spanish at www.mgatc.com/blog/handymkv-makemkv-handbrake-automation/

DEV Community