Dynamically Loading Models: A Guide to Model Registry Patterns

Abhik Sarkar — Wed, 26 Feb 2025 17:29:11 +0000

Introduction to Registry Patterns

The Registry Pattern is a design pattern that enables dynamic registration and retrieval of class implementations in a program.It acts as a central database where classes(in our context, model architectures) are registered with a unique key.Later, these classes can be retrieved and instantiated based on configuration files or runtime decisions.This pattern is particularly useful in machine learning and deep learning frameworks, where the ability to experiment with different architectures without altering the core codebase is crucial.

Initial Encounters with Model Loading

Traditionally, loading different model architectures based on a configuration file in PyTorch involved a straightforward but rigid approach.Consider the following example, where we define three variants of the ResNet architecture in a file named resnet.py:

# resnet.py
import torch.nn as nn

class ResNet18(nn.Module):
    def __init__(self):
        super().__init__()
        print("Resnet18")

class ResNet32(nn.Module):
    def __init__(self):
        super().__init__()
        print("Resnet32")

class ResNet50(nn.Module):
    def __init__(self):
        super().__init__()
        print("Resnet50")

With a corresponding configuration in config.yaml:

arch: ResNet18

The Python script to dynamically load the model might look something like this:

import yaml
import importlib

# Load the configuration
with open('config.yaml', 'r') as config_file:
    config = yaml.safe_load(config_file)

# Dynamically import the corresponding ResNet class
resnet_module = importlib.import_module('resnet')
arch_name = f"{config['arch']}"
resnet_class = getattr(resnet_module, arch_name)

This method, while functional, lacks flexibility and scalability.As the number of models grows, this approach becomes increasingly unwieldy.

Better Method Model Registry

Inspired by patterns seen in OpenMMLab projects, adopting a Model Registry approach offers a more elegant and scalable solution.A Model Registry acts as a centralized repository where each model class is registered with a unique identifier.This allows for dynamic model loading based on runtime decisions or configuration files, greatly simplifying the code and enhancing its maintainability.

Here's a simplified example of implementing a Model Registry:


# Model Registry Implementation
class ModelRegistry:
    _registry = {}

    @classmethod
    def register(cls, name, model_class):
        cls._registry[name] = model_class

    @classmethod
    def get_model(cls, name):
        model_class = cls._registry.get(name)
        if model_class:
            return model_class()
        else:
            raise ValueError(f"Model type '{name}' not registered.")

Model classes can then be registered to this registry:

ModelRegistry.register('resnet18', ResNet18)
ModelRegistry.register('resnet32', ResNet32)
ModelRegistry.register('resnet50', ResNet50)

And later retrieved dynamically:

model = ModelRegistry.get_model(config['arch'])

Even Better Method: Decorator - Based Registration

Taking inspiration from the decorators seen in MMObjectDetection, we can further refine the Model Registry pattern by employing Python decorators.This approach simplifies registration by automatically registering each model class upon definition:


# Decorator-based model registration
class ModelRegistry:
    _registry = {}

    @classmethod
    def register(cls, name=None):
        def decorator(model_class):
            nonlocal name
            if name is None:
                name = model_class.__name__.lower()
            cls._registry[name] = model_class
            return model_class
        return decorator

    @classmethod
    def get_model(cls, name):
        model_class = cls._registry.get(name)
        if model_class:
            return model_class()
        else:
            raise ValueError(f"Model type '{name}' not registered.")

By annotating model classes with @ModelRegistry.register, we achieve automatic registration:


@ModelRegistry.register('resnet18')
class ResNet18(nn.Module):
    ...

@ModelRegistry.register('resnet32')
class ResNet32(nn.Module):
    ...

@ModelRegistry.register('resnet50')
class ResNet50(nn.Module):
    ...

Conclusion

The journey from a straightforward implementation to a flexible, decorator - based Model Registry pattern illustrates the power of adopting design patterns from leading projects like MMObjectDetection.This approach not only facilitates easier experimentation with various model architectures but also significantly enhances the code's structure and readability. By leveraging the Registry Pattern, developers can maintain a clean and extendable codebase, enabling rapid iteration and innovation in machine learning projects.

Hopefully this helped you understand codebases of openmmlab Projects better.

Reference / Links

1.MMDet

2.MMOCR

3.Python Patterns

The Magic Behind 4A464946: How It Defines JPEG Files

Abhik Sarkar — Wed, 26 Feb 2025 17:20:25 +0000

The Magic Behind 4A464946: How It Defines JPEG Files

Unraveling the Enigma: My Journey with 4A464946

In the vast universe of digital files, a curious enigma captured my attention: the magic number 4A464946. This sequence of characters might appear ordinary at first glance, but it holds the key to one of the most popular image formats in the world: JPEG. Join me as I unravel the mystery of magic numbers and explore their significance in digital file identification.

The Hex Editor Revelation

My expedition into the world of magic numbers began with a simple tool: a hex editor. As I examined a JPEG file, the hex code 4A464946 revealed itself, a silent sentinel at the start of the file. This magic number is a beacon, signaling to any software that encounters it that the file in question is a JPEG. Accompanying this article, I'll be sharing screenshots from my hex editor, comparing the magic numbers of a JPEG and a ZIP file, highlighting their distinct identities.

The Alchemy of File Magic Numbers

Magic numbers are more than just arbitrary codes; they are the cornerstone of file recognition in computing. Each file type, from images to compressed archives, possesses a unique magic number that software checks to determine how to process it. The number 4A464946 is exclusive to JPEG files, making it an essential part of their digital DNA.

A Compendium of Digital Spells: The Magic Number Table

To illuminate this concept further, I've compiled a table of various file magic numbers. This compendium serves as a window into the diverse world of file formats, each with its unique identifier:

File Format	Magic Number (Hexadecimal)	Description
JPEG	4A 46 49 46	Image file format known for compression
GIF	47 49 46 38	Widely used for animated images
PNG	89 50 4E 47	Popular for lossless image compression
PDF	25 50 44 46	Standard for documents
ZIP	50 4B 03 04	Common archive file format
RAR	52 61 72 21	RAR archive format
MP3	FF FB	Audio file format
WAV	52 49 46 46	Waveform Audio File Format
AVI	52 49 46 46	Audio Video Interleave file
MP4	00 00 00 18 66 74 79 70	Multimedia container format

This table is just a glimpse into the myriad file types that populate our digital landscape, each marked by its own magic number.

Conclusion: Embracing the Magic in the Mundane

As my exploration comes to a close, I've developed a newfound appreciation for these hidden markers in our digital files. The magic number 4A464946, along with others, forms the backbone of file identification, silently ensuring our digital experiences are seamless and error-free. The next time you open a JPEG image, remember the magic number 4A464946 - a modest string of characters with an indispensable role in our digital world.

DEV Community: Abhik Sarkar

Dynamically Loading Models: A Guide to Model Registry Patterns

Introduction to Registry Patterns

Initial Encounters with Model Loading

Better Method Model Registry

Even Better Method: Decorator - Based Registration

Conclusion

Reference / Links

The Magic Behind 4A464946: How It Defines JPEG Files

The Magic Behind 4A464946: How It Defines JPEG Files

Unraveling the Enigma: My Journey with 4A464946

The Hex Editor Revelation

The Alchemy of File Magic Numbers

A Compendium of Digital Spells: The Magic Number Table

Conclusion: Embracing the Magic in the Mundane