DEV Community

Erik Pförtner
Erik Pförtner

Posted on

Aether Datafixers: A Lightweight Data Migration Framework for the JVM

#java #tutorial #opensource #productivity

Have you ever shipped a new version of your application, only to realize your users' saved data is now incompatible? 😱 Or found yourself writing brittle migration scripts that break when the data format changes unexpectedly?

Data schema evolution is one of the most underestimated challenges in software development. Whether you're building a game with save files, a desktop application with user configurations, or a service that stores structured data — at some point, your data format will change.

Today, I'm excited to introduce Aether Datafixers 🔧 — a lightweight, type-safe data migration framework for the JVM that makes evolving your data schemas painless.

// Migrate player data from v1.0.0 to v2.0.0 in one line
TaggedDynamic migrated = fixer.update(oldData, fromVersion, toVersion);
Enter fullscreen mode Exit fullscreen mode

TL;DR: Aether Datafixers lets you define versioned schemas and sequential fixes that automatically migrate your data forward. It's format-agnostic (works with JSON, Jackson, or custom formats) and inspired by Minecraft's DataFixer Upper — but much simpler to use.

🔧 What is Aether Datafixers?

Aether Datafixers is an open-source Java library that implements the forward patching pattern for data migrations. Instead of writing complex bidirectional converters, you define a chain of sequential fixes that transform data from old versions to new ones.

The Core Philosophy 

v1.0.0  →  v1.1.0  →  v2.0.0  →  v2.1.0
            Fix₁       Fix₂       Fix₃
Enter fullscreen mode Exit fullscreen mode

Each version has a Schema that defines the data structure. Each transition has a DataFix that transforms the data. When you need to migrate from v1.0.0 to v2.1.0, the framework automatically chains Fix₁ → Fix₂ → Fix₃.

Why Forward Patching?

  • ✅ Simpler reasoning: Each fix only needs to know about two versions
  • ✅ Composable: Fixes chain automatically, no manual coordination
  • ✅ Maintainable: Adding a new version means adding one new fix
  • ✅ Testable: Each fix can be tested in isolation

🎮 Inspired by Minecraft

If you've ever wondered how Minecraft handles world saves from 2011 that still load in 2024 — the answer is DataFixer Upper (DFU). Aether Datafixers takes the same battle-tested approach but with:

  • Cleaner API: No deep type theory knowledge required
  • Better documentation: 75+ pages of guides and examples
  • Focused scope: Just data migration, no kitchen sink

📦 Module Overview

Module Purpose
aether-datafixers-api Core interfaces — no implementation logic
aether-datafixers-core Default implementations
aether-datafixers-codec GsonOps, JacksonOps for JSON handling
aether-datafixers-examples Complete runnable examples
aether-datafixers-bom Bill of Materials for version management

🚀 Quick Start: Your First Migration in 5 Minutes

Let's build a simple migration that renames a field. We'll take player data and rename playerName to name.

Step 1: Add the Dependency

Maven:

<dependency>
    <groupId>de.splatgames.aether</groupId>
    <artifactId>aether-datafixers-core</artifactId>
    <version>0.1.0</version>
</dependency>
Enter fullscreen mode Exit fullscreen mode

Gradle (Kotlin):

implementation("de.splatgames.aether:aether-datafixers-core:0.1.0")
Enter fullscreen mode Exit fullscreen mode

Use the BOM (aether-datafixers-bom) for coordinated version management across all modules.

Step 2: Define a TypeReference

TypeReferences identify your data types for routing:

public final class TypeReferences {
    public static final TypeReference PLAYER = new TypeReference("player");

    private TypeReferences() {}
}
Enter fullscreen mode Exit fullscreen mode

Step 3: Create Your DataFix 

public class RenamePlayerNameFix extends SchemaDataFix {

    public RenamePlayerNameFix(SchemaRegistry schemas) {
        super(
            "rename_player_name",      // Fix identifier
            new DataVersion(100),       // From version
            new DataVersion(110),       // To version
            schemas
        );
    }

    @Override
    protected TypeRewriteRule makeRule(Schema inputSchema, Schema outputSchema) {
        return Rules.renameField(GsonOps.INSTANCE, "playerName", "name");
    }
}
Enter fullscreen mode Exit fullscreen mode

Step 4: Wire It Together with a Bootstrap 

public class MyBootstrap implements DataFixerBootstrap {

    public static final DataVersion CURRENT_VERSION = new DataVersion(110);

    private SchemaRegistry schemas;

    @Override
    public void registerSchemas(SchemaRegistry schemas) {
        this.schemas = schemas;
        schemas.register(new Schema100());  // v1.0.0
        schemas.register(new Schema110());  // v1.1.0
    }

    @Override
    public void registerFixes(FixRegistrar fixes) {
        fixes.register(TypeReferences.PLAYER, new RenamePlayerNameFix(schemas));
    }
}
Enter fullscreen mode Exit fullscreen mode

Step 5: Use It! 

// Create the DataFixer
AetherDataFixer fixer = new DataFixerRuntimeFactory()
    .create(MyBootstrap.CURRENT_VERSION, new MyBootstrap());

// Your old data
JsonObject oldData = new JsonObject();
oldData.addProperty("playerName", "Steve");
oldData.addProperty("score", 100);

// Wrap in Dynamic
Dynamic<JsonElement> dynamic = new Dynamic<>(GsonOps.INSTANCE, oldData);
TaggedDynamic tagged = new TaggedDynamic(TypeReferences.PLAYER, dynamic);

// Migrate!
TaggedDynamic result = fixer.update(tagged, new DataVersion(100), new DataVersion(110));
Enter fullscreen mode Exit fullscreen mode

Before:

{ "playerName": "Steve", "score": 100 }
Enter fullscreen mode Exit fullscreen mode

After:

{ "name": "Steve", "score": 100 }
Enter fullscreen mode Exit fullscreen mode

That's it! 🎉 Your data is migrated, and any unknown fields (like score) are preserved automatically.

🧩 Core Concepts

Let's dive deeper into the building blocks of Aether Datafixers.

TypeReference: Routing Your Data

A TypeReference is a string-based identifier that routes data to the correct schemas and fixes:

public static final TypeReference PLAYER = new TypeReference("player");
public static final TypeReference WORLD = new TypeReference("world");
public static final TypeReference INVENTORY = new TypeReference("inventory");
Enter fullscreen mode Exit fullscreen mode

When you call fixer.update(), the framework uses the TypeReference to find which fixes apply.

DataVersion: Versioning Strategy

DataVersion is a simple integer-based version identifier. You can use any numbering scheme:

// Sequential
new DataVersion(1);
new DataVersion(2);
new DataVersion(3);

// SemVer-inspired (recommended)
new DataVersion(100);  // v1.0.0
new DataVersion(110);  // v1.1.0
new DataVersion(200);  // v2.0.0
Enter fullscreen mode Exit fullscreen mode

The SemVer-inspired approach (100, 110, 200) leaves room for patch versions and makes version relationships clearer at a glance.

Schema: Defining Data Structure

Each version has a Schema that defines what types exist at that version:

public class Schema100 extends Schema {

    public Schema100() {
        super(100, null);  // No parent (first version)
    }

    @Override
    protected void registerTypes() {
        registerType(new SimpleType<>(
            TypeReferences.PLAYER,
            dynamicPassthroughCodec()
        ));
    }
}
Enter fullscreen mode Exit fullscreen mode

Schemas support inheritance — a child schema only needs to register types that changed:

public class Schema110 extends Schema {

    public Schema110() {
        super(110, new Schema100());  // Inherits from v1.0.0
    }

    @Override
    protected void registerTypes() {
        // Only register changed types
        // Unchanged types are inherited from parent
    }
}
Enter fullscreen mode Exit fullscreen mode

DataFix: The Migration Logic

A DataFix transforms data from one version to the next. Extend SchemaDataFix for access to input/output schemas:

public class MyFix extends SchemaDataFix {

    @Override
    protected TypeRewriteRule makeRule(Schema inputSchema, Schema outputSchema) {
        return Rules.seq(
            Rules.renameField(ops, "oldName", "newName"),
            Rules.transformField(ops, "gameMode", this::convertGameMode),
            Rules.addField(ops, "newField", defaultValue)
        );
    }
}
Enter fullscreen mode Exit fullscreen mode

Dynamic: Format-Agnostic Data Wrapper

The Dynamic<T> class wraps your data and provides format-agnostic operations:

Dynamic<JsonElement> dynamic = new Dynamic<>(GsonOps.INSTANCE, jsonObject);

// Read
String name = dynamic.get("name").asString().orElse("Unknown");
int level = dynamic.get("level").asInt().orElse(1);

// Write (returns new instance — immutable!)
Dynamic<JsonElement> updated = dynamic
    .set("name", dynamic.createString("NewName"))
    .set("level", dynamic.createInt(5))
    .remove("obsoleteField");
Enter fullscreen mode Exit fullscreen mode

All operations return new Dynamic instances. The original is never modified.

DynamicOps: The Format Adapter

DynamicOps<T> is an interface that defines how to work with a specific format:

// For Gson
DynamicOps<JsonElement> gsonOps = GsonOps.INSTANCE;

// For Jackson
DynamicOps<JsonNode> jacksonOps = JacksonOps.INSTANCE;

// Same migration code works with both!
Enter fullscreen mode Exit fullscreen mode

🎮 Complete Example: Game Save Data Migration

Let's build a real-world example: migrating game player data through three versions.

The Scenario

Version ID Structure
v1.0.0 100 Flat structure: playerName, xp, x, y, z, gameMode (int)
v1.1.0 110 Renamed fields, nested position object, gameMode as string
v2.0.0 200 Added health, maxHealth, computed level from experience

Version 1.0.0 Data 

{
  "playerName": "Steve",
  "xp": 2500,
  "x": 100.5,
  "y": 64.0,
  "z": -200.25,
  "gameMode": 0
}
Enter fullscreen mode Exit fullscreen mode

Version 2.0.0 Data (Target) 

{
  "name": "Steve",
  "experience": 2500,
  "level": 5,
  "health": 20.0,
  "maxHealth": 20.0,
  "position": {
    "x": 100.5,
    "y": 64.0,
    "z": -200.25
  },
  "gameMode": "survival"
}
Enter fullscreen mode Exit fullscreen mode

Fix 1: v1.0.0 → v1.1.0

This fix handles field renaming, type conversion, and restructuring:

public final class PlayerV1ToV2Fix extends SchemaDataFix {

    public PlayerV1ToV2Fix(SchemaRegistry schemas) {
        super("player_v100_to_v110", new DataVersion(100), new DataVersion(110), schemas);
    }

    @Override
    protected TypeRewriteRule makeRule(Schema inputSchema, Schema outputSchema) {
        return Rules.seq(
            // Rename fields
            Rules.renameField(GsonOps.INSTANCE, "playerName", "name"),
            Rules.renameField(GsonOps.INSTANCE, "xp", "experience"),

            // Convert gameMode from int to string
            Rules.transformField(GsonOps.INSTANCE, "gameMode", this::gameModeToString),

            // Group x/y/z into position object
            groupPositionFields()
        );
    }

    private Dynamic<?> gameModeToString(Dynamic<?> dynamic) {
        int mode = dynamic.asInt().result().orElse(0);
        String modeString = switch (mode) {
            case 1 -> "creative";
            case 2 -> "adventure";
            case 3 -> "spectator";
            default -> "survival";
        };
        return dynamic.createString(modeString);
    }

    private TypeRewriteRule groupPositionFields() {
        return Rules.dynamicTransform("groupPosition", dynamic -> {
            double x = dynamic.get("x").asDouble().result().orElse(0.0);
            double y = dynamic.get("y").asDouble().result().orElse(0.0);
            double z = dynamic.get("z").asDouble().result().orElse(0.0);

            Dynamic<Object> position = dynamic.emptyMap()
                .set("x", dynamic.createDouble(x))
                .set("y", dynamic.createDouble(y))
                .set("z", dynamic.createDouble(z));

            return dynamic
                .remove("x").remove("y").remove("z")
                .set("position", position);
        });
    }
}
Enter fullscreen mode Exit fullscreen mode

After Fix 1:

{
  "name": "Steve",
  "experience": 2500,
  "position": { "x": 100.5, "y": 64.0, "z": -200.25 },
  "gameMode": "survival"
}
Enter fullscreen mode Exit fullscreen mode

Fix 2: v1.1.0 → v2.0.0

This fix adds new fields and computes derived values:

public final class PlayerV2ToV3Fix extends SchemaDataFix {

    public PlayerV2ToV3Fix(SchemaRegistry schemas) {
        super("player_v110_to_v200", new DataVersion(110), new DataVersion(200), schemas);
    }

    @Override
    protected TypeRewriteRule makeRule(Schema inputSchema, Schema outputSchema) {
        return Rules.seq(
            // Add health fields with defaults
            Rules.addField(GsonOps.INSTANCE, "health",
                new Dynamic<>(GsonOps.INSTANCE, new JsonPrimitive(20.0f))),
            Rules.addField(GsonOps.INSTANCE, "maxHealth",
                new Dynamic<>(GsonOps.INSTANCE, new JsonPrimitive(20.0f))),

            // Compute level from experience
            computeLevel()
        );
    }

    private TypeRewriteRule computeLevel() {
        return Rules.dynamicTransform("computeLevel", dynamic -> {
            int experience = dynamic.get("experience").asInt().result().orElse(0);

            // Level = sqrt(experience / 100), minimum 1
            int level = Math.max(1, (int) Math.sqrt(experience / 100.0));

            return dynamic.set("level", dynamic.createInt(level));
        });
    }
}
Enter fullscreen mode Exit fullscreen mode

After Fix 2 (Final v2.0.0):

{
  "name": "Steve",
  "experience": 2500,
  "level": 5,
  "health": 20.0,
  "maxHealth": 20.0,
  "position": { "x": 100.5, "y": 64.0, "z": -200.25 },
  "gameMode": "survival"
}
Enter fullscreen mode Exit fullscreen mode

The Bootstrap 

public final class GameDataBootstrap implements DataFixerBootstrap {

    public static final DataVersion CURRENT_VERSION = new DataVersion(200);
    private SchemaRegistry schemas;

    @Override
    public void registerSchemas(SchemaRegistry schemas) {
        this.schemas = schemas;
        schemas.register(new Schema100());
        schemas.register(new Schema110());
        schemas.register(new Schema200());
    }

    @Override
    public void registerFixes(FixRegistrar fixes) {
        fixes.register(TypeReferences.PLAYER, new PlayerV1ToV2Fix(schemas));
        fixes.register(TypeReferences.PLAYER, new PlayerV2ToV3Fix(schemas));
    }
}
Enter fullscreen mode Exit fullscreen mode

Running the Migration 

public static void main(String[] args) {
    // Create the fixer
    AetherDataFixer fixer = new DataFixerRuntimeFactory()
        .create(GameDataBootstrap.CURRENT_VERSION, new GameDataBootstrap());

    // Load old v1.0.0 save data
    JsonObject v1Data = loadSaveFile("player.json");
    Dynamic<JsonElement> dynamic = new Dynamic<>(GsonOps.INSTANCE, v1Data);
    TaggedDynamic tagged = new TaggedDynamic(TypeReferences.PLAYER, dynamic);

    // Migrate to current version
    TaggedDynamic result = fixer.update(
        tagged,
        new DataVersion(100),    // From v1.0.0
        fixer.currentVersion()   // To v2.0.0
    );

    // The framework automatically chains: Fix1 → Fix2
    saveMigratedData(result);
}
Enter fullscreen mode Exit fullscreen mode

You can run this example yourself! Clone the repository and execute:

mvn exec:java -pl aether-datafixers-examples

🔄 Format Agnosticism: One Migration, Any Format

One of the most powerful features of Aether Datafixers is format agnosticism. The same migration code works with any serialization format.

Using Gson 

Dynamic<JsonElement> dynamic = new Dynamic<>(GsonOps.INSTANCE, jsonElement);
Enter fullscreen mode Exit fullscreen mode

Using Jackson 

Dynamic<JsonNode> dynamic = new Dynamic<>(JacksonOps.INSTANCE, jsonNode);
Enter fullscreen mode Exit fullscreen mode

Same Operations, Different Formats 

// This code works identically with Gson OR Jackson
String name = dynamic.get("name").asString().orElse("Unknown");
Dynamic<?> updated = dynamic.set("level", dynamic.createInt(5));
Enter fullscreen mode Exit fullscreen mode

Custom Formats

Need to support YAML, TOML, or a binary format? Implement DynamicOps<T>:

public class YamlOps implements DynamicOps<YamlNode> {
    @Override
    public YamlNode createString(String value) { /* ... */ }

    @Override
    public YamlNode createInt(int value) { /* ... */ }

    @Override
    public DataResult<String> getStringValue(YamlNode input) { /* ... */ }

    // ... other operations
}
Enter fullscreen mode Exit fullscreen mode

Once implemented, all your existing migrations work with the new format automatically.

💡 Why Choose Aether Datafixers?

vs. Manual Migrations

Aspect Manual Aether Datafixers
Version chaining Manual coordination Automatic
Format support One format per migration Any format
Type safety Runtime errors Compile-time checks
Unknown fields Often lost Preserved by default
Testing Complex setup Each fix testable in isolation

vs. Minecraft's DFU

Aspect DFU Aether Datafixers
Learning curve Steep Moderate
Documentation Limited Comprehensive (75+ pages)
Dependencies Heavy Minimal
Use case Minecraft-specific General purpose
API clarity Complex type system Clean inheritance

✨ Key Advantages

  1. 🔒 Immutable by design: All Dynamic operations return new instances — thread-safe by default
  2. 🔗 Composable rules: Chain multiple transformations with Rules.seq()
  3. 🛡️ Fail-safe: DataResult provides explicit error handling without exceptions
  4. 🧱 Extensible: Custom DynamicOps, custom rules, custom everything
  5. 🎯 Well-tested: Built on patterns proven in one of the world's most-played games

📚 Getting Started

🔗 Links

Quick Install 

<dependency>
    <groupId>de.splatgames.aether</groupId>
    <artifactId>aether-datafixers-core</artifactId>
    <version>0.1.0</version>
</dependency>
Enter fullscreen mode Exit fullscreen mode

🗺️ What's Next?

  • v0.2.0: Additional codec implementations, extended rewrite rules, performance optimizations
  • v1.0.0: Stable API surface, production-ready release

🎉 Wrapping Up

Data migration doesn't have to be painful. With Aether Datafixers, you get:

  • ✅ Forward patching that scales with your application
  • ✅ Format agnosticism that future-proofs your code
  • ✅ Type safety that catches mistakes at compile time
  • ✅ Battle-tested patterns from one of gaming's most successful franchises

If you're building any application that persists structured data, give Aether Datafixers a try. Your future self (and your users) will thank you. 🚀

Aether Datafixers is MIT licensed and open for contributions. Found a bug? Have a feature request? Open an issue on GitHub!

Star on GitHub

Top comments (0)