DEV Community

Jeroen van der Heijden
Jeroen van der Heijden

Posted on

πŸš€ What if your Data Solution Contains Migration History?

ThingsDB 1.8.0 introduces significant features designed to help teams manage collection structure changes with confidence and clarity. The headline feature is a robust Commit History system, complemented by improvements to structure-only exports.


πŸ’Ύ Feature 1: Version Control for Your Collection Structure

When you work with ThingsDB, you typically have a clear separation between two types of collection operations:

  1. Routine Operations: Queries used by your application at runtime (e.g., adding an item, updating a record). These are the "normal" changes.
  2. Structural Operations: Queries used for setup or migrations (e.g., adding or changing types, updating procedures). These represent major changes to your application's data schema.

The new Commit History feature is specifically designed to version-control these Structural Operations.

Enabling and Using Commit History

When you choose to enable the commit history on a collection using the set_history() function, ThingsDB enforces a strict rule: any structural change must be explicitly committed.

This is why functions that modify a collection structure (like new_type() or set_type()) now require an explicit call to commit() before they can be executed. This ensures every critical change is logged with an explanatory message.

Example: Making a Committed Migration

This example shows the required structure for a migration script when commit history is enabled. The commit() call stages the message first, and the commit is finalized upon successful execution of the entire query.

// 1. Stage the commit message and enable structural changes
commit('Adds an initial user type for authentication');

// 2. Perform the actual structural change
new_type('User');
set_type('User', {
    username: "str",
    password_hash: "str",
});
Enter fullscreen mode Exit fullscreen mode

The resulting commit object permanently stores the commit message, the user who ran the query, and the entire block of code that was executed.

πŸ” Feature 2: Searching and Auditing Commit History

The new history() function is your primary tool for exploring this permanent archive of structural changes. You can search, filter, and audit every critical modification made to your collections.

Key Filtering Options

The history() function accepts several powerful filters:

Option Description
id Retrieves a specific, unique commit object instead of a list.
first/last Limits the search to the X oldest (first) or X newest (last) commits.
before/after Filters commits that occurred prior to or after a specified datetime object.
match Uses a regular expression to search across the commit's user name, message, and code.
has_err Filters based on execution status: true for failed commits, false for successful ones.

Example: Finding Failed Migrations

This code snippet demonstrates how to search for all commits that failed (i.e., resulted in an error) within the last year, allowing you to quickly debug structural issues:

history({
    scope: '//app_data',
    detail: true,
    has_err: true,
    after: datetime().move('years', -1),
});
Enter fullscreen mode Exit fullscreen mode

🧹 Feature 3: Pruning Old History

History is valuable, but it shouldn't be infinite. The del_history() function allows you to prune old, unnecessary records using the same powerful filters:

Example: Removing Old History

This code removes all commits that were created more than six months ago in the //stuff collection, helping to keep your history archive clean:

del_history({
    scope: '//stuff',
    before: datetime().move('months', -6),
});
Enter fullscreen mode Exit fullscreen mode

⚠️ Warning: Disabling commit history via set_history(<scope>, false) will permanently delete all existing historical records for that scope. This action cannot be undone.

🎁 Bonus Feature: Improved Structure Exports

Finally, the export function has been improved. A structure-only export (export({dump: false})) now includes complex enumerators as well, ensuring you get the complete "framework" or schema of a collection without any data, making it easier to manage and deploy database templates.


Conclusion: ThingsDB 1.8.0 brings professional-grade schema versioning to your data solution. Start using Commit History today to ensure every structural change is recorded, audit-able, and traceable!


Documentation links:

Top comments (0)