Simplifying Data Persistence in Ballerina: A Deep Dive into `bal persist`

Building modern cloud-native applications often requires seamless integration with various data stores. Whether you're working with MySQL, PostgreSQL, Redis, or even Google Sheets, managing data persistence can become complex and time-consuming. Enter bal persist – a powerful feature in the Ballerina programming language that streamlines data persistence across multiple data stores with a unified, type-safe approach.

What is bal persist?

bal persist is a comprehensive data persistence feature in Ballerina that allows developers to store and retrieve data across different data stores using a consistent syntax. Think of it as your universal translator for databases – write your code once, and it works seamlessly whether you're using MySQL, MSSQL, PostgreSQL, Redis, Google Sheets, or even in-memory tables.

The beauty of bal persist lies in its simplicity: you don't need to learn different syntaxes for different databases. Define your data model once, and bal persist handles the rest.

The Three Pillars of bal persist

The bal persist feature is built on three core components that work together to provide a seamless data persistence experience:

1. Data Model

The data model is the single source of truth for your application's data structure. It's defined using Ballerina record types in a dedicated file within the persist directory of your project.

Here's a simple example of defining an Employee entity:

type Employee record {
    readonly int id;
    string name;
    int age;
    float salary;
}

Key features:

• The readonly keyword designates primary key fields
• At least one primary key field is required per entity
• Multiple primary keys are supported for composite keys
• The Ballerina VS Code extension provides validation, code actions, and even Entity Relationship diagram visualization

2. CLI Tool

The bal persist CLI tool is your code generation powerhouse. It automatically generates type-safe client APIs from your data model, along with all necessary configuration files and setup scripts.

Key commands:

• persist init: Creates the persist directory and an initial data model file
• persist add: Sets up the model file and integrates with bal build for automatic code generation
• persist generate: Provides one-time code generation without build integration
• persist pull: Introspects existing databases (MySQL, MSSQL, PostgreSQL) to generate data models
• persist migrate (experimental): Generates SQL scripts to update database schemas when your data model changes

The best part? With bal build integration, your client APIs are automatically regenerated whenever you build your project!

3. Type-Safe Client API

The generated client API provides a strongly-typed, resource-oriented interface for interacting with your data store. It's generated in the generated directory and includes a Ballerina client object with resources for each entity in your model.

Here's how simple CRUD operations look:

// Create a new employee record
EmployeeInsert employee = {id: 1, name: "John", age: 30, salary: 3000.0};
int[]|error employeeId = sClient->/employees.post([employee]);

// Get an employee record by ID
Employee|error employee = sClient->/employees/1;

// Update an employee record
Employee|error updated = sClient->/employees/1.put({salary: 4000.0});

// Delete an employee record
Employee|error deleted = sClient->/employees/1.delete();

// Get all employees as a stream
stream<Employee, error?> employees = sClient->/employees;

Notice how clean and intuitive the API is – it uses Ballerina's resource access syntax with type safety built in.

How It All Works Together

The bal persist workflow is elegantly simple:

1. Define Your Data Model: Create entity records in the persist directory using Ballerina record types
2. Configure Your Data Store: Specify which data store you're using (MySQL, PostgreSQL, Redis, etc.)
3. Generate Client API: Run bal build or use CLI commands to generate the type-safe client
4. Use in Your Application: Import and use the generated client in your business logic

Supported Data Stores

bal persist currently supports:

• Relational Databases: MySQL, MSSQL, PostgreSQL
• In-Memory: In-memory tables for testing and caching
• NoSQL: Redis
• Cloud Services: Google Sheets

The unified API means switching between data stores is as simple as updating your configuration – no code changes required!

Key Benefits

1. Type Safety

Every operation is type-checked at compile time, reducing runtime errors and improving developer productivity.

2. Database Agnostic

Write your code once and switch between data stores without rewriting your data access layer.

3. Developer Experience

With VS Code integration, you get:

• Real-time validation of data models
• Code actions for quick fixes
• Visual ER diagrams
• IntelliSense for generated APIs

4. Automatic Code Generation

No manual boilerplate – the CLI generates everything you need, from client classes to database setup scripts.

5. Migration Support

The experimental persist migrate command helps you evolve your database schema as your application grows.

Real-World Use Cases

Microservices Architecture

Build microservices with different data persistence needs. One service uses PostgreSQL for transactional data, another uses Redis for caching, and a third uses Google Sheets for simple data – all using the same bal persist patterns.

Multi-Tenant Applications

Easily support different data stores for different tenants without code duplication.

Database Migration

Start with an in-memory table for prototyping, then seamlessly migrate to a production database when ready.

Change Data Capture (CDC)

Build CDC services that react to data changes across different data stores with consistent APIs.

Getting Started

Ready to try bal persist? Here's a quick start:

1. Initialize persist in your Ballerina project:

   bal persist add --datastore mysql

1. Define your data model in persist/model.bal:

   type User record {
       readonly int id;
       string username;
       string email;
   }

1. Build your project (this auto-generates the client):

   bal build

1. Use the generated client in your application:

   import ballerina/io;

   public function main() returns error? {
       Client dbClient = check new();

       UserInsert newUser = {id: 1, username: "alice", email: "[e-mail]"};
       int[] userId = check dbClient->/users.post([newUser]);

       io:println("Created user with ID: ", userId);
   }

Best Practices

1. Use the VS Code Extension: Leverage the built-in validation and visualization tools
2. Start Simple: Begin with basic entities and add relationships as needed
3. Leverage Introspection: Use persist pull to reverse-engineer existing databases
4. Plan for Migration: Consider using persist migrate (experimental) to manage schema evolution
5. Test with In-Memory: Use in-memory tables for unit testing before deploying to production databases

The Future of Data Persistence in Ballerina

bal persist represents a significant step forward in how we handle data persistence in cloud-native applications. By providing a unified, type-safe interface across multiple data stores, it eliminates much of the complexity and boilerplate traditionally associated with data access layers.

Whether you're building microservices, REST APIs, or data-intensive applications, bal persist offers a pragmatic approach to data persistence that scales with your needs.

Learn More

• Official bal persist documentation
• Data Model Guide
• CLI Tool Reference
• Client API Documentation
• Supported Data Stores

---

Have you tried bal persist in your projects? Share your experiences and use cases in the comments below!