Why MongoDB Became the Perfect Fit for My Serverpod Project

When building a backend for my Serverpod project, I found myself at a crossroads regarding which database to use. Initially, I started with PostgreSQL, a well-established relational database known for its reliability and robustness. However, as my project evolved, I realized that MongoDB might be a better fit. In this article, I want to share my thought process, the challenges I faced, and why ultimately MongoDB emerged as the database of choice.

Understanding the Nature of My Project

The project I was working on is dynamic in nature, requiring frequent schema changes and the ability to handle unstructured data. It involved supporting a variety of data types, some of which weren’t well-suited to the rigid structure of a relational database. Serverpod, which typically integrates well with PostgreSQL, provided me with a starting point, but I found myself battling with complex migrations and cumbersome schema updates.

I needed a solution that could adapt quickly to changing requirements. Flexibility was key, and this is where MongoDB’s schemaless architecture stood out. Unlike PostgreSQL, where every schema change meant a carefully planned migration, MongoDB allowed me to evolve my data model rapidly without the added overhead of migrations.

The Benefits of Switching to MongoDB

1. Schema Flexibility

One of the main reasons I switched to MongoDB was its ability to store documents without enforcing a strict schema. This flexibility was a game-changer for my project. I could easily add or modify fields without worrying about breaking existing data or dealing with time-consuming migrations. As the project was still in an experimental phase, this kind of flexibility allowed me to iterate faster and adapt the data structure as I learned more about my needs.

Example: Suppose I have a collection called users that stores user information. Initially, I only store basic details like name and email:

{   "name": "John Doe",   "email": "[e-mail]" }

Later, I realized I needed to add a preferences field to store user-specific settings. With MongoDB, I could simply add this field to new documents without modifying the existing ones:

{  
 "name": "John Doe",   "email": "[e-mail]",  

 "preferences":
 {   
  "theme": "dark",  
   "notifications": true 
  }
 }

2.Handling Complex and Nested Data

My project involved a lot of nested data, such as user activity logs and settings, which PostgreSQL could handle, but not as efficiently as I would have liked. With MongoDB, I could store this data in a natural, JSON-like structure without needing to normalize it across multiple tables. This made my data model much easier to work with and greatly simplified my queries.

Example in Serverpod: Suppose I needed to store user activity logs in my Serverpod project. With MongoDB, I could create a collection called activity_logs where each document represents a user's activity, including nested data like timestamps and activity details:

{   "user_id": "12345",   "activities": [     {       "timestamp": "2024-11-12T10:00:00Z",       "action": "login"     },     {       "timestamp": "2024-11-12T11:30:00Z",       "action": "viewed_profile",       "details": {         "profile_id": "67890"       }     }   ] }

In Serverpod, I could easily interact with this collection using MongoDB’s driver, allowing me to query and update user activities efficiently without dealing with complex joins or normalization.

3.Speed and Developer Productivity

When working with PostgreSQL, I often found myself spending extra time writing JOIN queries to gather related data from different tables. This increased the complexity of my code and reduced the readability of my queries. With MongoDB, I could store related data together in the same document, reducing the need for complex queries and improving performance in some scenarios.

Additionally, using MongoDB allowed me to be more productive as a developer. I could focus more on building features rather than worrying about the intricacies of database management. The integration of MongoDB’s native drivers with Serverpod also made it easy to work with data directly from my application code.

4.Scaling Considerations

While PostgreSQL is excellent for handling transactions and relational data, MongoDB shines when it comes to scaling horizontally. My project has the potential to grow rapidly, and I wanted to prepare for that. MongoDB’s ability to shard data across multiple nodes means that it can handle large-scale reads and writes more efficiently than a typical relational database setup. This was a crucial consideration for future-proofing my application.

5.Ecosystem and Tooling

MongoDB comes with a rich set of tools that have made managing my database much more straightforward. The MongoDB Atlas platform, for instance, offers excellent monitoring, automated backups, and seamless scaling options. These features made MongoDB an attractive choice as I could focus on building my project without worrying as much about infrastructure.

Using MongoDB with Flutter and Serverpod

Integrating MongoDB with Flutter and Serverpod is straightforward with the use of MongoDB’s Dart driver. Here’s a simple example to demonstrate how to use MongoDB in a Flutter app with Serverpod as the backend.

Step 1: Setting Up MongoDB in Serverpod

First, add the MongoDB Dart driver to your Serverpod project by including it in your pubspec.yaml:

dependencies:
  mongo_dart: ^0.9.0

Then, initialize a MongoDB connection in your Serverpod server-side code:

import 'package:mongo_dart/mongo_dart.dart';

class MongoDBService {
  late Db db;
  Future<void> connect() async {
    db = await Db.create("mongodb://localhost:27017/my_database");
    await db.open();
  }
  Future<void> close() async {
    await db.close();
  }
  Future<List<Map<String, dynamic>>> getUserActivities(String userId) async {
    var collection = db.collection('activity_logs');
    return await collection.find({'user_id': userId}).toList();
  }
}

In this example, the MongoDBService class is responsible for managing the connection to MongoDB and interacting with the activity_logs collection.

Step 2: Creating API Endpoint in Serverpod

Create an endpoint in Serverpod to fetch user activity logs:

import 'package:serverpod/serverpod.dart';
import '../services/mongodb_service.dart';

class ActivityEndpoint extends Endpoint {
  final MongoDBService _mongoDBService = MongoDBService();
  @override
  void initialize(Server server, String name, Serverpod pod) async {
    super.initialize(server, name, pod);
    await _mongoDBService.connect();
  }
  @override
  void terminate() async {
    await _mongoDBService.close();
    super.terminate();
  }
  Future<List<Map<String, dynamic>>> getUserActivities(Session session, String userId) async {
    return await _mongoDBService.getUserActivities(userId);
  }
}

This endpoint will call the MongoDBService to retrieve the activity logs for a given user.

Step 3: Accessing MongoDB from Flutter

In the Flutter app, make an HTTP request to the Serverpod backend to retrieve user activity logs. You can use the http package for this:

import 'package:http/http.dart' as http;
import 'dart:convert';

Future<List<dynamic>> fetchUserActivities(String userId) async {
  final response = await http.get(Uri.parse('http://localhost:8080/activity/getUserActivities?userId=$userId'));
  if (response.statusCode == 200) {
    return jsonDecode(response.body);
  } else {
    throw Exception('Failed to load user activities');
  }
}

This function will fetch the user activities from the Serverpod backend and return the list of activities.

Challenges of Using MongoDB

Of course, switching to MongoDB wasn’t without challenges. The biggest adjustment was moving away from the structured world of SQL. I had to rethink how I queried my data, shifting from SQL joins to embedding data in documents or creating manual references. There was also a learning curve in understanding MongoDB’s aggregation framework, which, while powerful, required a different mindset.

However, these challenges were outweighed by the benefits I gained in flexibility and scalability. The effort I put into learning MongoDB’s way of handling data paid off in the end, making my project much more adaptable to change.

Final Thoughts

Choosing between MongoDB and PostgreSQL for my Serverpod project came down to the specific needs of my application. While PostgreSQL is a fantastic database, especially for applications with structured, relational data and complex transactions, MongoDB’s flexibility and ease of scaling made it the right choice for me. The ability to handle schema changes effortlessly and the natural way it manages nested data helped me move faster and made my application more resilient to change.

If you’re at a similar crossroads, I recommend assessing the nature of your data and how much flexibility you need. MongoDB is a powerful tool, but it shines best when your data isn’t well-suited to a rigid schema, or you need to iterate quickly without worrying about migrations. In my case, this switch was the key to making my project successful.