What's the Status of MongoDB and Prisma 2?

Note: This post is no longer up to date. The MongoDB connector is undergoing initial user testing as of April 2021.

Prisma is a database toolkit that is amazing for relational databases and can be easily ported between Postgres, MySQL, MariaDB, SQLite, and Aurora. But right now Prisma does not currently support non-relational databases like MongoDB or DynamoDB.

I know many other developers are interested in bridging this gap and I was curious what work had already been done on the problem and how I could help contribute. I found a lengthy issue called MongoDB support for Prisma 2 that was opened on January 6 and had 58 comments.

I read through all the comments and have done my best to summarize the situation for others.

January - March

At the beginning of 2020, Nikolas Burk said that Prisma would be primarily focusing on stabilizing their relational drivers before working on other types of databases like MongoDB. The next month he also mentioned that MongoDB was working on a Rust driver that would provide the foundation for a MongoDB connector.

April - May

Saghm Rossi from MongoDB provided an update on April 23 that the process of rewriting the driver internals to be async was completed and that the beta would be released soon. The beta was released on May 5.

Søren Bramer Schmidt from Prisma responded the next day to say that with the Rust driver being used in the Prisma Query Engine they were now working towards a MongoDB connector.

June

Saghm announced the release of version 1.0.0 of the Rust driver on MongoDB's blog on June 8. Nikolas Burk asked users the next day for more info about what they're building with MongoDB and some features they'd like to see.

Here's a few relevant comments:

ppxb

In my application I used Nest.js and MongoDB with @nestjs/graphql. It's a little bit difficult to organize the codes.

Nicklas Reincke

Am doing something similar to @ppxb. I am using Nest.js as an API gateway (with Apollo Federation) to other microservices also written with Nest.js (with GraphQL endpoints).

The gateway and the services are deployed to Google Cloud Run while the services can communicate over Cloud Pub/Sub topics.

The microservices each should use Prisma 2 and have their own database, preferably MongoDB. But to work with Mongoose is a bit of a headache when it comes to structure the code correctly.

Adam Duro

Having Prisma support Mongo aggregation pipeline in at least some fashion (either as a raw query type function, or with some kind of generic to be able to describe the expected output) as well as have support for geo queries, would be a huge win.

There are many other "ORMs" that claim to have cross SQL/Mongo support often leave these features out, which drives people back to Mongoose, which is a pain to use in this new TypeScript heavy world we now live in.

Bradley Bain

Having access to the aggregations pipeline through Prisma would be great (no need to get super fancy with it, even just a straightforward and barebones way to perform an aggregation with raw mongodb aggregation syntax would be miles ahead of existing solutions).

Also another feature that would be great to have is multi-tenancy/clustering support.

Mongoose didn't originally build with multi-tenancy in mind, so even though they technically support it now, it's not very pretty (having to explicitly switch the connection and then reload in the models after switching, since all mongoose schemas are scoped to a single connection). Something like:

// prisma schema file

datasource mongdob {
  url      = env("DEFAULT_DB_URL") // or CLUSTER_URL?
  provider = "mongodb"
}

// main.js

const client1 = new PrismaClient()
const client2 = new PrismaClient('DIFFERENT_DB_URL')

const [ db1, db2 ] = await Promise.all([ client1.connect(), client2.connect() ])

July - August

The entirety of comments on the issue in July and August were summed up by Alejandro Morales on August 8:

I AM SAD I NEED PRISMA MONGO

A lot of people appeared frustrated, but there were also many who came to Prisma's defense and encouraged commenters to think about maybe contributing themselves instead of just complaining about how lame their amazing free tool was.

Others pointed out that learning Rust is a high barrier to entry for this issue. The issue trails off from there without any further actionable information.

Rusoto

Harry Manchanda suggested Rusoto as a possible solution for the DynamoDB connector issue #1676.

I read this comment at mongoDB issue #1277 and seems like you would be looking for a Rust Driver for connecting to dynamoDB (similar to mongoDB)

⏬ So would like to point to these links below ⏬

https://rusoto.org/index.html
https://crates.io/crates/rusoto_dynamodb

I'll have to do more research into Rusoto, but on first glance it looks like it supports MongoDB equivalent things like DocumentDB since it's an AWS SDK but it doesn't support MongoDB proper.

Questions

1. Does Prisma need anything else from MongoDB to implement a connector or is the MongoDB Rust driver good to go?
2. Is anyone at Prisma currently working on a MongoDB connector or accountable for tracking its progress?
3. If not what can contributors do to help Prisma out?

Conclusion

Having gone through all 58 comments the only thing I came away certain of is that Saghm is a boss. On top of just shipping the Rust driver in the first place he's also created great docs on GitHub and on MongoDB's website. He's also written blog posts to clearly communicate his progress.

Comments

[Nikolas Burk]

Hey Anthony 👋 thank you so much for this great writeup and for summarizing the comments from the issue 🙏 We are very aware that people are eagerly waiting for MongoDB support in Prisma 2! I want to shed some light on the current situation and how we are currently thinking about a MongoDB connector by responding to your questions.

Does Prisma need anything else from MongoDB to implement a connector or is the MongoDB Rust driver good to go?

As far as I'm aware, we don't need anything else from MongoDB on a technical level.

Is anyone at Prisma currently working on a MongoDB connector or accountable for tracking its progress?

We are not actively working on a MongoDB connector at the moment. This is also reflected on our roadmap.

Note that there are a few technical prerequisites for the MongoDB connector, such as the ability to use native database types in the Prisma schema (the good news is that this is under active development and likely to be released within the next 2-3 months). Another feature that we need to get done before we can get serious about a MongoDB connector is Prisma Migrate. Migrations for SQL and NoSQL databases mean very different things. We need to be confident about our solution for SQL databases before we can "afford" expanding to NoSQL ones.

If not what can contributors do to help Prisma out?

Unfortunately I believe there's not much that contributors can do at this point, unless they have a lot of free time and come equipped with some very solid knowledge about Rust, JS/TS and databases altogether.

While Prisma is open source, the software we're building is extremely complex. The onboarding processs for new developers who need to get into our codebase typically taskes several days/weeks until there's a level of familiarity where they can independently contribute. In that sense, it seems practically impossible that somebody who's not part of our engineering orgnization would be able to grasp the complexity of the codebase and would be able to contribute. We are aware that this is not ideal though and one of our internal engineering initiatives for the next months is to start better documenting the internals of Prisma – hopefully this will make it easier for external folks to contribute in the future!

However, I actually think that the complexity of the codebase is not the main problem in this case. The bigger one is that there a lot of unknowns and unresolved "product questions". The Prisma schema as well as the Prisma Client API are currently optimized for relational databases. MongoDB brings a new database paradigm and we'll need to figure out how this can play nicely together with our existing tools.

So, what can you do to help?

Talk to our product team! Share your thoughts about the MongoDB connector with the product team directly or keep commenting in the GitHub issue. Your input will be the foundation for what we're building, so if you're eagerly waiting for the MongoDB connector, th best thing you can do right now is help us figure out what it actually should look like and how it should work.

As I said, we are very aware that people want MongoDB support (so do we 😄), but there are a few roadblocks we need to get out of the way before we can start serious development on it.

[ajcwebdev]

Thanks so much Nikolas for this response. I think that a lot of the frustrations people have around these issues come from feeling that they are in the dark and so these types of run downs can generate a lot of good will and ease some of the tension.

I certainly appreciate the unique challenges that come from trying to occupy the open source and the corporate development worlds simultaneously. As someone who is firmly in the open source world at the moment I can say that Prisma is almost indispensable to multiple large scale open source projects that I see going on right now.

Developers love Prisma and they want to help make it better. The more processes Prisma can put in place to allow developers to make meaningful contributions will lead to more developers wanting to make meaningful contributions.

Unfortunately I believe there's not much that contributors can do at this point, unless they have a lot of free time and come equipped with some very solid knowledge about Rust, JS/TS and databases altogether.

I had the exact same thought, it is a very specific combination of skill sets and unfortunately I lack the Rust chops at this time (ask me again in a year and we'll see). But I do know JS and a decent amount about databases so hopefully some Rustaceans in the community can help me bridge the gap.

I will definitely be looking into the resources you provided. I think having this stuff on Notion but not on GitHub might be causing some of the feeling of a communication gap (says the guy on dev.to). Notion is amazing though don't get me wrong.

I'd love to know if/when the MongoDB connector is at a stage where it's possible to build out basic prototypes and proofs of concept. I think that's the sweet spot of what I can help contribute. One of the things I really enjoyed about the project I built with Redwood and Fauna was that it was something that hadn't been done before, not because it took any crazy CS breakthroughs but because it just took a lot of tinkering and getting familiar with the idiosyncrasies of each of them.

The only way to get these large, oddly-shaped software projects to fit together is by having lots of people throwing lots of code around to see what happens and what corners they run into. And I do have a lot of free time.