Who's going to write us a post on alternatives to mongoose ?

github logo ・1 min read

Hi everyone ! Today I am writing my first Request for Post and I'm really excited because I didn't wrote a post for a while ! Sorry about that ! 😅

The idea came to me yesterday, I was working on a project and I needed to connect the app to a Mongo database. I used mongoose like every time but I also asked myself is there other options that mongoose ?

I thought it's a good opportunity to solicit you with this new tag that we owe to @rhymes . Thank's to him !

I think it may interest more than one person.

Like every time, do not hesitate to correct my English and thank's for reading my post ! 😃

twitter logo DISCUSS (28)
markdown guide
 

Why not just plain mongodb driver for node? For me mongoose is unexplainable phenomenon - people go towards NoSQL databases, but on the other hand they chose strict model, giving up the flexibility NoSQL gives them.

Personally, if I need a model I use SQL database. If I need to interact with flexible data, then I use NoSQL.

 

Not only does people seem to love using a NoSQL db with models, but also create relations to other objects in the database... Something that a RELATIONAL DB might be a tad bit better for! ;)

The node community seems to love mongodb for any possible usecase, and I personally have quite a hard time understanding why!

 

well, I have suspicions that the reason is that most node tutorials use mongodb (and then don't show how to set it up securely... and the mongodb documentation doesn't have a quickstart showing how to securely set up the database...)

I'm sure you are right there, I also think that a whole lot of people who develop using nodejs as primary platform are people who started their development career with JavaScript, reading how-toos on stuff like w3-schoold, learning how to code by ctrl+c ctrl+v.

I myself come from a more c-style background, and when it comes to web languages like PHP (which I use most), I see similar behaviour but with mysql, even in the grade that people still use the deprecated (since years and years ago) mysql_* extension/api. It hurts! ;)

well, great part of the blame goes on the recruiters. There is quite a bit of people who do it the right way, and it's easy to spot them. But mostly they're very inexperienced.

That's why I give very simple test for candidates: write a TODO api (yeah, the thing you can google in a few seconds). The twist is, I give them the sqlite database, instead of mongo, and require them to attempt to write a unit test. then I ask some questions. Then I judge by how they approached the problem, and do they understand what they're doing. I've had candidates who barely managed to write half of it, but were able to fix in an hour when I pointed out single mistake. I've also had a candidate who faced with a simple "I want the results sorted" problem said "I could do it in Angular, but I don't know how to do it in node"...

(I come from a C/C++ background, then was assembler, then java and C#, switched to node for most work, but slowly moving towards PFP recently... there at least I (probably) won't have to work with idiots who think they're senior devs after finishing a "senior"-themed bootcamp)

I like you, I think we have the same mindset when it comes to this kind of stuff, I just hope you are not as misanthropic as I am due to this though! :P

won't have to work with idiots who think they're senior devs after finishing a "senior"-themed bootcamp

Totally killed it! 😁😁😁

 

It's a kinda narrow minded view that you put up here. For us it is most of the time a mixed model.
Some parts are fixed, some are free and to put indexes on fixed parts inside your application helps to speed up things (in code and query)

Also we use the same models for Elasticsearch writes where a semi fixed model is useful to build up fast indeces.

Also using relations to other objects in the database is sometimes useful, because it would produce a massive overhead in writes to update redundant data that is not used that often.

From the basic point of view you are right use relational databases for fixed modals and relations, but it is not practical.

There are also other reasons to use mongoDB not purely because of it is noSQL and SQL, but for performance

But if you want "the best of both worlds" why not use a hybrid db? Both mysql (latest versions) and pgsql have document storage just as well as relational storage.
When it comes to mongodb and you say performance, I would love to see a few case studies where mongodb is better (when it comes to performance) than any major relational db for usages that are not purely a document db usage.

Using a mixed model is not a bad thing, and I see no issue with it, but wouldn't it be a whole lot better in that case to also either use a mixed database or use multiple databases? If you are looking for performance, multiple databases and good clustering is the way to go either way, right?

I personally try to use some type of schema or schema-like structure for my documents too, I don't really see that as a negative thing, but relations in mongodb is in most cases (especially at higher loads) a bad idea.

"Why not use a hybrid"
Largescale apllications have only been possible ps 11 and mysql is still struggeling with a large multi slave setup. Also the scalability is way more easy with mongo. You even have the choice between Eventual Consistency and Immediate Consistency and you can do ACID transactions as well.

"Why not use a mixed model"
Welcome to the real world: Having one Database System, one Caching and one Search Engine Database is hard, adding an additional layer of complexity to this is for many companies an overkill. We don't live in a perfect world. The goal is to get a product running, if i can do it with one database, that fits both needs. Why split it up?

Software based relations are not that bad if it happens on a controlled level. Keeping all subdocuments up to date is not always the best way. Yes the idea is have one dataset represent a complete site. This is on no site a valid argument, the insert / updates can kill you there. Use relations, but think before to use them

Software based relations are not that bad if it happens on a controlled level

We are talking about small datasets now I guess? Cause then, sure, no problem at all!

Welcome to the real world

It sounds like you think that I have no experience from "the real world", I have worked with databases quite a bit, and sometimes with datasets that are quite large. I do know how the "real world" works, and I also know that there is no golden hammer. If you think that mongodb is a one size fits all solution, go ahead, but in "the real world" it isn't.

Welcome to the real world: Having one Database System, one Caching and one Search Engine Database is hard, adding an additional layer of complexity to this is for many companies an overkill. We don't live in a perfect world. The goal is to get a product running, if i can do it with one database, that fits both needs. Why split it up?

from my experience (real-time systems, primarily game backends) even if you want single database the better performing approach is usually to use postgres keeping fixed data in columns (and using relations on that), and flexible/document-like metadata in a TEXT column in JSON. Keeping such mixed data in mongo had always been a performance hit for me.

Note that I'm not saying about "JSON" column type, or document storage capabilities of the RDBMS - I query data as a relational one, and then JSON.parse the document part of the data, and process directly in code. I do not know how well would it work for other use cases though.

@mfahlandt , I totally agree with @johannestegner . Majority of developers in JavaScript aren't "developers in JavaScript"; they are developer in a specific library or package. Since MongoDB (or its variants like RethinkDB, etc.) are classic copy-cut-code examples, it's easier for them to blatantly use it without giving any second thought.

@mfahlandt , Mongo isn't made as a drop-in replacement for SQL or RDBMS; quite simply, it's a schema-less document store.. The "real world" example you mention, let me ask you, do you use any model libraries? Mongoose? Camo? If you do, why are you structuring unstructured data and giving up the entire philosophy around NoSQL?

Scalability isn't "easier" per-se in Mongo. Try using Aurora or a RDS backed MySQL/Maria instance. Scaling it is a breeze. You are giving solutions to problems which don't exist.

For Example you have a mixed type where you still want to query
{
account_id: {type: String},
content: Schema.Types.Mixed,
textfields: Schema.Types.Mixed,
created_at: {type: Date, default: Date.now()},
updated_at: {type: Date, default: Date.now()},
deleted: {type: Boolean, default: false},
new_assets: [Schema.Types.Mixed],
tags: [String]
}

I want to enforce that fields are set because you have multiple teams working on this input and still you have the possebility for all the query options that you do not have in postgres subschema.

I never stated that Mongo is the one and only solution for everything. I stated that it is often overkill to use each specific database type for the perfect fitting moment. Why? Because you often don't have the developers that can work with both options, or don't want to work with one or annother option. Or you have to comply with company politics.
Sorry but the Real world is not all the newest and most advanced stuff and most companies simply don't have the ressources or the understanding to use each type for each option. Even if you have the people maybe it is just not worth implementing it. These are business and political decission that define things in the real world. If you work in fancy startups or on greenfields you mostly can do what you want. But this stops at the point you have multiple teams and legacy system that need to work.
You are all describing the perfect world and we are not in the perfect world.

Scalability is harder in mysql when you dont want to pay the higher prices for RDS (and they are not worth it, in no case), you even use RDS and loose all of the finetuning options for your database. I mean come on you can't even export users with a dump in RDS PostgreSQL.
I prefer having all inside of Kubernertes Clusters and there scalability is easier with mongo than any RDS System. Why? Because I'm not vendor looked and i laugh at everyone when AWS double it's prices.

Also your are basically in a VPC without any conenction to the outside.

That was a mere remark with no intent to be hostile; regardless.

Again, you are defining a schema, a structure. Something which is probably better for RDBMS. Your problem is related to the way the data is retrieved and not the way the data is stored, indexed and analyzed.

...you don't often have developers...

That's the fault of your recruitment and on-boarding team. MySQL and Mongoose are the two most popular database systems and any good developer would have no problem in picking it up and using it. His/her mistakes can later be fixed in a CR session. Even if they don't know how to use it, one of the developer can mentor them: that's the entire point about writing software; learning new things and making yourself better.

If you are still stuck in that loop of using technologies like they are instead of going with something which has been battle tested then I feel sorry for your systems.

...or you have to comply with company politics...

Pardon me, but wherever I have worked, we never had company politics in the Engineering team. We discussed, experimented, failed, re-tried. (Oh, before you comment that this is a startup, no.)

...not worth implementing...

That's a different scenario in itself.

After all of your talks related to scaling, a quote comes to mind: over-engineering is just as bad as under-engineering. That doesn't, however, serve as an excuse to use Mongo and define relations through virtuals or ref because you "want it to feel like a RDBMS."

I am not defining or describing a "perfect world." I am defining a continuously moving and advancing world. While I agree that the newest and the shiniest technology shouldn't be a backbone of an entire infrastructure, battle-tested and proven tech should be.

If you work at a startup and think that "this is a greenfield and I can experiment with whatever I want", you aren't doing it correctly.

Scalability is harder...

No, it's not if you know what you're doing. The direct refuting argument to what you said is that enterprises (with "politics", as you so portray), have a dedicated DevOps team which takes care of all of this so the issue doesn't even arise in the first place.

...pay higher prices...

What do you reckon RDS costs? It's a managed instance giving you the free time to write code: your job.

...fine-tuning options...

What? Which RDS instance did you use?

...export users...

You can. You need to create a port-to-port SSH tunnel and access the RDS instance like it's on your own network. You can do everything in that instance like you would in your own database instance.

...Kubernetes cluster...

Isn't that a little more complicated? Which goes directly against your "remote teams" and "don't know how to use it", etc. argument?

...Mongo than any RDS system...

Umm. I am at a loss here. You want to scale but are unwilling to use AWS for it and do all of this manually at the expense of your time and effort. What.

...AWS doubles its prices...

Elasticity of demand and supply. It doesn't double its prices; the changes, if you are region-specific, are negligible. Usually 10-15 USD/year. That that too when you are on-demand.


...you're basically in a VPC...

There was no need for hostile remarks. But, I am glad that the VPC doesn't have any connection to the outside. Maybe I am in the subnet for HIPAA/PCI compliant instances. Good thing! Thanks! :D

I think we reached a point in the discussion, that is pointless to continue.

e.g.

You are basically saying: kick all people out of companies, no matter how log they have worked with the company, because they are not able/willing to learn another database. Not the most social part.

Also you seem to never have worked for companies that came from other sectors to web/IT where the old politics also apply to new sectors. And these companies may not have a dedicated DevOps team.

First: Coding is not my Job. Completly relying on a hosted service by one vendor is stupidly dangerous. There are reasons why projects like Istio are comming up to have multicloud solutions. And sorry to say this there the way to go is Kubernetes. I would never use any dedicated vendor product, just because you do not know what will happen.

Maybe it is a different use case in germany, but we are stuck with legacy code, legacy system and lots of lots compliences that even state the servers have to be on german soil for certain things.
But i know this as well from other western europe countries and companies in the us.

We surely have reached that point.


I am not saying that. And I never said that "kick them if they are not willing to learn another database." I am saying that it's a sign of a good developer to learn something new and apply it. If you think that you are going to have one skill set and apply it everywhere without learning anything new, then... I am not sure what to say. Now, before you misconstrue that and think that you need to be on your toes and learn everything to be good.

Also you seem to have never...

I have worked with a lot of companies in a lot of different sectors; food, education, marketing, sales, ops, logistics and I am yet to see this level of dystopia you mention. Maybe it's because the new heads of these companies are doing their jobs right and keeping things where they belong.

Completely replying on a hosted...

Umm. What? You do realize that Amazon.com and a lot of other applications you use on a daily basis (Netflix, for example) are deployed almost exclusively on AWS.

Your argument of "the way to go is..." is a classic example of how when someone is comfortable with one technology, they use that regardless of anything happening for every single project.

As far as Istio is concerned: there is a reason why AWS has faired for such a long time.

I don't understand why your entire argument chain shifted from database technologies to a cloud provider.

Ok let's get back to why it might be useful do define models:

The database needs protections from it's foremost enemy: the developer
If you have some basic layered structure of data you force the developer to think instead of just vomiting data in your database.
If you have at least partial structured data you also have a bullshit filter of what comes into a database.
If you have structured data you can be shure that for example an idex of a field exists. There is a reason why you can set indexes in mongo.

Schema and Data Modeling is part of MongoDB, heck it is even in the documentation: docs.mongodb.com/manual/core/data-...

You could now come again with the argument why not use PG, simple less options querying the unstructured Data within you possible model.
A Mongo Cluster is also working way more reliant as soon as it comes to cross region scale, even if you use a managed service like RDS

You brought up the point it is easier with using DaaS and then brought up some strange views using mono cloud SaaS.
Why not use single Vendor cloud?

How nice that you choose Netflix, roughly one year ago AWS East Us crashed and was down, half in the US Netflix, Github and some other service have not been available for over 6 hours.
Gues what, Netflix moved to Multicloud and uses Google Cloud as well, because of this.

Also cloud platforms, regardless of form factor or delivery method, are not all interoperable with each other or with legacy systems. There is often an assumption that all will play and talk nicely together and push and pull data easily; this is not the case. Therefore, the ability to mix and choose between offerings in a multi-cloud environment

You cannot easily move your infrastructure to annother provider. The question might be why would i? You do not know exactly what is on layer below, there might come changes that you did not antacipate. Or simply a matter of cost.

Additional you have advantages like lower lateny, higher security.

Most of the bigger companies, are or currently moving to multicloud enviroment.

I drop out of this discussion now, because in my opinion the view of this discussion is only in seeking to find flaws.

 

I use both relational and document. The reason I choose document for a particular thing (usually settings) is because it has a nested hierarchy. This kind of structure is very painful to load/save in a relational model, but simple as a document. I use strict modeling either way.

 

hmm.. never thought about it that way, probably because I haven't had to work with such type of data so far. If I use some sort of a nested hierarchy it's usually relational data, and if I work with something that's not relational I often have to deal with dynamically changing model, of which I'm interested only in a small unchanging part (other part are metadata used by the frontend application I have to return, but I don't care about)

I understand. It is very convenient to use JSON as a hash of keys and values without a pre-defined structure. I generally work with typed languages, so regardless of data store my data usually has an expected structure.

Maybe I shouldn't have commented because I don't use Mongo. I put both kinds of data (document and relational) in Postgres. But I just wanted to provide a viewpoint for why someone might want to use structured data in a JSON store. I usually find that configuration settings fit really well into document storage (and doing so instead of chopping this up into relational tables means I don't need an ORM). But relational is very nice to track set memberships and for its built-in indexing capabilities.

Best wishes!

Maybe I shouldn't have commented

At least for me your comment was a valuable insight :)

I generally work with typed languages, so regardless of data store my data usually has an expected structure.

In typed languages I do (obviously) have a model for my data, but in js where (in my case) large part of the model is of no interest to me and changes rapidly, I prefer to just clearly separate data and functionality, and work without model at all. (It does mean I need more validation, but I can put off validation to where I actually need part of the model)

 

As someone who has been using nosql type databases for years (including XML databases) in many projects, I'll say this.

The object stores (nosql dbs) gives us flexibility to store anything, right, but the structure and relationships are built into the code.

The code is the source of truth vs. whereas the database is the source of truth (relationships etc) in a SQL (relational database).

Then, at some point some odd people :D decided to use both at the same time, which is when Object Relational Mappings for SQL happened.
Which I found strange, as I think the best way to speak to a SQL database is SQL.

 

but ORM libraries have another advantages, other than just performing the mapping: they (usually) enure safety (there are still a lot of developers who don't use parametrized queries when going the pure SQL approach), and often they have built-in migration system. You still need to write the migrations, but the system for running them, and rolling back is there for you.

 

If any of you just want a simple NoSQL DB with zero setup that also scales in case you make something really awesome. Then use Firestore from Firebase. It is awesome!

 
 

you can use mongojs if you want you may know that. But yeah I din't see any option better than mongoose for now.

 
Classic DEV Post from Jun 23

What Advice Would You Give Your 20-year-old Self?

If you could go back in time, what advice would you give your 20-year-old self?

yafkari profile image
A 19y Bob and Alice's friend.