DEV Community

Cover image for Choosing the right frontend database for a single page application

Choosing the right frontend database for a single page application

Johannes Millan on October 19, 2019

I am currently looking for a better way to save and retrieve and save data for my favorite side project. I did some research and I thought it might...
Collapse
 
radex profile image
Radek Pietruszewski

Hey Johannes,

Radek here - main author and maintainer of WatermelonDB. It's really interesting to hear your thoughts on it because most of your complaints are EXACTLY what I've been working on fixing over the last couple of weeks!

IndexedDB is just stored as one big string

Not anymore! I've completely rewritten LokiJS's IDB adapter to store things in little chunks: github.com/techfort/LokiJS/pull/808 and are working on further performance improvements: github.com/techfort/LokiJS/pull/811 — it's actually not one record per IDB object, because IDB is slow at this. It's more profitable (for mass insert and reading) to store objects in chunks than one-per-record or one-huge-chunk

Some issue with fast consecutive writes (but that might have been me using it wrong)

That's true, there has been some performance regressions, and it was never 100% optimized. This is changing. I've improved fast consecutive writes on web by 6x, and on iOS by a ridiculous 23x. Further 5x on top of the 23x is coming to iOS, and a huge perf upgrade to Android as well.

Strongly focused on react

That's true at the moment, but that's something I hope the open-source community will improve.

WatermelonDB itself is actually 100% React-agnostic at this moment. The part that connects 🍉 to React (withObservables) is by design a separate component. It's what we use, so it's what we documented and published, but in reality, the only thing that's missing to use WatermelonDB in any framework is a small bit of code to connect RxJS Observables to your components. Easy!

Typescript Errors

That one I have no updates on - TypeScript types are entirely community-led, and I encourage you to contribute to that effort!

Collapse
 
johannesjo profile image
Johannes Millan • Edited

Hey Radek,

thanks for your reply! That's really great news! I really like what WatermelonDB hast o offer. I'll definitely give it another go.

it's actually not one record per IDB object, because IDB is slow at this. It's more profitable (for mass insert and reading) to store objects in chunks than one-per-record or one-huge-chunk

I got the same feeling during my experiments. It's a shame that this seems to be the case. Would be cool if IndexedDB would behave more like a real database rather than a fancy version of localstorage.

The changes don't seem to be available in the latest npm version yet. How would you recommend to go about testing them (which of the new pre-releases should I use)?

Collapse
 
radex profile image
Radek Pietruszewski

I got the same feeling during my experiments. It's a shame that this seems to be the case. Would be cool if IndexedDB would behave more like a real database rather than a fancy version of localstorage.

Agreed. It's a shame that WebSQL got ditched -- it was very powerful, and fast, too.

The changes don't seem to be available in the latest npm version yet. How would you recommend to go about testing them (which of the new pre-releases should I use)?

Try v0.15.0-8 -- and if that doesn't work, v0.15.0-7 — you should be able to insert tens of thousands of records per second with the latest changes -- but even more optimizations are coming.

Thread Thread
 
johannesjo profile image
Johannes Millan

Thank you! I'll give it another go!

Collapse
 
dzcpy profile image
dzcpy

I Really hope that it can support encryption on both mobile platforms and desktop. There's a pull request but it's been inactive for so long.

Collapse
 
michielnuyts profile image
Michiel Nuyts

dev.to/michielnuyts/going-offline-...

I've been using Dexie for the past week. Let me know if I can help with something.

Collapse
 
johannesjo profile image
Johannes Millan

Thank you very much! I setup the very basic stuff and it seems to do the job. Using it in a reactive way (auto stream data updates via an Observable) unfortunately seems to be slow for larger datasets (starting at 100 entries):
github.com/johannesjo/super-produc...

So if I were to use Dexie I probably still would have to go with an in memory store for speed (I probably would keep using the redux stuff already implemented). Or do you maybe know if there is a smarter way do this?

Atm. I am checking out gun which offers a lot of possibilities.

Collapse
 
michielnuyts profile image
Michiel Nuyts

Yes I'm using Overmind for managing my state. I'll have a look at the repo this evening.

Thread Thread
 
johannesjo profile image
Johannes Millan

That would be really nice! Let me know, if you have any questions! The code is not too pretty atm. just quickly hacked in and the redux stuff isn't probably set up in the most helpful manner as well... The basic idea is to move from the current model structure of loading everything related to a project into the store, to having the possibility to access all the tasks all the time to allow for mixed lists.

Keep in mind that I haven't yet decided which road to go, but nevertheless it would be super cool to be able to discuss these things with someone! I've to admit that even though it might not be the best fit I'm currently intrigued to learn more about gunjs and graph databases. Totally new area to me :)

Thread Thread
 
michielnuyts profile image
Michiel Nuyts

Are you on twitter or something to chat about it? I'll show you how I'm using Dexie and my in-memory store.

Thread Thread
 
johannesjo profile image
Johannes Millan

I am: @hugoderhungrige

Though there is also a chat function here. Maybe we can use that?

Thread Thread
 
michielnuyts profile image
Michiel Nuyts

Sure we just need to follow each other in order to be able to chat :)

Collapse
 
spirobel profile image
spirobel

how does dexiedb compare to lokijs? dexie has uselivequery dexie.org/docs/dexie-react-hooks/u...
and loki has techfort.github.io/LokiJS/DynamicV... dynamicView. So both have reactivity. Dexie is more closely tied to indexeddb while loki seems to be more tied to nodejs. So maybe dexie is better in the browser while loki is better in the backend. Or maybe loki is better in general, but I dont know because I didnt use it yet. Any thoughts?

Collapse
 
seanmclem profile image
Seanmclem

Posted twice?

Collapse
 
johannesjo profile image
Johannes Millan • Edited

Not on purpose. Just wanted to correct some spelling mistakes. There seems to be a bug with dev.to:

github.com/thepracticaldev/dev.to/...

I archived this post, but it's still showing up. Don't know what else I could do... I could try to unpublish but I'm afraid that this just will create another copy.

Collapse
 
johannesjo profile image
Johannes Millan
Collapse
 
dfahlander profile image
David Fahlander

Hi! Might be late in the game but the with latest Dexie release 3.2 there's reactivity built in with examples for different frameworks.