DEV Community: Flavien Normand

Realtime Database or Cloud Firestore, when and why?

Flavien Normand — Thu, 05 Jan 2023 15:18:12 +0000

When using Firebase, wether it's for the web or mobile, you'll eventually come to this question: Should I use realtime Database (rtdb) or Cloud Firestore to store this?

In this article we're not going to find the answer, we'll find the right questions for you to get the answer.

Realtime Database

Firebase Realtime Database, also known as "rtdb", is a realtime key:value database. It's very performant for small data operations and has good transactions capabilities to integrate with Cloud Functions easily. To simplify to the extreme, you can see rtdb as a big JSON where you can observe any property at any depth.

Pricing

Realtime database's pricing is based on traffic, the cost being $1/GB transferred after the first 10GB (the free tier). Storage cost is also $5/GB after the first GB that's in free tier.

Cloud Firestore

Cloud Firestore is a realtime document database, it scales very well with big amounts of data and documents, allowing stuff like indexes and querying with an easier approach than rtdb. To simplify to the extreme, it's realtime mongodb with less features.

Pricing

Cloud Firestore's pricing is based on the # of operations you'll do on the database. These prices are:

$0.06 per 100K read (50K free)
$0.18 per 100k write (20K free)
$0.02 per 100k delete (20k free)
$0.18 per GB stored (1GB free)

Ok, cool, but which one to pick?

Now that we have elements about these databases, it's easier to pick one, mainly because these are designed to be cheap if properly used for that they're designed for.

For instance, store big documents that you won't update often on realtime database and your pricing will explode.

My general rules are:

Counters, realtime gauges and other things that need to be updated a lot should go on realtime database.
Larger documents should go on Firestore.
If it's a large document and it should be updated often, consider splitting it into what will be updated often and what won't.

Which gives us these questions when picking which database to use to store something in Firebase:

Is it going to be updated more than once per second? Yes: RTDB, No: Firestore
Is it larger than a couple of primitive fields? Yes: Firestore, No: RTDB
Do you need to search through your data? If yes, well Firestore but you should definitely use the algolia extension for that !

Conclusion

In conclusion, make sure to pick the right database for the right thing if you don't want to be one of these nightmare stories people tell you when you're talking about firebase. Big things go to Firestore, things that need frequent updates go to Realtime Database.

Handle concurrency in Cloud Firestore with a single boolean

Flavien Normand — Mon, 02 Jan 2023 09:19:41 +0000

Working with Cloud Firestore really feels amazing, realtime, scalability, easy to use, nice SDKs !

Then you stumble on concurrency, and it hurts. Cuncurrency has always been a big issue in many online apps, and there's a lot of ways to handle it, today i'm going to present to you one of the easiest ways I've ever seen, using a boolean that's quite hard to find when you just google "firestore concurrency" stuff.

First of all, the problem with concurrency

Let's imagine this flow of actions, with two users: A and B, both observing the same document ({ "name": "bar", "size": 10}) and editing it.

A edits field "name" to set it to "foo".
B edits field "size" to set it to 25. These actions happen with less than a second between each other.

Because of the Firestore restriction that limits write ops to 1/s/doc (a given document can only be edited once per second), this is what B will see, in sequence:

{ "name": "bar", "size": 10} // First read
{ "name": "bar", "size": 25} // Optimistic local write
{ "name": "foo", "size": 10} // Result from A's edit
{ "name": "foo", "size": 25} // Server result from B's edit

As you can see here, the user will see the data flicker from previous to new state, and this is because we don't filter on "wait until all operations are done".

The hasPendingWrites boolean

Firebase Web JS SDK has a nice tool to handle this kind of issues: hasPendingWrites (see more details in docs). What this does it tell us if the snapshot we're getting can be considered as stable or not for current client, as it won't be stable if it has pending writes.

This is even available in other Firebase SDKs but we're not going to cover them here.

How to use it

Here I'm going to give an example with @angular/fire's approach, using rxjs, but you can use it from any JS SDK library, or the SDK itself if you want to, as it's a SDK-level property.

With RxJS, you can simply use the filter operator on your snapshot listener:

docSnapshots(doc(firestore, 'exampleCollection', 'exampleKey'))
.pipe(
    filter(snap => !snap.metadata.hasPendingWrites),
    map(snap => snap.data())
)

Et voilà ! You now have a document data observable that will only emit data once all your write operations have been resolved 🎉

Optimisations Firebase, moins cher et plus performant

Flavien Normand — Mon, 18 Jan 2021 16:24:40 +0000

Créé en 2011 et racheté par Alphabet (Google) en 2014, Firebase se présente aujourd'hui comme une solution Backend as a Service très complète et performante. Son approche temps réel permet des applications très réactives et vivantes sans investissement sur le développement d'une solution backend.

Malgré cela, vous avez déjà peut-être hésité à l'utiliser, parce que le pricing n'était pas des plus clairs et peut être aussi parce que vous avez pu voir les fameuses firebase horror stories, comme celle-ci: https://hackernoon.com/how-we-spent-30k-usd-in-firebase-in-less-than-72-hours-307490bd24d (TL;DR: une entreprise a payé 30k€ à cause d’une mauvaise utilisation de firebase qui lui a créé une situation où la facturation grossissait de manière exponentielle)

Comme toutes les solutions cloud, une mauvaise utilisation peut conduire à de mauvaises surprises côté facturation, comme le démontre l'article ci-dessus. Nous allons voir ici comment éviter ces pièges en comprenant comment Firebase fonctionne, et comment il vous facture.

Realtime Database

Realtime Database, c'est la fonctionnalité de base de Firebase : une base de données sous forme d'un JSON géant dans lequel vous pouvez piocher et observer.

Le pricing

Dans Realtime Database, vous êtes facturés au Go transféré, ce qui en fait une base idéale pour stocker des petites données qui changent beaucoup, des indicateurs, des compteurs.

Ce qui peut mal tourner

Du fait que Realtime Database facture au Go transféré et qu'il propose du temps réel, si vous avez une donnée qui grossi de façon linéaire avec le nombre de vos utilisateurs, alors vous avez un pricing exponentiel. En effet, chaque nouvel utilisateur engendre un listener de plus, et fait grossir la donnée, ce qui envoie la modification aux autres utilisateurs.

C'est comme ça que #UnaVacaPorDeLaCalle a vu son pricing exploser.

Comment éviter ça?

Si vous avez de gros documents, préférez Firestore, et ne faites jamais une récupération d'un tableau complet pour avoir sa longueur, préférez une Cloud Function qui ira mettre à jour un compteur à la place, que vous pourrez récupérer facilement.

Firestore

Cloud Firestore, c'est une base de données NoSQL documents en temps réel. Sortie de beta il y a maintenant un an environ, elle est actuellement disponible dans la suite firebase.

Le pricing

Firestore vous fait payer par opération sur un document, et non sur le volume transféré, ce qui ouvre les possibilités pour des volumes de données plus importants.

Les limitations

Même si Firestore est plus intéressant pour les gros volumes de données, il faut garder à l'esprit qu'il a des limites que vous ne pouvez pas modifier:

Max. 1 opération par seconde par document.
Max. 1MB par document.

Ce sont les deux plus importantes pour le pricing,

Ce qui peut mal tourner, ou pas

Firestore est beaucoup moins sujet à des explosions de prix, de par son pricing par opération et le fait que récupérer une collection ne vous coûte qu'une opération au total et non une opération par document.

Cependant, il est important de savoir ce qui coûte des opérations, surtout dans les règles, qui peuvent récupérer un document au prix d'une opération.

Comment optimiser?

Dans Firestore, l'idée derrière l'optimisation est de réduire le nombre d'opérations de lecture que vous allez faire. Un design idéal veut donc que vous ayez un document qui contient tout ce dont un utilisateur a besoin. Cependant, cela reste à la merci de la limite de l'opération par seconde par document, et donc rend chaque "document utilisateur" modifiable uniquement une fois par seconde.

Même si vous ne pouvez pas atteindre cet idéal, avoir moins d'opérations de lecture obligatoires (au lancement de votre application, sans interaction utilisateur) est la priorité et peut fortement impacter votre pricing Firestore.

Conclusion

En conclusion, pensez toujours à l'utilisation de vous allez faire d'une des bases de Firebase et pensez toujours aux anti patterns qui peuvent vous coûter cher afin de les éviter, utilisez les cloud fonctions avec les events de base de données afin de faire des opérations de type count.

Vous êtes intéressé•e par Firebase et vous voudriez en apprendre plus ? N'hésitez pas à nous contacter pour une formation.

Enhance serialization with @kaiu/serializer

Flavien Normand — Mon, 16 Dec 2019 17:04:39 +0000

In today's frontend applications, there's one thing we almost all have in common: we want to serialize JSON strings into objects.

Thanks to the amazing tool that JSON is in Javascript, this became an easy and safe way to handle serialization inside our applications.

But what if your objects need to be an instance of a class?

In a project, we had an issue that I think many developpers are facing: having methods inside objects that you're getting from an API.

Let's say you have a Foo class inside your typescript application:

export class Foo {
    bar: string;
    baz: number;

    getBarBaz():string {
        return `${this.bar}${this.baz}`;
    }
}

If you get an object that comes from an endpoint in which you're storing foos, you want the models to have the barBarBaz() method.

Let's say you have this JSON coming from an API:

{
    "bar": "example",
    "baz": 0
}

You will want it to end up as an instance of Foo, so you'll end up having to create manual mapping for each class you want to persist inside the API, something like this:

export class Foo implements APIModel {
    bar: string;
    baz: number;

    getBarBaz():string {
        return `${this.bar}${this.baz}`;
    }

    hydrateFromJSONObject(obj: Partial<Foo>):Foo {
        this.bar = obj.bar;
        this.baz = obj.baz;
    }
}

But then you start to work with more complex structures, with complex field that are class instances themselves, or even arrays of class instances. And things start to become quite difficult to handle without having to write a lot of repetitive code.

But there's a solution !

Because we found this hard to maintain, we started thinking about a solution that uses the power of decorators to make all this easier for everyone.

That's why we created @kaiu/serializer, a simple library to deal with this easily, using simple decorators.

Example

import { Serializer } from '@kaiu/serializer';

const serializer = new Serializer();
const foo: Foo = serializer.deserialize<Foo>(JSON.parse(jsonString), Foo);

console.log(foo.getBarBaz()); // Will properly print "example0" if given the example json.

There is some super cool decorators for specific use cases, like @FieldName (docs) which allows you to map a field to a different name in your application, or @DeserializeAs (docs) which allows you to map a field to a class instance.

Some advanced use cases are also implemented, like inheritance handling, so you can have a parent class with multiple children classes, the serializer will be able to use the matching child class based on a given field value (docs)

For the angular devs

As our initial use case was for an Angular application, we created a simple wrapper that provides the serializer as a service in a module, with easy configuration injection. You can find it on npmjs: @kaiu/ng-serializer

Feel free to star the project on github or create issues and PRs, they are always welcome 😄

If you have any question, please don't hesitate to ask them in the comments or ping me on twitter: @Supamiu_.

Apollo on Angular, almost a love story

Flavien Normand — Wed, 11 Dec 2019 13:04:45 +0000

Apollo

Apollo is a (if not the) graphQL client for frontend applications.

It is available for React, Angular, Vue and many more, with great docs available in order to get started easily and understand how to use its power.

Apollo-angular

Apollo-angular is the Angular wrapper for the apollo client. It fits particularly well with Angular because all its API was designed to use Observable and reactive paradigm. If you use Angular daily you should know how much it uses Observables and how it can be annoying when an external API doesn't. Apollo-angular will help you to develop efficiently with Apollo and Angular, by providing everything you need the way you need it.

Getting started

When getting started on apollo-angular, you can simply follow the example shown in the docs, which will allow you to install the library, then configure its module and finally provide a factory for Apollo.

But then, it's done?

Even if this example is perfect in order to get you started with the library, it is far from being good for production use, because of how you have to setup authorization header.

The main reason being that if your application needs to change something on the client's configuration, you won't be able to create a new Apollo instance, instead you will get an error saying apollo has been already created. This example is only good if your token has no expiration date, or if it's shorter than your application's life (if it's not meant to be opened for this long).

The reason for this is that Apollo client has to be single instance, it is made to be singleton and once you created one, especially in an Angular context, where everything is a service, it's hard to delete it.

The solution

Context

In my case, I'm using Firebase for the authentication part, in order to query my graphQL API, created using Hasura. To secure the application, it requires a classic JWT authentication to match a role in the IAM, unauthenticated access is forbidden, in order to avoid unwanted spam of requests from people wanting to just hammer the API.

The problem

When the application is running, as it's a SPA, it can live longer than the JWT, meaning that I will have to refresh it, but because of that, I would have to create a new Apollo instance using the method linked above, and we know it, it's not going to work.

The solution

When you think about it, graphQL requests are just POST requests to the graphQL endoint, which means they use the http service from Angular. And this means that we can use interceptors !

This allows the token to be reloaded anytime, without having to worry about the Apollo instance, as it can safely be kept as a singleton, with a proper authorization header that can be reloaded !

You can now enjoy your graphQL API with the same application instance during several hours ! 🎉

If you enjoyed this article do not hesitate to share it on twitter. If you have any questions or feedback please don't hesitate to tweet me @Supamiu_