Firebase Emulators Setup - Tips to overcome trouble, hurdles and pitfalls to get them up, running and data resyncing

Koen (K.W.H.J.) van der Kamp — Sat, 01 May 2021 23:20:15 +0000

If you do not have any experience with the implementation of firebase emulator functionality, start at step 1.

Firebase emulators setup. Sounds easy ...
Firebase emulator UI. What should I see ...
Frontend onto ‘firestore emulator’. Surprisingly no data ... Restarting, trouble! Ports occupied! Now what? Solution
Copy online database content into the local environment. So far, so ...? Finally ...!
Using the local environment data, anytime, just one command?
Resync local environment with online database content...

Preface

The process described is performed with an existing working Angular app on top of Firebase Serverless Services to present online stored data. From ‘cost- and efficiency perspective’ this should and can be improved.

Current status at time of writing
Frontend

        "@angular/core": "^11.2.10",
        "@angular/fire": "^6.1.4",
        "firebase": "^8.4.1",
        "firebase-functions": "^3.13.2",
        "firebaseui": "^4.8.0",

Functions

        "firebase": "^8.4.1",
        "firebase-admin": "^9.6.0",
        "firebase-functions": "^3.13.2",
        "firebase-tools": "^9.10.0",

I am playing with Visual Studio Code.
My terminal of preference is Git Bash $

Be aware! Getting the firebase emulator functionality running locally, Java functionality from Oracle is required. If you, for whatever reason, do not have Java running ‘under the hood’ figure this out in advance. See Install, configure and integrate Local Emulator Suite.

1. Firebase emulators setup

Open a terminal and navigate to the project's root folder.
Type $ firebase init emulators
On the terminal, the emulators setup questions are popping up. Questions asking you to select the emulators of your choice. If you are not familiar start with the Functions -, Firestore - and Hosting Emulator.

Next, the selected emulators with their configured port numbers are presented on which they will run. Download your emulators. If, at this stage the emulators aren’t downloaded, it will happen automatically during the emulators execution step next, don’t worry. Close the terminal.

2. Firebase Emulator UI

With the Emulators in place, open another terminal, at the level of your project's root folder. For me this is ~/functions/~.
Type $ firebase emulators:start
A schema pops up with information about the installed emulators.

On one of my machines a Java dialog pops up telling me on which port the FIRESTORE_EMULATOR_HOST is ‘living’, whereas on another device not.
On both devices, however on the Visual Studio Code terminal the firebase emulators are announced ‘ready’.
Notice the port numbers for the firestore emulator host e.g. may differ at your side.

If you don't remember the port numbers configured, find them in the firebase.json file.

View the firebase emulator UI in your favorite browser.

In most case, especially if you start the emulator functionality for the first time, the tables at the [Authentication] and [Firestore] tabs are empty. Data for local use seems not available!

Downloading your online data into your local environment will be your next challenge.

But first, you need to hook your frontend onto the host, the ‘firestore emulator’.

3. Hook frontend onto the ‘firestore emulator’ host

First, go to file app.module.ts and find the following import.

Import {
    AngularFirestoreModule
} from '@angular/fire/firestore';

Load the desired ‘settings’ and alias them. It should look like this

Import {
    AngularFirestoreModule,
    SETTINGS as FIRESTORE_SETTINGS,
} from '@angular/fire/firestore';

Next, go to the providers array and add a new provider-set as follows

        {
            provide: FIRESTORE_SETTINGS,
            useValue: environment.emulator ? {
                host: 'localhost:8080',
                ssl: false
            } : undefined
        },

Notice the provider is going to ask your app about the emulator property in the environment.ts file and it ask if the emulator-property is set to ‘true’.
Notice that at this port during configuration the CLI has been added.

No emulator property is currently set. Add it to the environment object in environment.ts.

Export const environment = {
    production: false,
    emulator: true,
    firebase: {
        apiKey:

Notice that in the environment.prod.ts also an environment object exists. If you omit the emulator property in this object with the value set to 'false' you will end up with an error during $ ng build --prod.

Now, when you run command $ ng serve the environment.ts file will be read and it will run against the local emulators. Otherwise when running command $ ng build --prod the environment.prod.ts file is read and it will run against the online firestore instance.

Coming to this stage, when running command $ ng serve, with the firebase emulators still running, the dashboard (localhost:8080 | http://localhost:4000/firestore - Emulator UI) is showing empty tables.

Two hurdles you might face here.

It is possible, being busy with hooking up your frontend to the firestore emulator host, you accidently have terminated your terminal, expecting you ‘switched off’ those running emulators. Your wrong! Those emulators are still running! The other way around, because data is not shown might give you the idea to re-start those emulators thru switching them off. Switching them off is not simply closing and re-opening a terminal and trying to start those emulators again.

View the next two screenshots. The first one is my frontend running against my online database and the second one running against the firestore emulator during my first attempt of getting those emulators properly running. No data!

Keep in mind at this stage in this story, we still did not learn how to get the data stored into the local emulator environment.

Don’t go, don’t stop! Keep on reading, solve these hurdles!

In case, for whatever reason, the terminal on which the firebase emulators were running, has been closed and you are trying to re-start them thru opening a separate terminal executing command $ firebase emulators:start or $ firebase emulators:start --only firestore, most likely you expect to get back on track again without trouble. Nope, you will, very likely, still having trouble, don’t worry!

Running command $ firebase emulators:start --only firestore the terminal will most likely tell the firebase emulators can’t be restarted because some ports are in use.

also at the dashboard a dialog has popped up telling you about a disconnection with a rather misleading instruction.

The ports are in use!

Trouble? Actually not!

Well, sit back, grab a mug of coffee or another relief, take a deep breath and continue!

Those ports ‘in use’ needs to be released, but how?

Here is where you get a bit frustrated because you might feel you are send off into the labyrinth of Google's digital swamp.

Just for learning purposes, old school stuff bubbles up. Windows netstat service. Open a Windows (or Powershell) prompt.
Type (PS) C:\>netstat -ano
A large list of port numbers are flashing by.

Search for the desired port numbers. Remember, those port numbers which are presented to you at the stage of the firebase emulators setup earlier on or the ones stated in the error messages.
Type (PS) C:\>taskkill /f /im [PID Firestore Emulator Host]
Likely also the other one on which the emulator UI is running.
Type (PS) C:\>taskkill /f /im [PID Firestore Emulator UI]

At this stage, both ports are released.

Hurdles

You did need another service from which you hardly knew its existence I presume (something from the old ages at the time the earth crust hasn’t yet cooled down, ha), it needs at least two commands and the used of another console.

Luckily, the old school days are over, new functionality arrived.
Since we are working on an Angular frontend on top of a firebase framework, we know better. We have learned about availability of many packages. Gamble, there might one be available able to release ports.

Yes! Several, for instance: Kill ports. This one needs only a few seconds to read! It is simple tool and able to release several closed ports at once. Nice!

Instead of the netstat distraction, use command $ npx kill-port --port X, Y or $ npx kill-port Z, P (X, Y, Z and P are port numbers, in this tutorial 4000, 8080).

Wait, are we now back on track again? No, still no local data available!

4. Copy online database content into local environment

We are coming close! First, we need to overcome another challenge!

Arriving at this stage, it should actually be mentioned, close everything in such a way as if you are starting fresh in the morning, having your device completely switched-off for the night (clean desk policy). However, if you did not, continue and face what happens. Important learnings!

We need another package for exporting from the online database and importing that data into the local environment. We are going to make use the node-firestore-import-export package. This package has excellent documentation.

This package helps to challenge a four steps process exporting and importing data.

At first, exporting the online database content, then, starting only the firestore emulator, next, telling the system to use the correct host port number and at last, importing the data into the localhost emulator environment.

Another hurdle

Before you are being eligible to use the export and import functionality from the just installed package you need the allowance from your online firebase environment.

Go into your online firebase console and download a service account key. Keep in mind! This key is basically an admin key for your entire app. Do NOT store this key in any version control system whatsoever. Take care to 'ignore' the container of the key it self in your version control system.

The service-account credentials can be found under Project Settings > Service Accounts. Hit link Generate new private key, download that JSON file and keep it in a private spot.

Once you have the service-account key (json file) properly stored, open a new terminal and install the node import-export package. Type $ npm install -g node-firestore-import-export

Once installed, you finally are eligible to use the export - and import functionality. Open a new terminal and go to your functions level again. Keep in mind the spot (sub level) where the account-service key file has been stored.

Type $ firestore-export --accountCredentials <(path-to-your-account-service-file/)your-account-service.json> --backupFile <(another-path-to-store-output/)your-output-name.json>
Notice you are allowed to change names for your service key and/of backup file.
Notice the 'your-output-name.json' file is actually the reflection of your online database. Consider it as your local test database.

If everything went well, you most likely end up with a green line in your terminal telling you

Next, type $ firebase emulators:start --only firestore
It is, again, very likely, you’ll met trouble.

It seems, again, you are not able to start the firestore emulator host because the system is telling you the firestore emulator host is (still) in use.

Head back to the instructions how to release that specific port. Best practice, release those ports on a different terminal. If you are still in the same flow, to the terminal on which the export command has been executed.

In case you did close the terminal, VS Code or quit for the day anyway, you do need to repeat the export step again.

So, the 2nd step in this process is retrieving data from online and should end up like this

Now with having ONLY the firestore emulator running, open another terminal and tell the system on which port number the firestore emulator host is living (3rd step).
Type $ export FIRESTORE_EMULATOR_HOST=0.0.0.0:<your-firestore-host-port-number>

Finally import the downloaded data into the local environment. Type $ firestore-import --accountCredentials <(path-to-your-account-service-file/)your-account-service.json> --backupFile <(path-to-you-backup-file/)your-output-name.json>

Execute the import command and answer the question

Hit ‘[y]’ and wait!

We have done it, sure!

Go to your dashboard and watch! Most likely you will see a progress bar telling you the process being busy with loading the imported localhost data.

Finally, the data is there!

Yippy! Really yippy? Actually not! You are not save, yet!

Instead, the biggest pitfall is on the rise!

At this stage, it is possible, in whatever flow you have performed this flow, up to this stage, you accidently did something wrong. Possibly a wrong firestore host port number has been used somewhere, for instance?

Check your host port stated at the terminal after have executing the command $ firebase emulators:start. Check also, in case a Java console dialog is visible at your status bar if the correct port number is printed, or if the correct port number is written in your angular.json and if the correct number is written in your app.module.ts (provider section).

Check all those spots for the correct number(s) otherwise you most likely get the following errors presented. Several ones for all entities exported which need to be imported into the local environment.

Having luck? No series of errors with code 14?

In case a list of errors are flashing by, one error object for each of the 'to be importing' entities:
{ ..., code: 14, details: 'No connection established', ...}
It means, the underlying Java functionality is not running!
So, you have to start all over again.
Start again with: $ firebase emulators:start --only firestore

Everything went well! Ha, ha, as said, you know already what is coming up, 'you are still not there, yet'!

Notice once you have added all the data into your local environment and you have been busy for a long time now, it is time for whatever happy hour you need. You have quit for the day!

Whatever you did, at restart, disappointment is on the raise.

Starting your device, starting your development environment, you did everything right, but data on the dashboard is not there, it’s gone! You’ll have to start all over again. Facing all pitfalls once again. Keep this in mind!

Finally, stay strong for a minute or two, then, only then you’ll be save, promised!

5. Using local environment data, anytime, just ONE command?

Luckily, firebase does have an option to back-up the localhost emulator data. Working in the same flow, it is likely the firestore emulator is still running and having all the data in place. If not, get back and try to reach this stage again. Open a separate terminal.
Type $ firebase emulators:export ./emulators.backup

What this command does is, it creates a folder called ~/emulators.backup/. In this case just below your root level which is apparently your ~/functions/ folder. At this stage, all the data, which for a second or two ago lived in memory, is stored in a certain kind of an unsupported text encoding file. Notice you were able to change the name into whatever you wanted.

I think it is not worthwhile to store this 'container' (emulators.backup) incl. content into a version control system, it should be ignore!

Now, only now, you are save! Really? Close everything, even your device, if needed. Come back at whenever time of your preference, get back on track, just open a terminal at your ~/functions/* level.

No, no, not simply executing the 'emulators.backup'. If you do, you will be blocked!

So, first restarting the firebase emulator is needed.
Type $ firebase emulators:start --only firestore
Thereafter ON ANOTHER TERMINAL
Type $ firebase emulators:start --import=./emulators.backup

These are two commands to remember.
But there is still another factor to also having some intrinsic value to be mentioned. Know on which port your local environment is running. If you are new to Firebase Emulators, (as I was a little while ago), you expect your frontend is running on localhost:4200, right!

In case you have 'restarted' (everyhting has been shut down, meaning you are a bit ignorant and started your development environment with $ ng serve right away and remembered on time to start your 'middle ware' as well (oké executing it on a different terminal $ firebase emulators:start --import=./emulators.backup), you might expect data on your frontend. No, it isn't there. Don't be tricked!

Notice, there is not a live 'connection' between those two. Look into the message when having executed $ firebase emulators:start --import=./emulators.backup.

Notice that you are familiar with having your frontend running on port 4200. It is simple, you hosting port is mentioned in the output. So, changing the port number (from localhost:4200 into local:5000), does show data. However, you're busy with developing, you will see the latest 'production' (from your ~/dist/), and not your latest achievements. Don't be surprised!

Notice, using firebase emulators as described, you might see a warning during execution of $ firebase emulators:start --import=./emulators.backup. I haven't figured out yet why this warning elevates. The warning is not very descriptive, it says
! emulators: It seems that you are running multiple instances of the emulator suite for project <your project>. This may result in unexpected behavior.

Also, it seems somewhat of a 'contradiction in terms' whereas you think about restarting firebase emulators you instantaneously receive a blocking message about 'ports in use'. How is it possible to get simultaneously running multiple instances of the emulator suite? I don't know yet! If somebody knows the answer, share it as a discussion item.

6. Resync local environment with online database content

Know in advance if port(s) are 'in use' (meaning if Java functionality 'under-the-hood' is running)!

I presume, you started fresh on a sunny bright morning or whatever day part of your preference on whatever spot you like (meaning just setting-up a new session).

Go on a terminal to the root of your 'functions' project
Type $ cd functions

Best practice, release occupied ports anyway, take the long road! Actually this means I contradict myself a bit, from own experience, if you take the short cut, which I tried several times by now, you will be surprised as I was, with an empty local environment (no data). It isn't straightforward in some way, as instructed somewhere else; for resyncing you only need to execute the import statement calling from the ~/emulators.backup/ folder.

So, from my own experiences so far, resyncing, actually means a repeat of these steps.

Release 'ports-in-use' (

I don't like the word 'kill' in this perspective. Live ain't not simply a game, I am not a favorite of gamification everything
Type $ npx kill-port --port X,Y (or $ npx kill-port X,Y)

Restart the Java engine

Type $ firebase emulators:start --only firestore

Open another terminal

Type $ cd functions

Export your latest online database content

Type $ firestore-export --accountCredentials <(path-to-your-account-service-file/)your-account-service.json> --backupFile <(path-to-you-backup-file/)your-output-name.json>

Set correct firebase host port

Notice you are still on the same terminal where the export command has been executed!
Type $ export FIRESTORE_EMULATOR_HOST=0.0.0.0:8080

Import the latest exported content from your memory

Notice you are still on the same terminal where the export - and correct port commands has been executed!
Type $ firestore-import --accountCredentials <(path-to-your-account-service-file/)your-account-service.json> --backupFile <(path-to-you-backup-file/)your-output-name.json>

Usage

Notice you are still on the same terminal where the export -, correct port - and import commands has been executed!
Type $ firebase emulators:export ./emulators.backup
Notice

Hit [y] and [Enter]
See what happens! Your data has been renewed!

It is fun, having the emulators dashboard open at the same time of execution. It is magic! That’s what it is!

Used sources
How to set up Firebase Emulator for local development (Jorge Vergara)
Easily kill a process running on a port (Ben Ilegbodu)
The Local Firebase Emulator UI in 15 minutes (David East)

DEV Community: Koen (K.W.H.J.) van der Kamp

Terminal message: "bash: syntax error near unexpected token '&' in Visual Studio Code"

Why?

How to fix!

Result

The collection reference method *.valueChanges(...) visualized (and explained)!