In this post will look into how we can add firestore data into their platform via the Next JS Web SDK and Node JS cloud functions SDK manually as well as set up cloud function firestore listeners to add the data automatically.
We will be discussing the following concepts in this post.
- Adding data via the cloud functions Node JS SDK with help of Express JS routes in a manual way.
- Automated Cloud Function listeners of Firestore to auto add data to MeiliSearch instance whenever a data is being added to Firestore.
This post requires you to have knowledge about Meilisearch API's which I have discussed in my previous posts in this series.
- Express JS routes via cloud functions.
For this you will need a firebase paid "Blaze" account with which you can deploy your express server. I am going to assume you already have one. So inorder to create a Express JS project with cloud functions you can refer to the below posts.
https://dev.to/sherifshahin/using-firebase-cloud-functions-with-nodejs-and-express-2pml
After you have initialised your firebase project, we are going to deploy an express server inside our cloud functions to get access to REST API routes like Get, Post, Delete and Update. For this tutorial we are going to use post routes to add our data points to the Meilisearch instance as well as getting data points from our deployed Meilisearch instance.
This is a very important point to note. The routes below will let us manually add the data points to Meilisearch. So for example if you add data to Firestore, or any other database, we can call this route and add it to the Meilisearch instance MANUALLY. This is database agnostic which means you can use any database to do this operation or no database at all. We are declaring these routes to give you an option to call the API routes when ever you add a document in your frontend application. It may be using firebase Web V9 via addDoc() or in a react native app with firestore().collection().add(). After you called these firestore addition functions we can call these API routes.
For example,
addDoc(data) - Firebase Web V9 SDK
await axios.post(`apiurl/add-documents`,data)
// Express JS Route called from axios
I am going to create 5 routes.
- Creating an index
- Deleting an index
- Adding data points directly into Meilisearch
- Deleting a data point
- Updating a data point
- Search Route
First lets setup the code for an express application and import the Meilisearch library.
const express = require("express");
const cors = require("cors")({ origin: true });
var useragent = require("express-useragent");
const { MeiliSearch } = require("meilisearch");
const mellisearch = express();
mellisearch.use(cors);
mellisearch.use(useragent.express());
mellisearch.set("trust proxy", true);
const client = new MeiliSearch({
host: "yourapi",
apiKey: "apikey",
});
mellisearch.get("/test", async (req, res) => {
res.send({ message: "ok" });
});
Now lets create a route for creating an index.
mellisearch.post("/create-index", async (req, res) => {
const index = req.body.index // "movies"
await client.createIndex(index, { primaryKey: "movieId" });
res.send({ message: "created" });
});
If we want to delete that index.
mellisearch.post("/delete-index", async (req, res) => {
const index = req.body.index;
await client.deleteIndexIfExists(index);
res.send({ message: "deleted" });
});
Add the data points to melli search
mellisearch.post("/add-documents", async (req, res) => {
const index = req.body.index;
const data = req.body.data;
await client.index(index).addDocuments([data]);
res.send({ message: "created" });
});
Deleting the data points
mellisearch.post("/delete-document", async (req, res) => {
const index = req.body.index;
const id = req.body.id;
await client.index(index).deleteDocument(id);
res.send({ message: "deleted" });
});
Updating the data points
mellisearch.post("/update-documents", async (req, res) => {
const index = req.body.index;
const data = req.body.data;
await client.index(index).updateDocuments([data]);
res.send({ message: "updated" });
});
Now the search route, here we can limit the amount of data points we retreive from the instance. We are going to limit it to 7 for now.
mellisearch.post("/search", async (req, res) => {
const index = req.body.index;
const search = req.body.search;
const response = await client.index(index).search(search, {
limit: 7,
showRankingScore: true,
});
res.send({ hits: response.hits });
});
Below is the complete working code. Also please install a library called cors which prevents cors errors when used in your frontend application.
npm i cors
The entire code
const express = require("express");
const cors = require("cors")({ origin: true });
const { MeiliSearch } = require("meilisearch");
const mellisearch = express();
mellisearch.use(cors);
const client = new MeiliSearch({
host: "yourhost.com",
apiKey: "yourkey",
});
mellisearch.post("/create-index", async (req, res) => {
const index = req.body.index;
const primaryKey = req.body.primaryKey;
await client.createIndex(index, { primaryKey: primaryKey });
res.send({ message: "ok" });
});
mellisearch.post("/add-document", async (req, res) => {
const index = req.body.index;
const data = req.body.data;
client
.index(index)
.getRawInfo()
.then(async () => {
await client.index(index).addDocuments([data]);
})
.catch(async (e) => {
await client.createIndex(index, { primaryKey: "movieId" });
await client.index(index).addDocuments([data]);
});
res.send({ message: "ok" });
});
mellisearch.post("/update-document", async (req, res) => {
const index = req.body.index;
const data = req.body.data;
await client.index(index).updateDocuments([data]);
res.send({ message: "updated" });
});
mellisearch.post("/delete-document", async (req, res) => {
const index = req.body.index;
const id = req.body.id;
await client.index(index).deleteDocument(id);
res.send({ message: "ok" });
});
mellisearch.post("/delete-index", async (req, res) => {
const index = req.body.index;
await client.deleteIndexIfExists(index);
res.send({ message: "ok" });
});
mellisearch.post("/delete-documents", async (req, res) => {
const index = req.body.index;
await client.index(index).deleteAllDocuments();
res.send({ message: "ok" });
});
mellisearch.post("/search", async (req, res) => {
try {
const index = req.body.index;
const search = req.body.search;
const response = await client.index(index).search(search, {
limit: 7,
showRankingScore: true,
});
res.send({ hits: response.hits });
} catch (error) {
res.send({ error: error });
}
});
module.exports = {
mellisearch,
};
I have provided additional code where, if the provided index is not created, it automatically creates it and adds the data to that index.
Remember to have a unique id like "moviesId" or "gameId" so that it gets indexed with the unique id in that object.
- Automated Cloud Function listeners of Firestore.
We can use cloud function Firestore listeners to listen to data being added, updated or deleted and automatically make changes in the Meilisearch instance. This cuts out the time and effort needed to manually add the data points as the other solutions discussed in this post. This can be also applied for real time database changes but we are focused on only Firestore in this post.
When document gets added to a collection in firestore.
exports.addToMelliSearch = functions.firestore
.document('movies/{movieId}')
.onCreate((snap, context) => {
const index = 'movies'
const data = snap.data();
await client.index(index).addDocuments([data]);
});
When document gets updated in a collection.
exports.updateMovieMelliSearch = functions.firestore
.document('movies/{movieId}')
.onUpdate((change, context) => {
const newValue = change.after.data();
const index = 'movies'
await client.index(index).updateDocuments([newValue]);
});
When document gets deleted in a collection.
exports.deleteMovieMelliSearch = functions.firestore
.document('movies/{movieId}')
.onDelete((snap, context) => {
const deletedValue = snap.data();
await client.index(index).deleteDocument(deletedValue.movieId);
// Or that particular data point's id.
});
These are the different ways you can add data into your melli search instance and get the data points.
But you can do this in literally in so many ways depending upon your tech stack with any language. I have provided the ways with which you can add data points in a Javascript + Express/Node JS way with Firestore as a database.
You can view the complete code and see how the search interface works in a live demo below.
Top comments (0)