Elasticsearch migration guide
Contents:
- Introduction
- Differences
- Migration script
- Front-end components
- Conclusion
If you are currently using Elasticsearch and plan to migrate to Meilisearch for your application, follow this guide to help you with the transition.
Differences
Before migrating, you may want to understand better what differentiates Elasticsearch from Meilisearch.
Elasticsearch is a distributed search engine that uses clusters and nodes to optimize performance, whereas Meilisearch is a single instance/node with customizable startup options. Meilisearch is still evolving, and many functionalities are not yet supported. You can think of Meilsearch as similar to the Elasticsearch component in the ELK Stack, but without the rest of the components like Kibana, Logstash and Beats.
You may see what new upgrades our developers are working on by visiting the public roadmap.
Migration Script
This guide will show you how to migrate data from Elasticsearch to Meilisearch using NodeJs and it has no impact on the programming language that will be used with Meilisearch in the future.
Initialize project
Start by creating a directory elastic-meilisearch-migration
and initialize it as an npm project:
mkdir elastic-meilisearch-migration
cd elastic-meilisearch-migration
npm init -y
Next, create a script.js
file:
touch script.js
This file will contain our migration code.
Install required packages
To get started, you'll need two npm packages. The first is @elastic/elasticsearch
, the JavaScript client for the ElasticSearch API, and the second is meilisearch
, the JavaScript client for the Meilisearch API. To install them, run the below command:
npm install --save @elastic/elasticsearch meilisearch
*Create client objects*
We need client instances of both Elasticsearch and Meilisearch to access their respective API. Paste the below code in script.js
:
const { Client: ElasticSearch } = require('@elastic/elasticsearch')
const { MeiliSearch } = require('meilisearch')
const esClient = new ElasticSearch({
cloud: {
id: 'ES_CLOUD_ID',
},
auth: {
username: 'ES_USER_NAME',
password: 'ES_PASSWORD',
},
})
const meiliClient = new MeiliSearch({
host: 'MEILI_HOST',
apiKey: 'MEILI_API_KEY',
})
First, we create an Elasticsearch client esClient
using elastic cloud credentials. You can also try other authentication methods. Replace the original values for ES_CLOUD_ID
,ES_USER_NAME
and ES_PASSWORD
.
Then, we create a Meilisearch client meiliClient
by providing the host url and API key. Also, replace MEILI_HOST
and MEILI_API_KEY
with their respective values.
*Fetch data from Elasticsearch*
This could be a challenging step because we will iterate the index until the entire index is not searched. In each iteration, we search for the subsequent 10,000 documents, format them, and store them. The iteration will end when we get an empty result. To achieve such a paginated search result, we will use the Search After method.
This logic has been reduced to the below code.
async function getElasticSearchDocuments() {
let isFetchingComplete = false
let search_after = null
let documents = []
const searchObject = {
index: 'ES_INDEX',
size: 10000,
body: {
query: {
match_all: {},
},
},
sort: {
'id.keyword': 'asc',
},
}
do {
if (search_after) searchObject.search_after = search_after
const result = await esClient.search(searchObject)
const hits = result.hits.hits
isFetchingComplete = hits.length == 0
search_after = hits[hits.length - 1]?.sort || null
documents = documents.concat(hits.map(hit => hit._source))
} while (isFetchingComplete == false)
return documents
}
Replace ES_INDEX
with the index name you want to export.
Upload data to Meilisearch
Add the following code:
const documents = await getElasticSearchDocuments()
const meiliIndex = meiliClient.index('MEILI_INDEX')
await meiliIndex.addDocuments(documents)
The variable documents
contain an array of documents ready to be uploaded to Meilisearch.
After that, we get the index instance and upload all the documents with the addDocuments
method. Meilisearch will create the index if it doesn't already exist.
Note: The
addDocuments
method is asynchronous, which means the request is not handled as soon as it is received, but it is put into a queue and processed.
Replace MEILI_INDEX
with the index name where you would like documents to be added.
That's all! When you're ready to run the script, enter the below command:
node script.js
Finished Script
Here's the complete script code.
const { Client: ElasticSearch } = require('@elastic/elasticsearch')
const { MeiliSearch } = require('meilisearch')
const esClient = new ElasticSearch({
cloud: {
id: 'ES_CLOUD_ID',
},
auth: {
username: 'ES_USER_NAME',
password: 'ES_PASSWORD',
},
})
const meiliClient = new MeiliSearch({
host: 'MEILI_HOST',
apiKey: 'MEILI_API_KEY',
})
async function getElasticSearchDocuments() {
let isFetchingComplete = false
let search_after = null
let documents = []
const searchObject = {
index: 'ES_INDEX',
size: 10000,
body: {
query: {
match_all: {},
},
},
sort: {
'id.keyword': 'asc',
},
}
do {
if (search_after) searchObject.search_after = search_after
const result = await esClient.search(searchObject)
const hits = result.hits.hits
isFetchingComplete = hits.length == 0
search_after = hits[hits.length - 1]?.sort || null
documents = documents.concat(hits.map(hit => hit._source))
} while (isFetchingComplete == false)
return documents
}
;(async () => {
const documents = await getElasticSearchDocuments()
const meiliIndex = meiliClient.index('MEILI_INDEX')
await meiliIndex.addDocuments(documents)
})()
Front-end components
Instant Meilisearch is a plugin connecting your Meilisearch instance with InstantSearch, giving you access to many (but not all) of the same front-end components as Algolia users. Here is an up-to-date list of components compatible with Instant Meilisearch.
Conclusion
In summary, we have discussed the primary advantages and limitations of both search engines, but our discussion has only touched the tip of the iceberg. The functionalities of Meilisearch and Elasticsearch surpass what has been covered in this article. To obtain a comprehensive comparison between Meilisearch, Elasticsearch, and other search engines, please consult the dedicated section in our documentation.
Top comments (0)