2.8 Update - ElasticSearch Research

This blog is part of blog debt, which should have been posted on Friday.

Release 2.8.0

We had our 2.8 release in week 8, and it was quite a big one, including the merge of Satellite into Telescope. Furthermore, in between 2.8.0 and 2.8.1, Telescope also had its own Docker Registry set up, and we have even started using the feed-discovery image from there.

For me, I didn't get much new code into 2.8. Instead, I had to do a lot of reading and experimenting on ElasticSearch indexing and autocomplete. Starting on Friday, I also managed to find some really weird bugs following the Satellite merge.

PNPM, Jest and Satellite.

Let's start with the bugs first.

The background was that on Friday, ElasticSearch-mock had a 2.0.0 release that would allow its new version to work with ElasticSearch v8.0+. Based on my blog from last week, this was the main thing left that was stopping us from updating to ElasticSearch v8.0+. So I got excited and had plans for ES to be updated in Telescope by the end of the day. Motto of this story, software development is never what you expect.

Much too many Satellite tests

I began with updating ES and ES-mock in the Satellite inside of Telescope. However, I quickly ran into these errors in my tests:

TypeError: Class extends value undefined is not a constructor or null

       9 |   const client = new Client({
      10 |     node: 'http://localhost:9200',
    > 11 |     Connection: mock.getConnection(),
         |                      ^
      12 |   });
      13 |   // Provide a fake health check
      14 |   client.cluster.health = () => Promise.resolve();

      at buildConnectionClass (node_modules/.pnpm/@elastic+elasticsearch-mock@2.0.0/node_modules/@elastic/elasticsearch-mock/index.js:121:32)
      at Mocker.getConnection (node_modules/.pnpm/@elastic+elasticsearch-mock@2.0.0/node_modules/@elastic/elasticsearch-mock/index.js:116:12)
      at new MockClient (src/satellite/src/elastic.js:11:22)
      at Elastic (src/satellite/src/elastic.js:32:14)
      at Object.<anonymous> (src/satellite/test.js:999:14)

By this point I'm already too familiar with this error. This meant that the ES client and the ES-mock weren't compatible. One of them was using the older version, while the other one was using the new one. But for me, this didn't make sense since I had updated both in Satellite. So I went to my fork of Satellite, with it being its own repo, and did the same changes.

All the tests passed.

While this stumped me even further, I also noticed some really odd numbers occurring when I ran tests on both sides. In Telescope, the Satellite test suite was running more times than it should be.

I filed the issue, and Prof set up a PR to fix it. Only, it wasn't as quick a fix as we had thought. It lead into an afternoon of code sleuthing for me, along with the Prof (humphd) and cindyledev, one of our alumni. This all happened during a pretty hectic afternoon, as in this time, Telescope 2.8.0 was being released, and the Docker Registry was being set up.

Tweaking Jest Config

Once that PR was merged, I still hadn't solved my mystery of failing Satellite tests. After quite a long while of pouring into the Jest Config Documentation, I believed I had found the culprit, moduleDirectories, and added this line in the Satellite jest config.

moduleDirectories: ['<rootDir>/src/satellite/node_modules', 'node_modules'],

From what I gather, it meant jest will look at the Satellite node_modules to find the dependencies it needs first, and if it doesn't find them, it'll look into the root node_modules.

PNPM and breaking the Search Service

I was ecstatic to get the Satellite tests working, only to find in horror that my Search tests were now failing.

I would encourage you, the reader, to read the comments in the issue and the PR. I hope it is detailed enough to explain the issue involved. Josue (alumni) and I tried to tackle this way into the late night, but to no avail.

A "simple" summary of this issue is that something is up with PNPM and its making it that our Services are using the local Satellite instead of the Satellite package they're supposed to. It broke Search because now, Search wants to use the updated ES inside of the local Satellite.

Learning to run the Search Service locally

In between this frustration and headache, I learned how to test the Search service locally. Not with building a docker image, but really locally.

Inside of the Search service, download env-cmd as a dev dependency.
Still in the Search service, create a script

"dev": "env-cmd -f env.local nodemon src/server.js",

In the Search service, create a env.local with the following:

ELASTIC_URL=http://localhost
ELASTIC_PORT=9200

Run pnpm install to get everything updated
In root (WSL for windows), delete redis cache and build the other docker images

rm -rf redis-data
docker-compose --env-file ./config/env.development up --build redis elasticsearch posts traefik nginx planet

Wait for all these images to be built. Then in another terminal run pnpm start
Wait for a bit for post data to be filled up again. Head Search, cd src/api/search and run pnpm dev
The Search service will be running locally on http://localhost:4445

ElasticSearch Autocomplete Research

I did a series of reading, to understand ES Autocomplete. Not just limited to these below:

Basically there are 3 main ways to implement autocomplete with ElasticSearch

Prefix queries are ruled out cause they run during search query time and are slow. Completion suggestors don't seem to work with multiple queries (in our case, advanced search), so that is ruled out as well.

Kibana

I soon realized that it was kind of painful to change the ElasticSearch indexing in the backend, and testing out those queries with the Search service. Perhaps I wasn't doing things right, but each time I did any changes in the Backend or Search, I'd have to rebuild the Docker Images over and over again. My potato of a laptop did not like it. Sometimes the errors returned aren't even helpful at all, and I get errors with only the message ResponseError: ElasticSearch Error but no context.

After a long while I finally caved and created an account with Kibana, which is advertised as a free interface to navigate the Elastic Stack cloud.

For me, I was looking to use the console in "dev tools".

Now, I can more easily send requests and test the correct indexing and search queries. After that, hopefully, all it will take is to translate these console commands into JavaScript.

Conclusion

In all honesty, it was a frustrating weekend for me. There was a very coincidental timing of all sorts, that ES-mock had to be updated on a Friday, and just after Satellite was merged into Telescope. I think my morale had been hit a bit, cause I still believe that if it hadn't been for the merge, I would've had ElasticSearch updated to 8.1.0 in all of Telescope by now. The merge had exposed a lot of bugs that I currently don't fully understand and don't really know how to tackle. All in all, a part of the fun and frustrations of software development.