DEV Community

loading...
The DEV Team

Algolia is OUT! Elasticsearch is IN!

molly profile image Molly Struve (she/her) ・1 min read

With the merging of the below PR, DEV no longer requires Algolia to run!

Remove ALL Algolia Code #7577

What type of PR is this? (check all applicable)

  • [x] Refactor
  • [x] Optimization
  • [x] Documentation Update

Description

Remove all traces of Algolia from our codebase and add documentation for Elasticsearch. The Elasticsearch docs will likely need to be worked on as we go but I gave us what I thought was a good foundation.

Once this is out we can remove Algolia from Heroku!

Related Tickets & Documents

https://github.com/thepracticaldev/dev.to/projects/6#card-32955489 https://github.com/thepracticaldev/dev.to/projects/6#card-32912217

Added tests?

  • [x] no, because they aren't needed

Added to documentation?

  • [x] YES!

alt_text

This is a huge step forward for our community because it means that those contributing no longer have to sign up for an Algolia account. Instead, contributors can simply download the open-source software from Elastic when they are setting up their development environments.

For more information on how DEV uses Elasticsearch, checkout our new docs page.

Happy coding!

Discussion (16)

pic
Editor guide
Collapse
ben profile image
Ben Halpern

It's really exciting to be on an open technology, generally vendor agnostic, and installable on our local machines.

I chose Algolia years ago for its remarkable performance, ease of use, and quality of client libraries. Those things are all true so I don't want anyone to see this as too black-and-white. Algolia is a fabulous tool. But as we need to evolve, this is. a big stepβ€”of course, for developer experience, but also giving us more tools to work with in building a tremendously powerful tool together.

Collapse
andrzejwp profile image
andrzejwp

Thanks for this comment Ben. We made a similar choice for Flotiq, unlike most other headless CMSes we decided to incorporate ES into our API, so that developers can use its powers.

I would really love to learn a bit more about the motivation, what were the particular features you guys thought are most important and why.

Collapse
rafi993 profile image
Rafi

🎊🎊🎊🎊🎊🎊 Awesome.

Collapse
glennmen profile image
Glenn Carremans

Great news!
Also wondering what was the reasoning for choosing Elasticsearch, I have recently learned about MeiliSearch and it looks very promising.

Collapse
molly profile image
Molly Struve (she/her) Author

Elasticsearch is a tried and true technology that has been around for a while and has a solid Ruby library. It is also a database that many of the major platforms like AWS and GCP support and host which allows greater flexibility when users decide they want to spin up their own communities.

It is also something I have worked extensively on in the past and that added experience was what allowed us to make this transition very quickly bc I knew exactly what needed to happen and how to do it. We considered other tools such as the new Redis search but the fact that it was so young and unfamiliar made us decide to stick with what we knew.

Collapse
somedood profile image
Basti Ortiz (Some Dood) • Edited

Hooray! πŸ₯³πŸ₯³πŸ₯³

Developer experience for the win!

Collapse
almenon profile image
Almenon

We've had a difficult time trying to keep Elasticsearch working correctly on our EC2 instance. It takes a lot of work to get it configured right, especially if things break.

If you don't mind me asking, what are you using to host Elasticsearch, and why?

Collapse
molly profile image
Molly Struve (she/her) Author

Right now we are using Elastic's hosted service. At my prior job our cluster was self-hosted on AWS. I could see us possibly going that route again so we have more control over our cluster.

If you are going to self-host I highly recommend going through an Elasticsearch training course beforehand. That will layout a lot of gotchas and can be super helpful to get you up and running.

Collapse
almenon profile image
Almenon

That's good advice, thanks.

Collapse
cescquintero profile image
Francisco Quintero πŸ‡¨πŸ‡΄ • Edited

Great work, Molly :)

Achieving great stuff in no time. πŸ…

Collapse
krusenas profile image
Karolis

how much effort does it require to maintain in general? I have recently added algolia search to my web and while it works I do feel like I am overpaying for what it's doing :)
Could it just run on a single container in kubernetes with a mounted volume? I wouldn't care about the persistence much as I could rebuild the index every day.
Do you need to perform some daily/monthly cleanups on it?

Collapse
molly profile image
Molly Struve (she/her) Author

It depends on how you use it but we dont do any maintenance on our index. The code handles adding and removing documents as needed to ensure it stays in sync with what is in our database.

Having a small ES cluster is easy to maintain. I like to say Elasticsearch is easy to learn and hard to master. The time you really have to know what your doing is if you are going to seriously scale it. Depending on document size a few hundred million docs is pretty easy to handle. Beyond that you want to know what you are doing.

If you do want to eventually scale checkout my scaling blog posts: dev.to/molly_struve/scaling-elasti...

Collapse
tinybat02 profile image
Tiny_bat

Have you ever considered Apache Solr before adopting Elasticsearch? They are both built on the same search engine, and are capable of doing almost the same thing.

Collapse
molly profile image
Molly Struve (she/her) Author

We definitely considered multiple technologies but in the end, given our use case and because of my expertise with Elasticsearch we decided it would be the best fit for us. We could ramp up quickly with it and then scale well into the future.