DevTool Intro: The Algolia CLI!

Khalid Elassaad — Mon, 15 Aug 2022 19:11:00 +0000

Wake up Neo...

We need you to reset the configuration for our pharmaceutical products index! We were messing around with the highlighting settings, and now results for the query “red pills” are highlighted blue, which is really very confusing…

Also, we’re a little concerned about the possibility of all experienced reality being a simulation run by machines to keep humanity enslaved!

Please fix the index first though!

—

Oh man, don’t you all just love getting woken up before sunrise on a Monday for stuff like this? In an ideal world, this would be a quick fix. With a flourish of my black trench coat, I would roll out of bed and land in a crouch in the middle of the room, laptop in hand. I’d press enter to skip the white rabbit and ignore the knocking at my door. With one hand, I’d adjust my too-dark pince-nez glasses while the other hand fires off a command in seconds. Something short and sweet and to the point. Something like this…

algolia settings import prod_pharmaceuticals_index -F pharm_settings_snapshot.ndjson

A glance at the terminal reveals the cheerful response:

✓ Imported settings on prod_pharmaceuticals_index

Nice… Now what was that other thing about machines enslaving humanity?

—

Meet the Algolia CLI!

If you’re anything like me, the command line is your friend and trusted ally. Whether I’m testing a quick API call or coding in my favorite IDE, working with the command line keeps me iterating at a smooth clip. Additionally, I love that anything I do through a command line interface (CLI) can be easily scripted and scheduled to run however and whenever I like. With a strong set of tools in my CLI toolbelt, solving problems and automating solutions is a walk in the park.

That’s why we are so excited to announce the Public Beta launch for the brand new Algolia CLI Tool, with the full release coming soon!

Algolia CLI makes uploading an index, automating common dashboard operations, or saving and reloading snapshots of your configurations, possible right from the command line! No API client needed!

So how do I start? Don’t worry, you won’t have to visit the oracle for this one… Getting started with the Algolia CLI is a breeze!

If you’re on MacOS, simply use homebrew to install the tool by running this command in terminal:

brew install algolia/algolia-cli/algolia

Want to poke at the code and build it yourself? The Algolia CLI is fully open-source, under an MIT license, and lives in a public github repository (which you can find here)!

Releases for Linux and Windows coming soon!

Let’s take a look at some specific use cases to see how the Algolia CLI can make your life easier, so you can focus on saving the world, you star!

—

Example #0: Set your active Algolia application with profiles

Any Algolia CLI command can be called with the --admin-api-key [string] and --application-id [string] flags to specify which application to interact with. To enable you to fire off commands without having to lug around api-keys and app-ids in your clipboard all the time, you may use the following command to register a default profile.

algolia profile add --name [string] --app-id [string] --admin-api-key [string] --default

✓ Profile 'test_pharm_app' (##APP#ID##) successfully added and set as default.

The --default flag tells the CLI which app to use with future commands when no profile or API key/app ID pair are provided. Only one application can be set as default at a time. Taking time to register profiles makes future interactions fluid, especially when working with 2 or more Algolia apps.

You can also interactively provide the fields for this command by running:

algolia profile add

The tool will then prompt you for each field, one at a time, and add the application as specified.

To see a list of all added applications, simply run:

algolia profile list

NAME                  APP ID      NUMBER OF INDICES  DEFAULT
test_pharm_app        #APP#ID#1#  2                  ✓
playground_pharm_app  #APP#ID#2#  1
prod_pharm_app        #APP#ID#3#  15

Awesome! We see here that we have multiple Algolia apps added, and that the app test_pharm_app is our default!

Note: app profiles are saved in ~/.config/algolia/config.toml

Let’s upload some data!

—

Example #1: Create an index and upload records from a file

I want to create a new index, give it a relevant name, and upload the records from a file on my local system. Can the CLI handle that? Of course it can!

algolia objects import new_index_name -F ./path_to_file.ndjson

Did you notice that we didn’t specify an app ID, api key, nor app profile? In this case, the CLI will act upon the default application (which is test_pharm_app as indicated in the previous step).

We can also pass content to the CLI for upload via stdin by replacing the -F flag with -. Let’s pipe some commands together! The following command produces the same result as the previous one(read file and upload contents to specified index), but uses cat to pass the contents of the file to the CLI tool’s stdin as opposed to passing the file path as an argument. Neat!

cat ./path_to_file.ndjson | algolia objects import new_index_name -F -

What is ndjson? Newline delimited JSON is the format the Algolia CLI reads from and writes to files. This means that any command that passes ndjson formatted data as output or accepts it as input can be piped together with an Algolia CLI command! We’ll see more of this in the next example

–

Example #2: Managing index settings snapshots

So far, we’ve connected our CLI to our Algolia apps, and we’ve uploaded some data to an index! Let’s take it a step further by snapshotting an index’s settings so we can restore them to a healthy checkpoint in the future.

Which indices exist in our default app, test_pharm_app?

algolia index list

NAME               ENTRIES  SIZE    UPDATED AT  CREATED AT  LAST BUILD DURATION  PRIMARY  REPLICAS
pills_treatments   1,000    100 kB  1 day ago   1 day ago   2s                            []
pills_cures        0        0 B     2 days ago  2 days ago  3s                            []

Let’s save the settings for index pills_treatments to a file on our system.

algolia settings get pills_treatments > ./pt_settings_snapshot.ndjson

Snapshot created! Now, reverting to the snapshot is as easy as…

algolia settings import pills_treatments -F ./pt_settings_snapshot.ndjson

✓ Imported settings on pills_treatments

We can also transfer settings from one index to another in one line! Let’s copy the settings for the pills_treatments index over to the pills_cures index! In the import command, we’ll use the - value after the -F flag to tell the CLI to read input from stdin instead of a specified file.

algolia settings get pills_treatments | algolia settings import pills_cures -F -

✓ Imported settings on pills_cures

Easy! One more scenario… we’ve tested this out on all our indices in the test app, now we’re ready to migrate these settings to our production app! We’ll target the pills_treatments_prod index in the prod_pharm_app profile.

algolia settings get pills_treatments | algolia settings import pills_treatments_prod -F - -p prod_pharm_app

✓ Imported settings on pills_treatments_prod

You’re beginning to believe, aren’t you?

—

Example #3: CI/CD pipelines that execute Algolia tasks

This is where things start to get really exciting. Consider the following workflow:

A database contains your data.
An automated script runs nightly to generate an index from the data in the database.
An automated script uploads that index to Algolia. Neat! Cool! Convenient! Let’s change that. We’re gonna mess things up a bit. How about we push a bug to prod?

Engineer Anderson is tasked with performing a database migration. He creates a change!
The change is reviewed by Engineer Becky, who says LGTM!
The change is submitted!
The CI/CD pipeline applies the change to the test database. No errors! Database migration tests pass!
The change is applied to the prod database. No errors! Database migration tests pass!
Engineer Anderson signs off with a smile on his face, goes home, and helps his landlady take out the garbage. …

That night, while Engineer Anderson is sleeping peacefully and dreaming about dangling below a helicopter in flight, disaster strikes!

…

The automated index script runs at night, generating an index from the data in the database.
However, the database migration transformed fields consumed by this script.
The output index is missing an important search attribute, but uploads successfully.
Searching the malformed index produces inaccurate and irrelevant results. Oh no!
It isn’t until 3 days later, as search metrics drop, that the problem is even detected. It takes Engineer Anderson another day to roll back the changes and fix the index. What can be done about this? How do you prevent this mess from ever happening again?

What if DB changes ran a few simple query test cases during the CI/CD pipeline? Each change would immediately generate an index and fire a series of test queries against it. Engineer Anderson’s change would have failed the CI/CD pipeline for producing different query results than expected. All it would take is the creation of a handful of test cases and a simple script to run the Algolia CLI’s algolia search command against those test cases to detect this issue in advance!

Let’s configure one such test case! When the index is working properly, we can execute a test query and save the results. The saved data is our “expected results”, against which we’ll compare the results of subsequent query tests. We’ll search an index from the previous example for “blue pills”:

algolia search pills_treatments_prod -p prod_pharm_app --query "blue pills" > blue_pills_expected_results.ndjson

Cool! We have a test case for the query “blue pills”! Let’s run this on the pills_treatments index in the test app, test_pharm_app, and see if it matches.

algolia search pills_treatments -p test_pharm_app --query "blue pills" > blue_pills_actual_results.ndjson && diff blue_pills_expected_results.ndjson blue_pills_actual_results.ndjson | wc -m

We query the test index and save the results. Then, we compare the expected results with the actual results by invoking the diff command, which outputs only the difference between the two files. Finally, we pipe the output of diff to the wc -m command, which counts the number of characters in the output from diff. We observe an output of 0, meaning there is no difference between the two files, which indicates that our query produced the same results in both indices and that our query test passed!

Along those lines, a non-zero output would mean that there is some difference in the two files, and our query test failed to produce identical output across both indices.

This is just one example of how you could integrate the Algolia CLI within a CI/CD pipeline. Imagine using the index settings management workflow to promote dashboard-configured settings set by your marketing team to a production app. The promotion can then invoke a run of query testing to validate the changes, and voila! The CLI becomes a force multiplier for your business users as well!

Use the Algolia CLI to automate tasks, improve your workflows, and keep your search experiences running seamlessly, even as you change and develop the content under the hood!

—

Get Involved!

We’ve seen that the Algolia CLI can handle almost anything, from simple operations like index upload to more complex workflows like snapshot restoration. Stay tuned as we push updates out and further expand the CLI’s capabilities to make your life as an Algolia developer even easier!

Remember, the Algolia CLI is in Public Beta! On MacOS, install with Homebrew:

brew install algolia/algolia-cli/algolia
Windows and Linux installers coming soon!

Or, clone the GitHub repo and build it yourself!

Want to learn more? Check out the Official CLI Documentation for a guided tutorial!

And don’t forget to mark your calendars for September 14-15! The Algolia DevCon fast approaches. We’ll be spotlighting the CLI tool and demonstrating its capabilities! The Developer Experience team will be there to host live Q&A and answer all your questions. You won’t want to miss it!

—

Deprecation Advisory - Before this CLI was created, we had an npm Algolia CLI package that served similar functionality. The Legacy CLI is DEPRECATED - We will not be maintaining/updating/supporting it going forward. However, we will not remove the package from npm (since customers are still using it). Instead, we’ve announced the deprecation in the Legacy CLI GitHub repo (and renamed the repo to “algolia-cli-old”), and are redirecting customers to the New CLI GitHub repo.