MongoDB Shell tips and tricks

Massimiliano (Max) Marcon — Mon, 21 Mar 2022 15:04:13 +0000

mongosh is the new MongoDB Shell. Its advanced capabilities make it a great tool to work with MongoDB. In this blog, we will go through a few tips and tricks to help you get the most out of mongosh and to make it a productivity booster for your MongoDB experience.

Before we start with tips and tricks, it’s worth mentioning that mongosh is built on top of Node.js, which means that the entire Node.js API is available inside mongosh. This is a big step forward from the legacy mongo shell, where the API available to developers was a limited JavaScript subset.

Install mongosh on your computer to follow along as you read this blog. If you’d prefer a pre-configured sandbox to experiment with, you can try the Docker image I put together while writing the blog.

Configure `mongosh`

As you’d expect from a modern tool for developers, you can customize mongosh to suit your needs.

The `config` API

The simplest way to configure mongosh is to use the config API. This API exposes a config object that lets you tweak the shell UX according to your preferences. You can find all the configuration options listed in the documentation. These are the ones our users and customers use more the most:

// Adjust how deeply JSON objects returned by queries and 
// MongoDB commands are expanded.
> config.set('inspectDepth', <number>)
// mongosh tries to print only the useful bits of
// MongoDB’s error messages.
// But sometimes, you may need to understand a bit better what
// is going on, or want to dig into how things work.
// With this config option, you can enable stack traces
// for all errors.
> config.set('showStackTraces', <true|false>)
// Configure the number of items displayed per cursor
// iteration, e.g., as a result of a query.
> config.set('displayBatchSize', <number>)
// Configure the external editor to be used by mongosh.
// Read more here: https://docs.mongodb.com/mongodb-shell/reference/editor-mode/
> config.set(editor, <command>)

If you need to set default configuration options that apply to all mongosh users on your operating system, you can do that with a global configuration file starting with mongosh 1.22.

`.mongoshrc.js`

For more advanced configuration, use JavaScript to add custom functions and shell helpers to your .mongoshrc.js configuration file.

For example, in my .mongoshrc.js, I load JS files that contain query/aggregation helpers I use for my daily work analyzing data stored in MongoDB.

One helpful function I use very often is giveMeADate(), which makes it simpler to generate date objects to pass into my queries:

function giveMeADate(year, month, day = 1) {
    month = (typeof month === 'number' && month < 10) ? `0${month}` : month;
    day = (typeof day === 'number' && day < 10) ? `0${day}` : day;
    return ISODate(`${year}-${month}-${day}T00:00:00.000Z`);
}

You can create libraries of helper functions structured in the way that makes the most sense to you and then load() or require() them in your .mongoshrc.js file.

Customize your prompt

I like colorful and informational prompts, which is why I am a happy oh-my-zsh user. When I use mongosh to work with my data, I want a quick overview of my connection status and information about the cluster I am connected to.

That’s why my prompt looks like this.

It says that:

Green cloud: I am connected to MongoDB Atlas, our cloud database.
Little 1: The cluster has a primary so it’s all good and healthy.
My read preference is secondary.
Chart icon: My queries will run against an analytics node and therefore won’t affect the normal cluster operations. If you want to learn more about this, check out the MongoDB documentation about workload isolation.
My database is “test.”

How do you set up your prompt to look like this? Easy: Just include a global function named “prompt” in your .mongoshrc.js file and make it return the information you need in the format you prefer. The prompt in the screenshot above is generated with this code.

And if you like pretty icons as much as I do, configure your terminal application to use Nerd Fonts.

Automatically time out sessions

When I connect to my production clusters, I often worry about leaving a shell session open accidentally. For security reasons, or due to the security policies and requirements of your organization, you may want your shell sessions to time out when they are not used for some time.

With mongosh, that is quite easy to do. You can add this behavior just with a few lines of JavaScript in your .mongoshrc.js. A few lines of code can automatically call exit() after a certain amount of inactivity or reset a countdown timer whenever a prompt is generated.

In my prompt customization script, you can see how that is done. The prompt even includes a nice visual hint (the stopwatch on the right) to show that the session will time out.

With this simple approach, a shell session could end even if a query is running. Therefore, I included a stayLoggedIn() helper function that disables the timeout for the current session.

Write query results to a file

It’s common for users to need to save the result of a query or a diagnostic command in a file. In mongosh, that is really easy to do with the standard Node.js API:

sample_mflix> const thrillers = db.movies.find({genres: 'Thriller'})
sample_mflix> fs.writeFileSync('thrillers.json', EJSON.stringify(thrillers.toArray(), null, 2))

The code above queries the "movies" collection to find all the thrillers and then writes them out as Extended JSON into a file named thrillers.json in the current working directory.

Fetch data from an API and store it in MongoDB

Sometimes, you’ll need to fetch data from an HTTP endpoint in JSON and store it in MongoDB. Before mongosh, you had to script this operation with your favorite programming language. Now, you can do it all inside the MongoDB Shell. Because the entire Node.js API is available to you, including all the HTTP functionality, you can write a script that fetches the data and then writes it into MongoDB with an insertMany() or with the Bulk API.

It’s even easier to do this if you rely on one of the NPM packages that were built to make HTTP more convenient. In the example below, I am using node-fetch. To follow this example, you will need to have Node.js and npm installed on your computer.

First of all, create a fetch-and-store.js file with the following content:

const fetch = require('node-fetch');
use('opencollective');
async function getOpenCollectiveData() {
  const data = await fetch('https://opencollective.com/webpack/members/organizations.json');
  const members = await data.json();
  db.members.insertMany(members);
}
getOpenCollectiveData();

This simple script fetches all the Open Collective members and their information and stores them into a collection named “members” in the “opencollective” database.
To run the script, we need to make sure node-fetch is installed. The command below installs node-fetch in the current directory, which is enough for mongosh to find the package. You can also install it globally to have it always available no matter what your working directory is.

$ npm install node-fetch@2

Now we can run the script:

$ mongosh <connection string> fetch-and-store.js

Once the script has finished, the data pulled from the Open Collective endpoint will be stored in MongoDB. You can see that from the Compass screenshot below.

Generate large volumes of synthetic data

Sometimes, you just want to generate large volumes of synthetic data for development and testing. One great module that helps with this is Falso.

Similarly to what we did with node-fetch above, we’ll install it:

$ npm install @ngneat/falso

And then we’ll create fake-people.js with the following content:

const falso = require('@ngneat/falso');
const people = [];
for (let i = 0; i < 1000; i++) {
  people.push({
    firstName: falso.randFirstName(),
    lastName: falso.randLastName(),
    avatar: falso.randAvatar(),
    phoneNumber: falso.randPhoneNumber(),
    email: falso.randEmail()
  });
}
use('fake-data');
db.people.insertMany(people);

When the script has run, there will be a “people” collection in a “fake-data” database with 1000 fake people’s data inside.

Analyze the schema of your collection

As the amount of data that you store in MongoDB grows and you have multiple applications reading and writing into your clusters, it might be useful to take a look at your collection and check that the schema is what you expect and is as homogeneous as possible across your documents.

This is something you could probably write a script for. The good news is that we have already scripted it for you, in the form of a shell snippet. Snippets are scripts that are packaged and stored in a registry to facilitate sharing and reuse. We have one dedicated to schema analysis.

Using it is extremely simple:

sample_mflix> snippet install analyze-schema
sample_mflix> schema(db.<collection>)

Below you can see an example of what that looks like and how you can use it to spot outliers in your data.

Conclusion

I hope this list of tips and tricks helped you discover how to get the most out of the MongoDB Shell. Do you have other mongosh customizations, scripts, and tweaks that you’d like to share with us and the community? Let us know in our community forum.

And if you have suggestions for how to improve the experience with mongosh, feel free to submit ideas in our feedback portal.

DEV Community: Massimiliano (Max) Marcon