We've selected our favorite tips and tricks created by Michael Crump and are delivering fresh technical content on Azure all April! Miss a day (or more)? Catch up with the series.
Don't have Azure? Grab a free subscription.
If you have built an application and are currently storing the data in a static JSON file, you may want to consider the MongoDB API for Microsoft Azure's Cosmos DB. You will have the document data storage you require for your application with the full management of Microsoft Azure with Cosmos DB along with the ability to scale out globally. This will permit you to create replication to regions where your customers are.
Today we'll take an existing database from an application and use some simple native MongoDB tools to teach you how to easily move some data over to Cosmos DB.
Required:
- Azure Account
- Azure Cosmos DB for MongoDB API
- Example Database JSON File
- Git
mongoimportfor your local shell
This example will focus on using a bash terminal which is possible on Mac OS X, Linux and Windows using Windows Subsystem for Linux.
To get started, the JSON file that's provided in the repository is a pretty simple dummy data DB created using json-generator.com.
You can create your own if you like, but to make it simple this tutorial provides a 1052 entry example document database to import into Cosmos DB.
Go to the Azure Portal and create an Azure Cosmos DB API for MongoDB deployment for our data.
It's very easy to get started by just typing "Cosmos Azure DB" in the search bar, you can then click on the selection that comes up under "Services."
Now click Add on the upper left and you'll be brought to the blade where you can begin selecting your new Cosmos DB account. There you can select a subscription you have linked to your account and select a resource group or create a new one. An account name will represent what hostname will be added to the fully qualified domain name created as part of the endpoint you'll connect your applications to. In this example a new resource group is created along with selecting the Azure Cosmos DB for MongoDB API and a region the data will be hosted in.
For this example we won't enable either Geo-Redundancy or Multi-Region writes, these are great ways to expand your data's availability but aren't necessary for a demo.
Click next and move on to the Network section, in this section, we'll need to create a virtual network and then permit ourselves to connect to our API endpoint.
We begin by creating a new VNET and selecting the 10.2.0.0/16 network. You will create a default subnet for our tutorial. In more complex solutions, you may want to create the subnet for this to be private only to further restrict your data's exposure. For now this example will use a single 10.2.0.0/24 subnet. Make sure you permit your IP so that you are able to access the endpoint created.
Click Review and Create your new Cosmos DB deployment and validate the settings. Now click Create and wait for your new deployment to be created.
Now let's make sure you've cloned the repository with your database example file:
git clone git@github.com:jaydestro/devtocosmodb.git
Now enter the directory and ensure you can start using the mongo utilities we need:
cd devtocosmodb
mongoimport --version
You should get some form of output like depending on what OS you are using for this tutorial:
mongoimport version: 4.0.2
git version: homebrew
Go version: go1.11
os: darwin
arch: amd64
compiler: gc
OpenSSL version: OpenSSL 1.0.2r 26 Feb 2019
By now the Cosmos DB deployment should be ready to go and get the connection string you need in order to import the data.
Click go to resource and find the Cosmos DB portal for your new database where we can begin storing collections of documents we can query using the MongoDB API.
Click Quickstart and find the "Others" section, this will provide you with the login information required to use mongoimport and import your json.
Now import example-docs.json file to Cosmos DB using mongoimport :
mongoimport -h exampledevto.documents.azure.com:10255 \
-d exampledevto -c example -u exampledevto \
-p YOURPASSWORDHERE \
--ssl --jsonArray --file example-docs.json
An expected output should look something like this:
2019-04-10T13:06:08.163-0400 connected to: exampledevto.documents.azure.com:10255
2019-04-10T13:06:11.051-0400 [##############..........] exampledevto.example 1.36MB/2.25MB (60.5%)
2019-04-10T13:06:14.155-0400 [########################] exampledevto.example 2.25MB/2.25MB (100.0%)
2019-04-10T13:06:14.155-0400 imported 1051 documents
Now let's look at the new collection and documents in the Data Explorer within the portal in Cosmos DB:
Now go to the "Connection String" section to start connecting this data to an application and select your favorite language such as JavaScript or Golang to query and manipulate the data.
We'll be posting articles every day in April, so stay tuned or jump ahead and check out more tips and tricks now.
Top comments (5)
I was actually pretty keen to try CosmosDB for a major project of mine though it turns out that it isn't fully MongoDB compatible. While this particular command might not be called very often, it turns out you can't rename collections which unfortunately is part of the migration process for Hangfire, a piece of software I can't live without. There is a UserVoice item for it but it isn't considered in part of its roadmap.
Besides not being able to use Hangfire MongoDB with CosmosDB, I am somewhat concerned this isn't the only thing that is missing. Given I am using libraries on my end to abstract the raw Wire Protocol calls, I wouldn't even know if a specific command will or won't work till I run it. Not really a good place to be if I want to run a production environment on it with MongoDB.
I think it's reasonable to say that you can consider using the MongoDB product like Atlas on Azure if Cosmos DB doesn't fit your needs.
Best of luck!
Oh yeah, most definitely. Atlas on Azure is what I decided to go with instead.
Can you comment on how to deal with RU limit errors? I am just trying to import 250 records using mongoimport and there is no setting for scale that will allow me to do this. What am I missing?
{Message: {"Errors":["Request rate is large. More Request Units may be needed, so no changes were made. Please retry this request later. Learn more: aka.ms/cosmosdb-error-429"]}
Feel free to ask questions or reach out.