Azure DevOps: CosmosDB & Integration Tests

Azure DevOps offers great support when it comes to project management life cycle. Wether delivery plans, tasks management, git repositories, pull requests, reviews, continuous integration, continuous delivery etc. Everything is so powerful and provides full control to ship the software with higher confidence and iterate fast.

I normally call DevOps a higher level orchestrator, It cannot solve the fast delivery and fast feedback loop. We need to setup a good base, like following coding practices, unit tests, required integration and maybe some end to end tests. Hookup all of these with Continuous Integration and Delivery pipelines.

I was working on Azure Functions which was hooked to CosmosDB. I want to write some integration tests for that. Function was coded using .NET core and binding was used to communicate with CosmosDB. Just few words about CosmosDB, which is very powerful database in terms og Geo replication and low latency. Back to the application and writing some integration tests. You can have a reserved database in Azure used only for integration tests during builds, but there are some drawbacks

1. Cleanup of data used during integration tests as tests might fail
2. Cost, as each read, write operation and storage of data will cost a bit of money Seems bit overkill for having a dedicated CosmosDB.

In order to solve this we have couple of options spin up a docker container provided by Microsoft which will expose endpoint and secret which can be passed to test runner as configurations. In order to run a container basically need a task to install from market place you can find the detail here
Guidelines above provide are designer pipeline approach of course recommended approach is yaml pipeline which is consider pipeline as code so that it can become versioned controlled as part of your code. Here is the yaml task

- task: CosmosDbEmulator@2
  displayName: 'Cosmos DB Emulator'
  inputs:
    containerName: 'azure-cosmosdb-emulator'
    enableAPI: 'SQL'
    portMapping: '8081:8081, 8901:8901,  8902:8902, 8979:8979, 10250:10250,
 10251:10251, 10252:10252, 10253:10253, 10254:10254, 10255:10255, 10256:10256, 10350:10350'
    hostDirectory: '$(Build.BinariesDirectory)\azure-cosmosdb-emulator'
    consistency: 'Session'

But there is a limitation in Azure DevOps it is only working with hosted agent vs2017-win2016. If we choose windows-latest it ends up in timeout during container spin up.

What if we want to target windows-latest hosted machine which has latest visual studio and tools, container is timingout because of some limitations, so we can't utilize windows-latest hosted agent. Also it takes nearly 13 mins just to spin up a container.

So how to execute integration tests fast and use the emulator on windows-latest hosted agent. Microsoft offers two versions of CosmosDB emulator one we have discussed in the form of docker container and other possibility is CosmosDB emulator. The emulator also comes with PowerShell module for interaction like starting a emulator etc.. Which exists by default on windows-latest, all we need is to import it and start the emulator, key and endpoint of CosmosDB emulator is always same until unless explicit mentioned to get a new key. Task to use in the yaml is:

- pwsh: |        
    Import-Module "$env:ProgramFiles\Azure Cosmos DB Emulator\PSModules\Microsoft.Azure.
CosmosDB.Emulator"
    Start-CosmosDbEmulator -NoUI -DefaultPartitionCount 100
  displayName: 'Cosmos DB Emulator'
  failOnStderr: true
  errorActionPreference: stop

Now we can target windows-latest also spin up time of emulator has been reduced from 13 minutes to nearly 4 minutes.

Now that's an encouragement to say yes to fast integration tests :)