DEV Community: JFK

Migrate Azure DevOps work item queries to a new organization

JFK — Wed, 18 Dec 2019 05:07:55 +0000

Note: I originally published this article on Medium on 12/6/2019

At Blue Chip Consulting Group, we are often brought in to help companies with mergers and acquisitions. Sometimes this means moving Azure DevOps projects from one organization to another.

There are several reliable tools to help with this type of work. The Azure DevOps Migration Tools project is a decent open source option which handles several key aspects of migration very well. Unfortunately, it's not well documented and requires a deep understanding of how to configure each task. OpsHub has a nice commercial offering which we've used with great success, but it focuses mostly on the migration of work items and expects core pieces such as areas, iterations, and user accounts to be set up ahead of time.

A recent engagement had a project with dozens of custom shared queries which needed to be migrated. I tried using the DevOps Migration Tool to accomplish this, but was unsuccessful after several tries so I went to the Azure DevOps REST API documentation to see what it would take to write something myself. It turns out that the query and create payloads looked very similar, so I decided to roll my own solution.

I typically work in C#, but decided to go with Powershell because of the dynamic handling of Json payloads that it offers. It was apparent from the documentation and some sample requests I tested in Postman that a recursive enumeration of the Get result would be easy to pipe into the target DevOps instance to create the queries in the target organization.

The script requires a few items to be in place before you run it

The project name : the script assumes you're moving the queries to a project with the same name in the destination DevOps organization. If you're targeting a different instance it should be an easy tweak to the PowerShell script to introduce a different target project name
The source and target organization names : These are the custom url segments that identify the organization and follow dev.visualstudio.com in the DevOps base url. This script will work even if you are using the old url format of .visualstudio.com.
Personal Access Tokens for each organization : The source organization requires Work Item Read and the target organization required Work Item Read/Write. Here's an article that shows how to create a PAT in your DevOps instance.

You'll want to set up areas and iterations in the target organization before running this script if your queries depend on them. DevOps Migration Tools handles this well if you enable the NodeStructuresMigrationConfig processor. Doing so is as easy as using these settings for the configuration.json:

 "Processors": [
 {
   "ObjectType": "VstsSyncMigrator.Engine.Configuration.Processing.NodeStructuresMigrationConfig",
   "PrefixProjectToNodes": false,
   "Enabled": true
 },
 …
 ]

And that's it. The script can be executed multiple times. It'll skip queries with the same name and folder hierarchy location in the destination organization.

The script can be found in this GitHub repository.

Use Fiddler to live test JavaScript changes to your web site

JFK — Wed, 18 Dec 2019 04:53:47 +0000

Note: I originally published this article on Medium on 12/3/2019

There are time when I want to quickly test changes to JavaScript code running on a live web site, but going through a full publish process is too slow or cumbersome. Some development platforms such as SharePoint can make it impossible to run a local instance and iterate on a development machine.

For these scenarios, I've found that using Fiddler to serve local file content in place of a live script is a quick, easy solution. Here's how I set it up using the Lutron web site as an example. I chose Lutron because, like the situation I found myself in, they use SharePoint for their public web site.

Load up the site in your browser of choice and hit F12 to see the content scripts. Lutron has a file called sp.init.js that looks like a good candidate.

Like most sites these days, the Lutron site is serving a minimized version of this file, so we'll format it using the pretty print button

And then save a copy off to the hard drive

Before moving on, let's disable the browser cache so that a request for the file is sent with every page refresh.

Now load up Fiddler and refresh the page. Fiddler acts as a proxy and captures all the requests being sent to the server. Note that you will need to enable SSL capture in Fiddler if the traffic is encrypted. After the page loads, search (control+F) for 'init' in Fiddler to highlight the request. Click on the appropriate line to select it.

Now click the AutoResponder tab and the Add Rule button. The current request Url is copied into the Rule Editor.

As you can see, I've provided the local path to the file I saved earlier (which doesn't need to have a matching file name). I've also checked the boxes for Enable rules and Unmatched requests passthrough. By default Fiddler opts for an exact match, but in this case there is a cache buster on the Url, so I will remove that part of the Url and the EXACT specifier, and then click the Save button to update my rule.

There are a number of rule matching options for AutoResponder should you require them.

Now I'll edit the code and begin my iterative testing.

With Fiddler enabled and the browser cache disabled a refresh will serve my local file instead of the one from the Lutron site.

This technique can also be used to test changes to other web site content, such as HTML and CSS.

Demo: restrict access to Azure key vault using service endpoints

JFK — Wed, 18 Dec 2019 03:18:50 +0000

Azure Key Vault is a service which provides secure storage and management of the secrets, encryption keys, and certificates an application requires at run time. It improves upon legacy secret management approaches such as storing values in configuration or source code while providing a consistent access experience across Azure PaaS, IaaS, and serverless platforms.

In this post I'll show how to lock down access to a key vault so that these secrets can only be accessed via the Microsoft Azure backbone network. To do so, I'll restrict access at the network level using a service endpoint bound to a subnet in a virtual network, and apply firewall settings to deny Internet traffic. The provisioned Azure function will retrieve secrets at run time from the key vault via the service endpoint.

The entire solution is scripted in Powershell and can be found in the linked github repository.

Azure Function

I'll be using an Azure function hosted on a dedicated App Service plan as the client application which will be accessing the key vault. At the time of this writing service endpoint access from consumption-based functions is not available which is why I'm using an app service plan.

Like just about every other Azure service offering available, App Service plans have continued to grow and evolve over time. One of the recent additions is regional vnet integration which allows you to connect your app service plan to a non-internet routable virtual network. This is an evolution of the previous implementation of vnet connectivity which used a point-to-site VPN and virtual gateway to establish this network flow. As of this writing, both of the approaches are still supported. The new approach makes sense when your Azure assets are in the same region, whereas the old approach is required if you're trying to connect to services in another region.

Key Vault

When working with key vault in Azure you interact with two resource endpoints with distinct access planes

The data plane is where the secured data is stored. This plane can only be reached when an access policy is in place which explicitly provides the appropriate permission (read, list, update, etc.) for the specified resource (secret, key, certificate) for the identified caller. This is the plane which applications access when they require keys, secrets, and certificates.
The management plane is where you perform administrative activites on the key vault including configuration of role based access control (access to the management plane) and access policies (access to the data plane) Requests to either plane require proof of authentication by a trusted Azure Active Directory. This is a very common approach when securing access to Azure platforms and APIs, and provides for strong governance.

Our team typically provisions a vault early in the development life cycle for each application or integration we deploy to Azure. This helps us reinforce the practice of not including secure data in code which might get committed to source control.

Architecture

The Powershell script will produce the following architecture:

Script settings

The script has a few variables which you'll want to update

# used to create uniquely named assets to avoid naming collisions
$yourUniquePrefix = "myPrefix"          

# storage account name. append prefix manually due to storage naming constraints (lowercase, alpha, max 24)
$storageName = "myprefixtestpaasstorage"   

# the name value from 'az account list' for the subscription where you want to deploy the demo
$subscriptionName = "YOUR SUBSCRIPTION NAME"

The script leverages the latest Azure CLI (version 2.0.77) which introduced support for az functionapp vnet-integration add.

What the script does

This script will deploy the following assets in the East US2 region:

a resource group
a vnet and subnet
a key vault
a storage account
an app service plan (Premium!) and function app with a managed identity
a deployed Powershell function app which retrieves and returns the key vault secret

It connects the app service plan to the vnet and sets up a service endpoint for the subnet. Finally, it provides the function app's identity get rights to the key vault's secrets.

The function uses Powershell and the local managed identity endpoint to demonstrate that it is able to retrieve the secrets. As of the time that this was written, key vault configuration references are not supported with service endpoints.

The script walks through four validation tests.

First it shows that it can read the secret using the Azure CLI over the Internet with a firewall rule in place which allows the caller's IP address.
Next it invokes the function, which returns the secret in the http response payload
It then removes the firewall rule and demonstrates that it can no longer read the secret from the Internet
It makes a final call to the function app to show that it is still able to access the vault through the subnet's service endpoint

The code is heavily commented to make it easy to follow along. Enjoy!