DEV Community: tswiftma

Trigger an automation pipeline after a Dev pipeline runs

tswiftma — Wed, 21 Dec 2022 20:39:19 +0000

In the perfect world you'd have multiple development environments, Dev/QA/Stage/Prod, especially if you have multiple microservices being developed concurrently. But in reality you might only have one Dev environment. So when do you run all of your automation integration tests? You might run them nightly on Dev or only on Stage after a release point. There are pros and cons to all these methods. Missing API regressions could be a very big con.

Another option is to trigger your service-specific API tests in an integration pipeline right after the corresponding Azure Devops Dev pipeline for the actual service runs. This is pretty easy to do in Azure DevOps YML. In my test integration pipeline under "resources:" I added a "pipelines:" section.

The options "pipeline", "source", "project" and "trigger" are simple and explained in this Azure DevOps article. Basically your integration test pipeline is triggered right after the Dev pipeline. In this example the Dev pipeline is named "pin-app-payor-job-service". Below is the trigger result in ADO Pipelines

My team writes failed test runs to a Teams channel so everyone can see them. I have DEV another article on how to do that here.

Make sure your automation tests are scoped to the service under test. Running "Every test under the sun" is annoying and generates too much noise. We're currently using Specflow with XUnit so we tag our Scenarios according to the API that it tests.

Then when we run tests in our pipeline yml with the "dotnet test" command we pass in the specific tag as a --filter Category for XUnit.

Hope this article helps you run your automation at the right time!

Getting an Azure OAuth2 token with a Service Principal

tswiftma — Wed, 16 Nov 2022 17:25:20 +0000

Trying to call the Azure REST APIs requires an OAuth2 token. This seems simple right? Ha ha unfortunately it's not.

There are a number of authentication methods to get a token using the AzureDefaultCredential class. Out of these "fall through" authentication options for local development I couldn't get the VisualStudioCredential option to work for me. This is the option to use your user credentials from Visual Studio. Seems like my company has some sort of difference between my local AD login and my Azure login. I was able to use the Azure CLI login for local development by logging in via command line but that doesn't help me when the tests are running in Azure Devops. Tests running in Azure Devops can also use the ManagedIdentityCredential to get a token but I hadn't gotten to that stage yet and I still needed to get an Azure OAuth2 token for local development.

So I chose to try using an Azure Service Principal because it can be used when developing locally. This requires setting up an App Registration in Azure AD. This Azure article is very good on how to setup the App Registration. Note that setting up the App Registration and assigning the Service Principal to your Application Role requires sufficient permissions. In my case I was a Subscription Owner.

Note that while setting up your App Registration you need to save the AzureSubscriptionId, ClientId, and ClientSecret(value) to use when requesting the OAuth2 token. Here's a method to get the token:

public async Task<string> GetAzureOAuthToken()
        {
            var requestUrl = AuthEndpoint + "/" + AzureSubscriptionId + "/oauth2/token";
            var client = CreateHttpClient();
            var values = new List<KeyValuePair<string, string>>();
            values.Add(new KeyValuePair<string, string>("client_id", ClientId));
            values.Add(new KeyValuePair<string, string>("client_secret", ClientSecret));
            values.Add(new KeyValuePair<string, string>("grant_type", GrantType));
            values.Add(new KeyValuePair<string, string>("resource", Resource));

            var requestBody = new FormUrlEncodedContent(values);
            var response = await client.PostAsync(requestUrl, requestBody);
            Assert.Equal(HttpStatusCode.OK, response.StatusCode);

            string responseStr = await response.Content.ReadAsStringAsync();
            var content = JsonConvert.DeserializeObject<AzureADTokenResponse>(responseStr);

            return content.access_token;
        }

Other parameters needed for the above code:

AuthEndpoint = "https://login.microsoftonline.com"
grant_type = "client_credentials"
resource = "https://management.core.windows.net"

The object AzureADTokenResponse in the code above is a C# class I created to deserialize the response from the request. Below is the code for this class.

public class AzureADTokenResponse
    {
        public string? token_type { get; set; }

        public string? expires_in { get; set; }

        public string? ext_expires_in { get; set; }

        public string? expires_on { get; set; }

        public string? not_before { get; set; }

        public string? resource { get; set; }

        public string? access_token { get; set; }
    }

Now if you deserialize the response without errors than grabbing the token is easy:

return content.access_token;

Ok you've got your Azure OAuth2 token. Now you have to try and use it to call Azure REST APIs. I'll have another post on how to call Azure Logic App APIs soon :)

Write Pipeline Failures to a Teams Channel

tswiftma — Mon, 29 Aug 2022 15:18:00 +0000

It can be super useful to write Azure DevOps pipeline failure events to a Teams Channel. It’s a easy way for the whole development team to know when something has gone wrong.

In the below example a failed automation pipeline run sends a notification to the an Automation Teams Channel that I created. The notification contains a link to the failed pipeline job in Azure DevOps.

Setup:

1) Select or create a Teams Channel to write failures to. I recommend creating a new dedicated channel for this to reduce noise from other notifications.

2) Select the Channel’s … menu then select “Connectors”

3) On the Connectors page select “Incoming Webhook” then select the Configure button

4) Give the Webhook a name and select the Create button

5) A Webhook URL will be created. Copy and save this URL then select the Done button. You will use this URL in your pipeline yml.

6) Add a step to your yml pipeline to write test failures to the Teams Channel. The step should run after your test execution.

In the pipeline yml create a new variable for your Team Channel Url using the Url value you copied above:



– name: TeamChannelUrl
value: 'https://nuancecommunications.webhook.office.com/webhookb2/2fe27450-5aff-4d80-9061-184ae51d90f2@29208c38-8fc5-4a03-89e2-9b6e8e4b388b/IncomingWebhook/6b8bfe1977214326ab198271cbf5fc93/b50aa873-d730-42b2-b5b2-26xa43abea7d'

After adding the variable in the pipeline yml add a new Powershell stage that writes failures to the Teams Channel. The step should be added after your automation tests (or pipeline) has been executed.

Below is the Powershell code you need that will grab the Azure Devops build information and send it to the Teams Channel. Update the title to reflect the automation/pipeline that was executed. In the below example I pass in the test_environment and test_category parameters that I am using in pipeline to further identify what failed.



– powershell: |
$body = [PSCustomObject][Ordered]@{
"@type"      = "MessageCard"
"@context"   = "http://schema.org/extensions"
"themeColor" = 'd72f00'
"title"      = "HAL Automation Pipeline Failure: Environment: ${{ parameters.test_environment }} Tests: ${{ parameters.test_category }} "
"text"       = "$(Build.DefinitionName)"
}
$body = ConvertTo-Json $body -Depth 100
$uri = "$(TeamChannelUrl)"
Invoke-RestMethod -uri $uri -Method Post -body $body -ContentType 'application/json'
displayName: Send Teams Notification
condition: failed()

Test your pipeline changes by running it. If you’re having trouble generating an automation or pipeline failure you can set the condition value above to condition: always( ). Just remember to put it back to condition: failed( )!

7) Link to pipeline failures. Now on a pipeline failure a link will be generated in the new Teams Channel entry to the failed pipeline job. Anyone can click it and see the pipeline failure. Now time to start debugging!

Software Engineering is a Total Trip

tswiftma — Fri, 17 Jun 2022 22:10:56 +0000

Software engineering can be really annoying at times but it is also hilarious and super intellectually challenging. I just switched Dev groups in my company. I worked on an internal Data Deletion GDPR service for almost 2.5 years. The project went nowhere and nobody actually wanted it. Requirements were vague. Devs and PMs left the project left and right. Yet the project lives on because some VP wants it to. IMHO, it is a huge waste of valuable developer time.

But despite all of that I loved the daily hilarity of working on a doomed project with so many outstanding Devs on it. From a technical standpoint I wrote the most complex C# API automation code ever. For example, automating APIs that accept two totally different types of JWTs and having to update a database during tests to accelerate background processes. Great challenging stuff. Bittersweet describes my feelings leaving a team of great people and even this project. Even though being a Dev is frustrating at times it's also awesome. We are blessed to have a career that is challenging, constantly amusing, and financially rewarding.

Generate a JSON list for a request body payload in C# using Collection Initializers

tswiftma — Thu, 31 Mar 2022 18:06:36 +0000

I recently had to write a test for a new JSON REST POST API. The request body of the POST contained a formatted JSON list. Typically the body of various REST calls are JSON formatted between two brackets { json }. But in this case I needed list brackets with the JSON [{ json }] for the body.

It's easy enough to create a C# class such that an object from it represents the JSON data between the brackets { }. In the below example the class is called SettingValue.

But how in C# do you create a initialized list of this JSON that can then be used in the body of the POST? You first create an object settingValue of type SettingValue. This is our { json }. Then you create a List object settingValueList of type SettingValue.

But what is the { settingsValue } at the end of the List instantiation? It is a Collection Initializer for the List with settingValue as the Element Initializer. Only objects of type SettingValue can be added to the list and the list is now created with default settingValue JSON. If you don't use an Element Initializer, the List will not have the default value we need.

Now you can update or add to the list and pass settingValueList to use as the body for your JSON POST. So your settingValueList list might end up looking something like this:

The solution seems simple to me now but for some reason in years of API automation I never have had to do it before. Previously I had only seen nested JSON Lists [ ] within JSON brackets { } and nesting lists in JSON can be done using nested C# classes, which would be the subject of another blog post :)

PS: Thanks to Dan Post and Bob Geary for a technical review of the post

WebdriverIO Automation Protocol State of Confusion

tswiftma — Thu, 30 Sep 2021 21:58:13 +0000

I'm still somewhat of a WebdriverIO noob and I learned an important lesson this week. WebdriverIO supports two different automation protocols, Webdriver and Devtools and they can behave differently! Here's a link to the WebdriverIO doc for more details.

1) Using Webdriver. Webdriver is the older Selenium protocol and uses a middle man browser specific driver to drive the browser. It's very good for cross browser testing support.

How to use it in WebdriverIO: If you specify to use a service that updates browser drivers in your wdio.conf.js file you are most likely using Webdriver. For example:

  // Test runner services
  // Services take over a specific job you don't want to take care of. They enhance
  // your test setup with almost no effort. Unlike plugins, they don't add new
  // commands. Instead, they hook themselves up into the test process.

  services: ['chromedriver', 'edgedriver'],

These services are all added via NPM packages. 'Selenium-Standalone' is another driver service.

2) Using Devtools: Devtools was developed with Chrome as an API to drive a browser directly without a specific driver. It also uses Puppeteer to drive the browser.

How to use it in WebdriverIO: If you don't specify to use a service in wdio.conf.js than you will default to using Devtools. Yes this confused the heck out of me because I was using the chromedriver service (aka Webdriver) for running Chrome and just specifying 'MicrosoftEdge' for a browser Capability which defaulted to using Devtools. If you look at the runtime logs you can see what WebdriverIO is using:

So what protocol do you want to use? Well I found out that Devtools doesn't support all WebdriverIO commands including browser.maximizeWindow( ). To keep the peace on my automation team I used Webdriver for all browsers. Also Devtools might not have the best cross browser support. But it does seem to be the future of browser automation and supports some pretty advanced dev features including network inspection so it might be worth a revisit in the future now that I'm less confused :)

WebdriverIO DevTools Edge driver options

tswiftma — Wed, 29 Sep 2021 18:03:06 +0000

I recently needed to add Microsoft Edge browser coverage to my Webdriver.IO tests. But when I added 'edge' (or 'MicrosoftEdge') as the browserName in the wdio.config.js capabilities, a couple of problems occurred:

1) browser.maximizeWindow( ) in my test project stopped working with the following error "Error: Command "maximizeWindow" is not yet implemented". It still works for chrome however. I found that quite a few people were hitting this problem but there really wasn't a solution available. I think that the problem lies with the actual MicrosoftEdge driver itself. So as a workaround I now always start the browsers in wdio.config.js with the args option '--start-maximized'. I also stopped using the call browser.maximizeWindow() altogether.

2) When the Edge browser came up it only used a defaultViewport of 1200x800 even if the browser was maximized. This was weird because my web app would only use part of the browser display. Looking at the WebdriverIO run logs showed that devtools:puppeteer was starting Edge with the option "defaultViewport":{"width":1200,"height":900}. I have no idea why it would default to this! The workaround was to set a 'wdio:devtoolsOptions' value for defaultViewport to null.

Below is the code to add to wdio.conf.js. Hope this is helpful in getting Microsoft Edge to run with WebdriverIO!

    capabilities: [
      // more capabilities defined here
      // or override capabilities from wdio.conf.js      
      {
        maxInstances: 1,
        browserName: 'edge',
        'ms:edgeOptions': {
          args: ['--start-maximized']
        },
        acceptInsecureCerts: true,
        //
        // Edge launches via devtools/puppeteer.
        // Need to set defaultViewport to null to use the whole browser
        'wdio:devtoolsOptions': {
          defaultViewport: null
        }
      }
    ],

Store your automation credentials in Azure KeyVault

tswiftma — Fri, 24 Sep 2021 18:47:08 +0000

Note: updated on 4/30/24 with a more complete solution

There's a common problem storing login credentials for automation that runs in Production or near-Production (aka Stage), where do you store them? You absolutely shouldn't check them into source code! I guess that you could encrypt them somehow but a better solution is to store them in Azure KeyVault and pull them out when you need them. So how can you do this?

Let's consider the two locations where automation will run:

1) In an IDE like Visual Studio (aka local) when developing tests.
2) In a pipeline like Azure DevOps to run tests on a schedule.

They both represent different authentication challenges. First let's tackle the local Visual Studio scenario.

In Azure create a new KeyVault for your test credentials and create a couple of secrets that represent a username and password. The secrets could also represent a ClientId and ClientSecret if you're creating tokens.

To access the KeyVault we will use the DefaultAzureCredential class. This class provides an authentication workflow and tries different authentication methods automatically. In our case the class will use VisualStudioCredential to access the KeyVault from our local machine. To enable this option you need to sign in to Visual Studio with your Azure account at Tools/Options/Azure Service Authentication/Account Selection:

To get the KeyVault values you need to add the following packages to your code to support using DefaultAzureCredential:

using Azure.Identity;
using Azure.Security.KeyVault.Secrets;

Here's some sample code to get the KeyVault data. We check if our tests are running locally using a bool to get the VS edition. Then if we are running local we create a new SecretClient using DefaultAzureCredential and we query the KeyVault for the secrets we're looking for.



// Check if automation is running in VS or from cmd line (aka pipeline)
var isRunningInVS = !string.IsNullOrEmpty(Environment.GetEnvironmentVariable("VisualStudioEdition"))

// Get KeyVault Uri and credential for local run
string kvUri = "https://myKeyVault.vault.azure.net";
var clientKV = new SecretClient(new Uri(kvUri), new DefaultAzureCredential());

var values = new List<KeyValuePair<string, string>>();
values.Add(new KeyValuePair<string, string>("grant_type", _GrantType));                

// Get API Credentials
if (authMode == AuthMode.Candidates)
{
    if (isRunningInVS)
    {
        clientKV = new SecretClient(new Uri(kvUri), new DefaultAzureCredential());
        _ClientId = (await clientKV.GetSecretAsync(ClientIdName)).Value.Value;
        _ClientSecret = (await clientKV.GetSecretAsync(ClientSecretName)).Value.Value;
    }

    values.Add(new KeyValuePair<string, string>("client_id", _ClientId));
    values.Add(new KeyValuePair<string, string>("client_secret", _ClientSecret));
    values.Add(new KeyValuePair<string, string>("audience", _AudiencePayor));
}

You now can securely pull data from KeyVault when running tests locally.

The second challenge is how does automation pull credential data when the same tests are running in a pipeline? In the case of using Azure DevOps you can store your credentials securely in Azure Library Variable Groups.

Create a Library Variable Group specific to your environment. The below example shows one for Development. You can then add your credential values and mask out any critical values such as ClientSecret:

Note that Library Variable Groups do have an option to directly link to your KeyVault but at this time I didn't have permissions to get it to work correctly.

Next is to access the Variable Group values in your pipeline yaml. Under the variables section, specify the group value as the name of your ADO Library Variable Group. When running the pipeline it will have access to the data in the variable group.

Note in the above example $(AuthorizationClientId) is the value that is pulled from the variable group and assigned to the yaml variable named Authorization.ClientId". You now have secure access to your credentials in your pipeline.

There's a few other pipeline steps that I won't go over in detail but here's a few tips:

Tip1: All of your variables for your automation should be stored in an appSettings.json file that is typically read during startup using the ConfigurationBuilder class. Remember not to store credentials in these files!

Tip2: At pipeline runtime you can substitute variables like we just created from the Library Variable Groups above using the FileTransform@1 task.

Good luck and let me know if you have any questions!

Help I need to choose a UI Automation tool!

tswiftma — Thu, 08 Apr 2021 19:09:07 +0000

Our team has the opportunity to choose a new UI Automation tool for a new web front end. We're considering Python/Selenium, WebdriverIO, Test Café or Cypress. Here's what we need to support for technology:

The user interface is developed on Node.js with React.
Need some support for iframes automation as some legacy apps will be plugged in.
Need the ability to pass in test script parameters including Host Header support for K8S and so we can run tests in CI/CD pipelines.
An open source tool with an active user community would be nice! We might be able to convince the company to pay for a tool that is totally awesome :)

We've done some basic automation POCs (Proof of Concepts) with Python/Selenium and WebdriverIO. Our automation people have some good coding skills in Java/C# but more limited skills in Python/Javascript.

What do you think? Thanks!

Using cmd switches with pytest to share data between tests

tswiftma — Thu, 25 Mar 2021 15:52:29 +0000

I'm kind of new to using Python Selenium with pytest and I needed to pass in come custom cmd line options when running tests with pytest. Specifically I needed the ability to pass in a custom url, hostheader, email and password value to my tests so they could run in different environments. I would think that everyone needs this for CI/CD tests today!

After lots of Googling for solutions I found that the suggestions on how to use switches were all over the place! I finally figured out to use the conftest.py file with the pytest_addoption method. Now there seems to be a lot of discussion on where to place the conftest.py file. I placed it above my \Tests folder, I haven't figured out the other complexities of it yet. Here's the content of conftest.py:

Note that in the pytest_addoption(parser) function I set default values for the switches that are for my most common test environment. The params(request): function is the key to returning an array of the params to the tests.

So how do you access the data in your tests? Really easy! Just pass params into your test functions and reference the params using params['MyParam']

Running the tests:

I can run my tests using only the pytest command because it has default values for the command line switches or I can substitute values by adding the actual switches themselves. Now you can run the tests anywhere in your CI/CD environment by substituting environment vars for the command line switches :)

Why I'm switching from C# to Python with Selenium

tswiftma — Thu, 18 Mar 2021 15:17:51 +0000

I was recently tasked with automating our product's new login wrapper. This involved hitting the server UI page, redirecting to a login page and validating if the login succeeds or fails. Another requirement was to have the ability to run the tests in Azure DevOps pipelines.

I'm typically not a UI automation guy, I usually focus on Rest APIs and code in C#. So I started with Selenium C#. While it was fairly easy to automate the login steps I was immediately blocked when I found out that I need to support host headers to run the tests in a Kubernetes (K8S) environment. Well Selenium out of the box doesn't support host headers...

I stared searching for possible solutions and the best one I found by far was Seleniumwire. Seleniumwire supports access to underlying requests/responses. You can modify them and get http response codes as well. I felt like I was back in my API testing world! The only difference was that Seleniumwire only supports PYTHON and not C#. But I had to have the Seleniumwire functionality even if my Python skills were somewhat noob-ish.

Lets look how easy it is to set a host header with Seleniumwire

import pytest
from seleniumwire import webdriver

def test_isAlive(url, hostheader):
    web_driver = webdriver.Chrome()
    web_driver.maximize_window()
    # check to set host header    
    if hostheader != 'none':
            web_driver.header_overrides = {'Host' : hostheader}

    isAliveUrl = '/api/isAlive'
    UIEndpoint = url
    loginURL = UIEndpoint + isAliveUrl
    web_driver.get(loginURL)

Now the above isn't the whole script but you get the picture. This solved a huge issue that wasn't possible with Selenium C#. Note that I check for a host header value of "none" if I don't want to use a host header.

Another cool Seleniumwire feature is to grab an http response code which I can then use in an assert to check that the page loaded. This can be an additional test to the builtin Selenium page load response check.

# Access requests via the `requests` attribute
    for request in web_driver.requests:
        if request.response:
            print(
                request.url,
                request.response.status_code,
                request.headers                
            )    

    assert request.response.status_code == 200

These Seleniumwire features might seem minor but they are actually huge helps in UI Automation. Selenium should get off of it's high "we need to act like a browser" horse and incorporate this library in all versions! Until they do I'm now a Python automation guy :)

Use a host header value for testing services on Kubernetes

tswiftma — Tue, 02 Mar 2021 15:53:55 +0000

What is a Host Header? Basically it's an address that the web server (or ingress service) will send a request to.

At my company we are in the process of moving some of our services from Azure App Services to AKS. To run my C# API Integration tests against the Kubernetes pods our DevOps guys told me that I needed to use host header values to hit the internal K8S endpoints of the services.

This was fairly easy to accomplish in C# as I just added a default parameter for the host header to my Post method (as well as my other REST methods).

/// <summary>
/// Helper method to add headers for PostAsync operation with no data.
/// </summary>
/// <param name="httpClient">HttpClient.</param>
/// <param name="url">url.</param>
/// <param name="hostHeader">Host header.</param>
/// <returns>HttpResponseMessage.</returns>
public static Task<HttpResponseMessage> PostAsJsonAsync(this HttpClient httpClient, string url, string hostHeader = "none")
{
    var stringContent = new StringContent(string.Empty);
    stringContent.Headers.ContentType = MediaTypeHeaderValue.Parse("application/json");
    var dataAsString = stringContent.ToString();

    var httpRequestMessage = new HttpRequestMessage(HttpMethod.Post, url)
    {
      Content = new StringContent(dataAsString, Encoding.UTF8, "application/json"),
    };

    if (hostHeader != "none")
    {
        // Host name for K8S tests
        httpClient.DefaultRequestHeaders.Host = hostHeader;
    }

    return httpClient.PostAsync(url, stringContent);
}

By default the Host Header value is "none" for normal automation runs. When the tests are run against the AKS deployment we pass the host header variable value into the K8S yaml pipeline.

- stage: Test_Online
    pool:
      name: 'linux-agents'
    displayName: Test Online
    condition: and(succeeded(),eq(${{parameters.switch}}, true))
    variables:
      - ${{ if eq(parameters.envtype, 'prod') }}:
        - name: MyApiHostHeader
          value: ${{parameters.deployenv}}-${{parameters.service}}-online.${{parameters.region}}.${{parameters.deployenv}}.test.com
      - ${{ if eq(parameters.envtype, 'nonprod') }}:
        - ${{ if eq(parameters.deployenv, 'dev') }}:
          - name: MyApiHostHeader
            value:

In my Visual Studio solution for the API automation I'm reading the hostHeader value from appSettings.json. In the K8S yaml pipeline we have to do a bunch of steps to run the job, including a FileTransform task to substitute the HostHeader variable we added earlier into the solution for the tests to run.

jobs:
      - job: Test_Offline
        steps:
        - ${{ if eq(parameters.skipTests, false) }}:
          - checkout: automation
            persistCredentials: true

          - script: |
              apt-get update
              apt-get install mono-complete
            displayName: Install Dotnet prerequisites

          - task: UseDotNet@2
            displayName: 'Use .NET Core'
            inputs:
              version: $(dotnet_core_version)
          - task: NuGetToolInstaller@1
            displayName: 'Use NuGet'
            inputs:
              versionSpec: $(nuget_version)
          - task: NuGetCommand@2
            displayName: 'NuGet restore'
            inputs:          
              restoreSolution: '**\MyAPITests.sln'

          #this task performs envvar substitution. we need to set the vars to override in lib var group or 
          #the env: block does NOT seem to work for passing in variable to task. we set these in global variables above
          - task: FileTransform@1
            inputs:      
              folderPath: '$(System.DefaultWorkingDirectory)'
              fileType: json
              targetFiles: '**/**/appSettings.json'

          - script: |
              cat MyAPITests/MyAPITests/appSettings.json
            displayName: show transform

          - task: DotNetCoreCLI@1
            displayName: 'Build solution: $(solution)'
            inputs:
              command: 'build'
              projects: |
                MyAPITests/$(solution)

          - task: DotNetCoreCLI@1
            displayName: 'Run API tests in solution: $(solution)'
            inputs:
              command: test
              projects: |
                HAPAPITests/**/*Tests/*.csproj
              arguments: '-c release -v d --filter Category=${{ parameters.testCategory }} --logger "trx;verbosity=detailed;LogFileName=DotNetApiAutomationTestResults.trx"'

          - task: PublishTestResults@2
            displayName: 'Publish Offline Test Results'
            inputs:
              searchFolder: MyAPITests
              testResultsFormat: 'VSTest'
              testResultsFiles: '**/*.trx'
              testRunTitle: '${{ parameters.service }} - Offline ApiAutomation ${{ parameters.deployenv }}'
              publishRunAttachments: true
              failTaskOnFailedTests: false
            condition: succeededOrFailed()

There's a lot of moving parts here with VS, Azure DevOps pipelines and AKS but we got it working!