DEV Community: Rich

From Acceptance Criteria to Playwright Tests with MCP

Rich — Fri, 23 Jan 2026 09:56:39 +0000

Modern UI test tooling has quietly raised the bar for who can participate. Playwright is powerful, but it assumes comfort with TypeScript, selectors, repo structure, and terminal use. That gap often collapses testing back onto developers, creating pressure to almost validate their own work. This proof of concept explores using a low-code approach using Playwright MCP for a different split of responsibility. Acceptance criteria stay in plain English, owned by test & Playwright MCP is used as an execution layer to explore the UI and construct real Playwright tests from those criteria. The outcome is not “AI-written tests”, but executable checks that preserve independent validation without requiring the test team to learn Playwright or its mechanics and focus on acceptance criteria.

The problem with UI testing

The core problem is not test capability, it is familiarity with coded tests. Testers are hired to reason about product behaviour, risk, and intent, yet modern UI testing assumes knowledge of tooling like Cypress/Playwright, and code-first test structures in JavaScript/TypeScript. While non-coded testing tools exist, they tend to be brittle, opaque, or tied to expensive platforms. The result is a gradual drift of test ownership back to developers, reintroducing the “ marking your own homework ” pressure that independent testing is meant to avoid.

Yes, developers could write the tests, but should they? Yes, the testers could learn JavaScript and the Playwright framework, but all this takes time, and what if a new hire comes from different tooling or frameworks?

When we write Features, Stories, or Tasks into our ticketing system at work, they are usually accompanied by requirements and acceptance criteria, to say that in practice, we already have plain-English specifications. With minor adjustments, those acceptance criteria can become explicit UX interaction specs, simply by being a little more deliberate and verbose in the detail.

Aim:

Leverage modern, free test frameworks without forcing the test team to become framework specialists.
Use automation to translate intent into executable tests, rather than asking humans to translate intent into code.
Preserve separation between development and validation, avoiding the slow drift toward developers testing their own assumptions.
Reduce onboarding friction when testers come from different tooling backgrounds, without lowering the quality or rigour of automated tests.
But mostly: Allow acceptance criteria to remain the primary artefact, written in plain English and owned by test.

Getting Started.

The idea is simple. We can use VS Code Co-pilot to take a plain-text test definition, parse this with the LLM to do the reasoning, allow it to call Playwright tooling via MCP and then output a shiny, validated and runnable TypeScript test at the end.

Getting setup

We need:

Visual Studio Code
GitHub Copilot , ideally with a stronger model such as the latest GPT or Claude Sonnet. The better and more appropriate the model, the better the outcome will typically be.
Optional but recommended: the VS Code extension Playwright Test for VS Code, which adds native support for Playwright tests.

What will not be covered:

Any Playwright specifics like best practices or common playwright pitfalls, this will focus purely on the auto-test generation.

Create a new folder and open it in VS Code. Run the following command to scaffold a new Playwright project:

npm init playwright@latest

This will install Playwright if it is not already present.

During setup, the tooling will ask for permission to install the create-playwright package and prompt for a few configuration choices, such as language (JavaScript vs TypeScript), test folder name (“tests” is fine), and whether to install browser dependencies. The defaults are generally sensible for this proof of concept.

MCP setup

At this point we have a standard Playwright project. Next, we need to install the Playwright MCP tooling.

This can be done by visiting the Playwright MCP repository on GitHub and clicking Install for VS Code:

https://github.com/microsoft/playwright-mcp

Once installed, ensure the MCP server is running. Open the VS Code command palette and search for “MCP: List Servers”. You should see Playwright listed.

Selecting it will show options to start, stop, and configure the server. You can start it from here if it is not already running, or choose Show Configuration to open the mcp.json file and inspect or adjust the setup directly.

Custom agent instructions and test prep

Like all LLM interaction, this works best when it is provided with appropriate context. We will do two things here:

Create a custom context that explains how the agent should utilise MCP and Playwright, and how it should behave as a tester.
Create a custom instruction set that references this context where appropriate, along with some quality-of-life instructions to help keep inputs and outputs aligned.

First, create two folders at the repository root:

prompts This is where our plain-language tests will live.
contexts This is where we place our custom context files. We can, and later will, utilise the .github/agents/instructions directory, but it will become clear shortly why this is kept separate.

Inside the contexts folder, create a new markdown file named playwright-tester-agent.md

# Agent Context
You are a Playwright test generator agent.
You're given a natural language scenario describing what to test in a web application.
Your task is to generate a valid Playwright test using @playwright/test in TypeScript.

# Important:
- DO NOT generate the full test code immediately based on the scenario alone.
- DO gather context by executing steps one at a time using the Playwright MCP Tools. These steps may include but not limited to:
    - Inspecting DOM structure.
    - Fetching selectors.
    - Validating element visibility

# Process
1. Parse the scenario and break it down into actionable steps.
2. For each step:
    - Use MCP to fetch the page context
    - Validate element presence, interaction type (click, type, wait, etc.)
3. Once all steps are validating and the context is collected,
    - Emit a final Playwright test using @playwright/test syntax in TypeScript.
    - Include appropriate waits, locators, and assertions based on message history.
4. Save the generated `.spec.ts` file into the `/tests` directory.
5. Execute the tests using the Playwright test runner.
6. If the test fails, re-evaluate using MCP context and regenerate until it passes.

# Notes/Guidance
- Use plain readable locators
- Avoid hardcoding unless required.
- Follow Playwright best practices for stability and retries.
- You may be testing Single Page Apps with initial load/waits. 

# GOAL
To generate reliable, maintainable, and context-aware Playwright tests using AI and MCP.

At this point, the context file exists, but unless it is explicitly assigned in the chat window, it will not be used.

Earlier I mentioned that this file could be placed in the native instructions folder. While that does work, it causes the context to be injected into every GitHub Copilot chat session for the project. That is not what we want. The goal is to apply this behaviour only when we are explicitly asking the agent to create or evolve tests.,

To achieve this, create a new global instruction set, named it something like: agent-pw.instructions.md. We can do this by:

Open the Copilot panel in Visual Studio Code
Go to Settings → Chat Instructions
Choose New Instruction File

This instruction file can selectively reference the playwright-tester-agent.md context only when test generation is requested, keeping normal Copilot usage unaffected while still giving us a highly opinionated, test-focused agent when we need it.

---
applyTo: '**'
---
If you are being asked to generate tests for a piece of code, and acting in agent mode, follow these instructions:
- Read the instructions in the contexts/playwright-tester-agent.md file carefully.
- Follow the step-by-step process outlined in that file to gather context and generate reliable Playwright tests.

On completion. If you used a `.md` file under the `/prompts` folder to generate the tests, update the prompt file to comment the corresponding test file path at the top, i.e. `# File: tests/**/login.spec.ts` This will help track which prompts generated which tests for updates. NO OTHER UPDATES can be made to the prompt file.

When generating tests the following rules should be followed:
- Creating new data, always use unique values to avoid conflicts with existing data.
- All tests that create data must also clean up that data at the end of the test to avoid polluting the test environment.
- You will NEVER delete data that you did not create within the test itself.

At this point we are essentially saying:

Read the agent context for being a tester if you are being asked to generate tests.
Update the input md with the test name so we can link the 2, useful for asking it to update a test.
Hygiene guardrails for creating and deleting data – which seemed to work.

Running our first test

Manually

First just to get a handle of how this works, if we go to the chat window, ensure it is in agent mode, and just ask it something like (Note that as we did the step above for custom instructions, we do not need to add the context):

Generate a Test for this scenario:

1. Navigate to www.google.com
2. Validate the search input is visible
3. Validate the "Google Search" button is visible.
4. Type "Sausages" into the search box, and press the search button.
5. Confirm we have been redirected to a search page, and the page contains information on Sausages.

Note: Using sites like Google is a poor real-world example, as headless browsers are often blocked by CAPTCHA. This is purely illustrative.

The agent will explore the page, call the Playwright MCP tools, and construct a Playwright test that satisfies the scenario.

If you run the generated test (placed under the /tests folder), either via the Tests tab in VS Code if you installed the Playwright extension, or via the CLI, you should see it pass successfully.

Persistent prompt files

The issue with doing this manually in chat is that none of it is durable. Once the chat window is closed, the intent behind a test is lost, updating it becomes harder than it should be, and you no longer have a clean record of what the test was actually trying to prove. You end up with code that “works”, but without the plain-English specification that explains why it exists..

Under our prompts directory, we can create a new markdown file (or a logical folder structure of .md files) for our tests, where the content is the same as the manual scenario. We can now goto the chat window with this test in context and ask it to “Make tests” (or “Update Tests” later)

This will create a new test file under Tests, the same as the manual scenario above.

Because our instruction set includes a rule to update the prompt file with the corresponding generated test file path, these files are automatically kept in sync, creating a 1:1 mapping between the prompt and the test. This means that if we later add another requirement and ask the agent to regenerate, it updates the existing test rather than creating a new one.

At this point we have a framework in place for:

Placing markdown files into a structured prompts folder
Generating and updating tests without needing to understand the underlying test tooling

Test your tests!

The auto-healing loop is powerful, but it needs clear boundaries. An agent that is allowed to regenerate tests until they pass can easily optimise for success rather than correctness, for example by weakening assertions or validating incidental behaviour.

Make sure that you debug (as in, watch, or record using Playwright) each test execution each time the test changes to ensure it’s doing what you think it should be doing.

Full Autonomy as an option

Agents and why we deliberately did not use all of them

Playwright MCP ships with multiple agentprofiles, typically oriented around planning, execution, and validation. On paper, this looks appealing: a fully autonomous loop that can explore an application, decide what to test, and generate the tests itself. These can optionally be added into the solution easily by running:

npx playwright init-agents --loop=vscode

..and honestly, its _ impressive._ In practice however, this is exactly the boundary we chose not to cross for this proof of concept.

The issue is not capability, it is authority. When the same agent is allowed to discover behaviour, define expectations, and then validate those expectations, you collapse intent and verification into a single feedback loop. That produces coverage, but it does not produce confidence. To contrast with the opening problem, in this case you are not just marking your own homework, you are writing the subject and the test as well. This could be even worse if AI also wrote the code!

But you should check them out anyway…

Once the tooling is installed, providing your running VS Code v1.105 (released October 9, 2025) or later, we will see 3x new agents in the chat window.

The agents are:

Planner
- This agent explores a URL and generates verbose test plans based on what it discovers, without requiring explicit direction, although guidance can be provided. I did find value in asking it to explore pages I had already written tests for , as it occasionally surfaced additional edge cases. The output is a Markdown test plan that can be reviewed, edited, or passed on to another agent.
Generator
- This agent takes a test plan, optionally produced by the Planner, and generates the required Playwright tests.
Healer
- This agent works against failing tests and focuses on repairing them. It is particularly useful for stabilising brittle UI tests where selectors or timing assumptions have drifted.

I won’t go into detail on this, as their docs have plenty of more verbose notes.

I did actually add these into my project, but only used them for focussed activities rather than in a fully autonomous mode.

What isn’t covered here / Assumed knowledge and scope

Testers still need to think like someone writing automated tests.

This post does not attempt to cover every aspect of building robust automated UI tests. Topics such as environment setup and teardown, test data seeding, authentication flows, and isolation between tests are all still critical to long-term stability and are deliberately out of scope here.

While this approach removes the need for testers to work directly in Playwright or TypeScript (as in, understand the frameworks), it does not remove the need for good testing judgement. Understanding how well-structured automated tests should behave, how to avoid hidden coupling between tests, and how to reason about state, data lifecycle, and failure modes remains essential.

In other words, this lowers the barrier to expressing tests in code, but it does not eliminate the need to think like someone who writes good automated tests.

Why markdown structure matters, aka. My examples are bad…

I chose to use basic numbered list for my test example. Whilst this is valid and may be suitable for basic tests, real prompt files should be structured like lightweight test specs in markdown, for example:

A Before Each Test section for navigation and preconditions
Optional After Each Test for clean-up
A numbered set of scenarios , each with a clear name and deterministic assertions

This keeps prompts durable, reviewable, and easy to evolve over time, and it helps the generated Playwright code stay aligned with the intent rather than becoming a pile of incidental checks.

# Google : Basic Search

# Before Each Test
- Navigate to https://www.google.com
- If a consent dialog is shown, accept it (only if present) so the page is usable.

# 1. Search

## 1.1 Can load the homepage and see the core controls
- Navigate to https://www.google.com
- Search input is visible.
- "Google Search" button is visible.

## 1.2 Can perform a search and reach results
- Navigate to https://www.google.com
- Search input is visible.
- Type "Sausages" into the search input.
- Click "Google Search" (or submit the search form).
- URL indicates we are on a results page (typically contains `/search`).
- Results page contains content related to "Sausages" (at minimum, the query appears on the page).

Final considerations

This is not about replacing testers with AI. While Playwright MCP can be configured to operate in a fully autonomous loop, the more interesting value is in using it as a bridge between human intent and executable tests.

By keeping acceptance criteria explicit and human-owned, and constraining the agent to translate rather than invent intent, you preserve independent validation while removing the need for testers to work directly in code. The result is not fewer testers, but better leverage of their time on behaviour, risk, and edge cases rather than tooling mechanics.

Full autonomy remains a useful option in specific scenarios, such as discovery, baseline coverage, or large UI changes. For validating that a system behaves as intended, however, a constrained, intent-driven approach tends to produce clearer tests and greater confidence.

The post From Acceptance Criteria to Playwright Tests with MCP appeared first on yer.ac | Adventures of a developer, and other things..

Vibing in Kiro to create a self-serve Portainer wrapper.

Rich — Mon, 18 Aug 2025 10:15:11 +0000

Original

Vibing in Kiro to create a self-serve Portainer wrapper. - yer.ac | Adventures of a developer, and other things.

Using an Agentic IDE to create a self-service Portainer dashboard

yer.ac

.

I recently (finally) got my invite to Kiro, Amazon’s new agentic IDE. I’d tinkered with it, but never had a real use case outside of the usual “make me a to-do app” test.

Then Friday afternoon rolled around. With 30 minutes to go, I was asked to start parts of our Docker infrastructure. It’s a simple click in Portainer, which got me thinking… we should automate this. Developers can sign in and toggle services on or off, but that still blocks non-technical users from self-serving. Why isn’t there a simple dashboard for stopping (scale to zero), starting (scale to 1), or restarting a Portainer service? At least, not one that doesn’t require infra changes or digging through old repos – and again, this is Friday afternoon.

As an aside, our internal environments have a 1:1 mapping between service and container, so service-level actions are the quickest path. Your mileage may vary. This post is also more “mental notes” than tutorial.

I wrote this before the recent Kiro announcement, see the final concluding thoughts.

Validating with PowerShell first

First I wanted to confirm that Portainer exposes an API I could hit to scale services. If that worked, I could feed it into Kiro later.I used ChatGPT to outline the Portainer API boundaries, then had it generate a small PowerShell harness to prove the calls and auth. Straightforward enough: API key (Portainer → Account → Keys), your Environment ID, and a service name.

The gist is:

Find the service by name
Pull its spec/version
Change the replica count (0 = stop, 1 = start)
Update the service
Verify

With that working — and being able to scale to 0 or 1 easily — I dusted off my Kiro preview access.

Code below for reference:

$BASE = 'http://portainer:9000/api'
$HEADERS = @{ 'X-API-Key' = 'your-key' }
$envId = 8 # change this to the environment id
$svcName = 'PRODUCT-SIT' # change this to the service name under the environment

# 1) Find the service by name
$filters = @{ name = @{ $svcName = $true } } | ConvertTo-Json -Compress
$enc = [uri]::EscapeDataString($filters)
$svcList = Invoke-RestMethod -Headers $HEADERS -Uri "$BASE/endpoints/$envId/docker/services?filters=$enc"

if (-not $svcList) { throw "Service '$svcName' not found on endpoint $envId." }
$svcId = $svcList[0].ID

# 2) Get full spec and version
$svc = Invoke-RestMethod -Headers $HEADERS -Uri "$BASE/endpoints/$envId/docker/services/$svcId"

# 3) Set replicas to 0 (replicated mode only)

if ($svc.Spec.Mode.Replicated -eq $null) { throw "Service is not replicated." }
Write-Host $svc
$svc.Spec.Mode.Replicated.Replicas = 0

# 4) Update the service with the new spec
$ver = $svc.Version.Index
$body = $svc.Spec | ConvertTo-Json -Depth 100

Write-Host $body

Invoke-RestMethod -Method Post -Headers $HEADERS -ContentType 'application/json' `
  -Uri "$BASE/endpoints/$envId/docker/services/$svcId/update?version=$ver" -Body $body

# 5) Optional verify
$verify = Invoke-RestMethod -Headers $HEADERS -Uri "$BASE/endpoints/$envId/docker/services/$svcId"
$verify.Spec.Mode.Replicated.Replicas

Spec driven Vibe Coding with Kiro

Kiro is another fork of VS Code with agentic AI built in, similar to Cursor. Where Kiro differs is its Spec Mode. We can write a compact prompt that defines data, actions, states, and constraints, and the agent treats it like a contract. For my prompt, I kept it rough. I just pasted my PowerShell as the API reference and wrote:

I can turn off/on my docker instances by altering the scale set to 0 or 1 in portainer.
I have a script to do this like this:
{{ powershell code }}
I would like system where I have a JSOn config file which holds the URL, APIKEY, and a list of service names.
Then I want a UI for this which will:
- Show all the services
- Indicator to say if the svc is running or not (based on the scale >0)
- An option to turn off (scale 0), turn on (scale 1), or restart (scale 0, wait, scale 1)

First, Kiro will convert the prompt into system requirements. Each requirement will have its own Acceptance Criteria and will be written in a BDD syntax.

Once we are happy with the Requirements and validated its what we want, we can move to Design Phase. The design phase validates the technology choices it will use for the frontend and backend, designs and maps out the interactions between components with some sample code, and finally it adds details on testing strategy which it will use to validate itself as it goes. Here we can make any amendments (I didn’t) before moving on. Interestingly, over every test I did it defaulted to Node/React/Typescript in every instance.

The final phase is the Task List and is where we can see all the steps the agent will execute to be able to achieve the end-goal. Each step has sub-tasks and a link back to the requirement for self-validation and guidance.

Whilst the screenshot above shows me post-execution, its simply a case of selecting “Start Task” next to an uncompleted item to kick the process off

Kiro also supports steering rules for guardrails and native MCP integration to plug in tools, APIs, and docs (Although I didn’t make use of this).

Let the magic happen…

I kicked things off by running the tasks in order — starting with the basic scaffolding and React setup. As it progressed, Kiro validated its own work with unit tests and checks along the way.

Like most agentic IDEs, Kiro can’t run commands without approval. Whenever it needed to spin up a server, run tests, or interact with the system, it prompted me first. You can add commands to a trust list, either as one-offs or using wildcards — for example, trusting all npm * commands versus just npm start. That means over time it can run more autonomously on repeat runs.

There were a few hiccups though. A common one was getting stuck on steps like “Validating that Express server starts,” since it doesn’t seem to realise the terminal is tied up if the command keeps running. Similarly, Jest sometimes hung after running tests, waiting for me to manually stop it -which left Kiro stuck as well.

When I intervened (Ctrl+C to the rescue), it usually recovered and checked the terminal output to confirm whether tests had passed. But a couple of times it jumped to the wrong conclusion and marked everything as fine despite failures – I assume because there was no proper exit code or error thrown.

In those moments, the chat window proved useful. For example, when it couldn’t confirm if the server was running (because the terminal was still tied up with npm start), I told it to try curl instead. It did, and even remembered that approach for later runs.

Interestingly, when I asked “Are you stuck?“, the agent responded quite sarcastically telling me “I’m not stuck thank you, I am just busy!!“.

The end result

The end result was quite impressive really given my very basic prompt. I updated my config with a few of our environments and ran the start command. This is way closer to a working system than I got from other agents.

It was 99% the way there, the only thing that didn’t work well was the restart which was simply sending a malformed model to the API resulting in a HTTP 400. I simply switched to Kiro chat and told it there was a problem in a specific file and gave it the API info again and it corrected itself. I don’t know if I can blame Kiro for that though as I was hesitate to give it my real API key for testing until the very end in a human controlled test – I didn’t want it going rogue and messing up our environments!

Final thoughts on the output

I mean, it did what I asked. In under half an hour on a Friday, with nothing more from me than a rough couple hundred characters and the occasional tap on “Trust command,” it built a system that actually worked and would be enough for my team to self-service our test environments.

Would I rely on this for anything beyond quick POCs? Probably not. It’s a fun tool/toy, and for an internal use case like this it got about 80% of the way there with almost zero effort from me. But if this were destined for production, I’d need to review every file to be confident, and that’s where I think these agents shine more as “peer coders” than full replacements (For example I quite like Co-Pilot).

There’s also the risk of people shipping AI-generated slop without understanding it. Honestly, if I’d been a younger dev, I might have looked at this output and thought it was ready to ship even though I wouldn’t have had the React experience (I still don’t!) to properly validate it.

More importantly, Kiro’s priced themselves out.

I got very lucky with my timing here as literally that evening Kiro officially announced the tiering and free-usage limits, as well as the pricing plans. It looks like free users don’t get any spec requests at all (other than the 100 trial ones that expire), and even the $20 tier gets a mere 125 Spec requests and 225 Vibe requests. Even users on $40 pro accounts were finding that they were burning through their limits within an hour.

The issue here is that this doesn’t equate to 125 Spec requests or 225 additional interactions through vibing, as:

Each interaction could cost at last 1 spec and 1 vibe each.
The agent re-validating itself uses credits
Some specs/vibes are consuming multiple credits during feedback loops – with one user finding they burnt through a lot of their allowance with 225 vibes equating to ~11 questions and another burnt through >100 vibe requests asking Kiro to validate its own requirements vs the design and task list, and many more burning through the allowance under an hour. Even the $200 plan seems low-value.

It was met by quite a bit of backlash in places like the Kiro subreddit and their official discord with many users stating how expensive the plans are, the lack of transparency and in general a bit of a rug-pull.

Its a shame really as the hype was quite big and it seemed like it addressed a bunch of issues that Agentic IDEs like Cursor and Windsurf had. Just unfortunate that the pricing plan has essentially killed the product off before it got to its full release.

Verbosity is suddenly an issue

I didn't think much about this (More is better, right?!) until I came to edit this post.

I did find that the task list was a little too verbose, and I didn't actually run the final few tasks which were all about adding additional integration testing, logging, comprehensive unit tests (which it already had a lot of), and "productionisation readiness". Overall there were 10 main steps, with most having between 1 and 5 subtasks.

There is nothing wrong with the requirements and task list - If I were writing a spec to give to another dev I would include all this information, but in a world where each interaction has a considerable cost there is even more risk that people will omit "non-critical-path" deliverables like testing and documentation like I did. In a world of AI slop where nobody understands the outputs, do we want to risk the omission of documentation? Also what happens if the model gets changed to default to enhanced unit testing? Suddenly the costs increase...

What’s next?

Honestly, despite the sour taste around Kiro’s usage limits which makes me unlikely to stick with it for further enhancements, I do see a solid future for the self-service tool itself. It solved a real pain point in our Docker infrastructure, and with a few minor tweaks (maybe continuing the “zero-code” approach with Co-Pilot?) and deploying it into our cluster, we’ll be saving time without interrupting any critical path work. Overall, I’d call the tool a success and a fun experience nonetheless.

The post Vibing in Kiro to create a self-serve Portainer wrapper. appeared first on yer.ac | Adventures of a developer, and other things..

Moving local workspaces between users VS2019/TFS [Self Hosted/ Azure Devops]

Rich — Wed, 10 Mar 2021 09:13:49 +0000

Whilst I use GIT for most my source control these days, I still have some projects in TFSVC. On a recent switch of Visual Studio accounts I temporarily lost access to my mapped workspaces as these are linked to the VS logged in user, rather than to the machine. This post will show migrating a workspace to another user.

There are 3 scenarios covered here:

Accessing a remote workspace for another user including any pending changes
Migrating a workspace with no pending changes
Migrating a workspace with pending changes

The low effort approach instead:

If you don’t have many mappings, and have access to your old workspace still. Simply shelve any changes, map the new workspace and then unshelve the old users changes to your new workspace – all in VS. No need to read on.

Prerequisites

You are logged into VS2019 with your new user that you wish the workspaces to belong to.

I am assuming that the workspaces exist on the current machine. If not you will need to use the tf command for setting the computer name to your current machine first.

This will be using the VS Developer Command Prompt. I will also be using Windows Terminal to launch this, as per the guide in Adding VS Developer Command Prompt To Windows Terminal (VS 2019).

You will (Or we did) need to run these commands on each users machine, as although the remote update call succeeded it didn’t actually appear to have changed the workspace. YMMV.

You will also need permission to administer workspaces. In Azure DevOps this is at organization level under repos, as below. Note that this is a global permission so ensure any commands are scoped to a particular user or workspace as there is no revert.

Accessing your old workspace (Pending changes intact)

If you don’t need to migrate, but simply need access you can do this by making the workspace public with the command

tf workspace /collection:https://domain.com "WorkspaceName;UserName" /permission:Public

This will display the VS popup for editing a workspace. From what I can tell this has to be done on the machine that contains the mappings(?). Press OK. If OK is greyed out, you do not have appropriate permissions.

You may need to restart VS, but should now see an accessible remote workspace which you can work against.

Migrating workspaces

Locating workspaces

Launching the VS Developer Command Prompt and executing the command tf workspaces will display the workspaces that are accessible on the system, but not necessarily the logged in user.

Use this to get the name of the collection, and the workspace. You can also run the command below if you want to view for an owner which is not yourself, or the workspace is remote. (replacing owner and collection as appropraite)

tf workspaces /owner:olduser@domain.com /collection:https://your-collection.com

Migrating (Workspace has NO pending changes)

Run the command below, replacing the workspace name, user and

tf workspace "WORKSPACENAME;OWNER@DOMAIN.COM" /newowner:newowner@domain.com /collection:https://domain.com /noprompt

This should attempt to migrate. If you have no pending changes this should resolve successfully.

Migrating (Workspace has Pending changes)

If you attempt the above but have pending changes you will get:

TF14006: Cannot change the owner of workspace WORKSPACE;USER to NEWUSER because the workspace has pending changes. To change the owner of a workspace with pending changes, shelve the changes, change the workspace owner, and then unshelve the changes

If you have access to the old workspace, shelve the changes and undo any checkouts and retry. If you do not have access, then continue reading.

For this I coud’n’t find a CLI route that allowed me do this fully remote. This is a bit convoluted but MSDN didn’t show any commands for executing this fully remote.

Follow guide above for “Accessing your old workspace” to get the remote workspace and ensure its mapped locally.
cd into any directory which is under that workspace.
Running tf status should display what changes are made to that workspace.
Now shelve this with: tf shelve "Migrate" /move /recursive /noprompt (You can also do this in VS)
Running tf status now should yield no changes and running tf shelvesets should show a single “migrate” shelveset.
Migrate the workspace with tf workspace "WORKSPACENAME;OWNER@DOMAIN.COM" /newowner:newowner@domain.com /collection:https://domain.com /noprompt
In VS you should now see your workspace. Unshelve in VS, or run the command tf unshelve migrate /noprompt

Alternatives

There is a GUI based tool called TFS Sidekicks – Although I have not tried it it comes highly recommended on various posts.

Common issues

TF14045: The identity [user] is not a recognized identity.

In this scenario, make sure you are defining the login rather than the Owner name – i.e. me@mydomain.com or DOMAIN\USER vs John Smith.

The post Moving local workspaces between users VS2019/TFS [Self Hosted/ Azure Devops] appeared first on yer.ac | Adventures of a developer, and other things..

Adding VS Developer Command Prompt To Windows Terminal (VS 2019)

Rich — Tue, 09 Mar 2021 14:02:58 +0000

In an effort to be using Windows Terminal for everything Powershell/Command related these days it occurred to me that I hadn’t moved my VS2019 Command Prompt to Windows Terminal. This meant having to open VS all the time (as well as make sure that the command was mapped in external tools!) (If you need help on that I cover that partially in the post on Unshelving TFS changes into another branch (VS 2017))

This guide will assume you already have Windows Terminal on your system. If you don’t you can get it from the Windows Store, or on GitHub.

Editing Your Terminal Settings

Open up Windows Terminal, and go to settings (Ctrl + ,)

This is the JSON that controls the Terminal. Find the list array.

Add a new JSON object to this array, as per below

{       
        "commandline": "cmd.exe /k \"C://Program Files (x86)//Microsoft Visual Studio//2019//Enterprise//Common7//Tools//VsDevCmd.bat\"",
        "cursorColor": "#EEEEEE",
        "cursorShape": "bar",
        "fontFace": "Consolas",
        "fontSize": 10,
        "guid": "{5ee0706e-b015-46b2-98a3-2122a8e627d3}",
        "historySize": 9001,
        "icon": "C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Enterprise\\Common7\\IDE\\Assets\\VisualStudio.70x70.contrast-standard_scale-80.png",
        "name": "Developer Command Prompt for VS2019",
        "padding": "0, 0, 0, 0",
        "snapOnInput": true,
        "startingDirectory": "%USERPROFILE%"
    }

This will tell Terminal to start cmd.exe and run the VS developer command line tools.

Now go back to Windows Terminal and you should see a new entry in the dropdown list which when selected should launch the VS prompt.

Windows Terminal with new option selected

VS command prompt within Windows Terminal

Adding An Icon

I added an Icon object that VS already had on the system so that it will look nicer in the dropdown. You can point it to any image on your system, but for reference the images that VS uses are stored permanently at:

C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\Assets

The post Adding VS Developer Command Prompt To Windows Terminal (VS 2019) appeared first on yer.ac | Adventures of a developer, and other things..

Supporting multiple configurations in Cypress

Rich — Fri, 07 Feb 2020 16:13:12 +0000

By default, Cypress will support a single configuration based on the optional file cypress.json as described in their documentation here.

Whilst this works fine for most, it would be great if we could have access to a cypress.dev.json for local development, or even better, a whole host of configuration files for use against a multi-tenant environment – for example cypress.clientA.json, cypress.clientB.json etc.

Whilst Cypress accepts a different config file during startup with the --config-file flag, it would be better if we could just pass the environment name through instead of the full file name and/or location, right?

Uses for environmental variables

I personally use these environmental files to store things like:

Base URL: Each client has its own SIT/UAT environments with different URLs
Default username and password for test environments.

Creating the different config files

We can create a root level folder named “Config”. Under here we can create as many files as we need to cover, for example I have config.ClientA.json which contains:

{
  "baseUrl": "http://clientA.internalserver.co.uk/",
  "env": {
    "someVariable": "Foo"
  }
}

And config.ClientB.json which contains:

{
  "baseUrl": "http://clientB.internalserver.co.uk/",
  "env": {
    "someVariable": "Bar"
  }
}

Editing the plugin file

First we need to import “path” and “fs-extra” packages by adding the following at the top of the index.js file within the /Plugins folder (if it doesn’t already exist!). These will allow the file to be located and subsequently read.

const path = require("path");
const fs = require("fs-extra");

Next we need the method which will take in a client name/environmental variable, locate the appropriate config file (being /config/config. name.json), and then reading that file back to the calling method.

function getConfigurationFileByEnvName(env) {
  const fileLocation = path.resolve("cypress/config", `config.${env}.json`);
  return fs.readJson(fileLocation);
}

and finally we need the index.js file to export this file. This will also have a fallback in place if one is not defined.

module.exports = (on, config) => {  
  const envFile = config.env.configFile || "local";
  return getConfigurationFileByEnvName(envFile);
};

The eagle eyed may realise that I am using config.env.configFile here which will mean passing an environmental flag in the command line rather than making direct use of the --config flag. This is personal preference, as I aim to expand on the env flags later so this will look cleaner.

Consuming the configuration

Now, when running the usual open command, we can make use of the --env flag to pass it the environmental variable. We do so with:

./node_modules/.bin/cypress open --env configFile=clientA

It should now launch the test runner with your different files environmental variables available via Cypress.env('key')

The post Supporting multiple configurations in Cypress appeared first on yer.ac | Adventures of a developer, and other things..

⚡lightning-fast testing of web applications with Cypress

Rich — Fri, 07 Feb 2020 12:00:00 +0000

Cypress (Cypress.io)is a automation framework for web app testing built and configured with Javascript. Automated front-end testing is definitely not new, but Cypress really is something different. It’s silly fast, requires almost no setup, has quick-to-learn syntax and has a really nice, feature packed test runner.

Why Cypress? I’ll let you read the summary on the summary page at cypress .io, whilst also stealing this image from their blurb

TL;DR: Why have all those libraries to manage, drivers to install and syntax’s to remember?!

Don’t we already have tons of testing frameworks?

Yes. I have previously used tooling like Selenium with C#, and know our QA team use paid tooling like Sahi Pro, for a start.

Whilst these tools are OK, they often feel clunky with tooling oddities and not-to-friendly syntax. Supplementary to this, a lot of these tools are Selenium based which means they are all sharing the same annoyances.

Setting up

To get going with Cypress, simply run the NPM command: npm install cypress --save-dev within the folder you want to use Cypress from. Note that Yarn variants are also available and can be found on their site.

If the command executes successfully, you should have a new ./node_modules directory and the package-lock.json.

To setup and open Cypress for the first time, simply execute the command below, whilst in the context of your installation folder.

./node_modules/.bin/cypress open

This will do a couple of things:

Create a folder named cypress within your working directory – This will be where all your test descriptions and configuration lives
Opens up the Cypress App.

Cypress test explorer launched for the first time.

Feel free to explore the examples which provide samples of common tests, but we won’t cover them in this post.

Project structure

If you open the Cypress folder in VS code, you will find the default project files for a Cypress project.

Integration : This folder will contain all the spec files for this project. Creating sub folders within here will be echoed in the test runner. For example, you may have a folder structure like ./integration/cms/account which contains just the tests for the account functionality. How you structure this is up to you.

Support: The support folder contains 2 files, index.js and commands.js. The index.js file will be ran before every single test fixture and is useful if you need to do something common like reset state. The index file also imports the commands.js file.

commands.js is imported by the index file, and is another place to store common code, but has the advantage that it can be called from any test fixture, at any stage. An example of this could be storing the login method here under a command named DoLogin which saves having to define this in every fixture.

Plugins: Contains a single file index.js which is a jump-off point for importing or defining changes to how Cypress works.

Diving into testing with a real example

Creating and running tests

First of all, I will delete the examples folder. For this post I will be “testing” the Twitter desktop site as all my real examples are for enterprise or private software.

NOTE : This software is not designed for general browsing automation and should only be used against websites you maintain/own. In fact, a lot of sites try to block this and I actually struggled to find a public site I could use this against consistently!

Create a test fixture/spec

Create a new file underneath the “Integration folder” named “MyTest.spec.js” the “.spec” is a naming standard for defining specifications which I suggest you keep, but isn’t strict.

The structure of this file should be as follows:

describe("Twitter example tests", function() {
  it("Page should load", function() {
    cy.visit("https://twitter.com/login");
  });
});

Each file contains a single description, which in turn can contain many steps. I advise a high level of granularity when writing tests, such as a test spec for testing the login page with several steps is fine, having one that tests your website with hundreds of steps, not so much.

If you save this file, and still have the test runner open, it should have automatically found this new test. If you closed the runner, simply re-run the ./node_modules/.bin/cypress open command again.

Clicking on this test will open up a new browser instance (based on the one selected in the drop down – seen in the top right of the screenshot above). The test runner will open a split-window with the executing tests (and results) on the left, and the browser view on the right.

Of course, this test passes as it doesn’t actually *do* anything! Let’s change this! You don’t need to close this runner either, as any changes to this test will be picked up automatically and re-ran.

Basic Interactions

For this example, we will take the existing test above and have it test logging in to the website and navigating to the settings panel.

Loading a web page : A redirect or page load is done with cy.visit(url). For this example we used cy.visit("https://twitter.com/login");

Locating an element: This is done similar to how jQuery finds objects in that you can find them on type, id, class or data attribute. The flow is always to find an item first, then chose what to do with it. For this we need to find 2 text boxes- one for user and one for password.

As Twitter does some magic with their element classes I will be locating the boxes by their unique attributes. If I use the code below, you can see the test will pass as it finds the element on the page. Hovering over the test in the test steps will highlight the matching field.

describe("Twitter example tests", function() {
  it("Page should load", function() {
    cy.visit("https://twitter.com/login");
    cy.get("input[name='session[username_or_email]']");
  });
});

Interacting with an element : Once we have located the element we can interact with it with methods such as .type(), .click() and more. In this example I want to set the username and password field appropriately and then click the enter button, so the code now looks like:

describe("Twitter example tests", function() {
  it("Page should load", function() {
    cy.visit("https://twitter.com/login");
    cy.get("input[name='session[username_or_email]']")
      .first()
      .type("MyHandle");
    cy.get("input[name='session[password]']")
      .first()
      .type("password1234");

    cy.get("form[action='/sessions']")
      .first()
      .submit();
  });
});

If we run this now we can see that the page is loaded, the form is filled out and the form is submitted. The test passes, but should fail as the actual login fails due to incorrect details.

Finding text: One way we could validate if the test above succeeds is to check for the existence of an object, or some text on the page which states the login was not a success. To do this we can add the line cy.contains("The username and password you entered did not match our records. Please double-check and try again."); which will check the entire DOM for that specific text. We could also find a specific element using .get() and chaining on the .contains() method.

Waiting: Waiting is part of all web applications, and although Cypress will retry a few times if it cannot locate an element, it does not have a long timeout. The cy.get() takes in an additional options object in which a timeout can be specified. For example: cy.get(".some-class-which-isnt-visible-yet", { timeout: 30000 }); would pause the execution of the test until the element is located, or the 30,000ms timeout occurs.

Code sharing and re-use

Lets say we have expanded our tests so we have a new test which detects if the word “Home” is displayed to the user on their dashboard once logged in.

describe("Twitter tweet tests", function() {
  it("When logged in the word Home appears", function() {
    cy.contains("Home");
  });
});

Running this will fail as it doesn’t know which website to use. We could use the cy.visit() method, but as each test is ran is isolation of the others we wouldn’t be logged in. Whilst we could just copy the login code from the first test into this one (either in the it method, or in a beforeEach block), its a little messy to do so and introduces duplication and more maintenance.

Commands & Shared Code

Remember that commands.js file under the Support directory? Lets create a new command which will do our login from a central place! We will simply cut and paste in the contents of the login section of the previous test, like so:

Cypress.Commands.add("twitterLogin", () => {
  cy.visit("https://twitter.com/login");
  cy.get("input[name='session[username_or_email]']")
    .first()
    .type("MyValidUser");
  cy.get("input[name='session[password]']")
    .first()
    .type("MyPassword");

  cy.get("form[action='/sessions']")
    .first()
    .submit();
});

This tells Cypress that there is a command available called “twitterLogin” and which steps to execute when this command is called. Now we can simply update the login.spec.js to be:

describe("Twitter tweet tests!", function() {
  it("Can compose a tweet", function() {
    cy.twitterLogin();
    cy.contains(
      "The username and password you entered did not match our records. Please double-check and try again."
    );
  });
});

Now, we can call cy.twitterLogin() from any of our spec files!

Final thoughts

Cypress may well become my favorite UI testing framework. In less than a day I was able to gain enough knowledge to put together a fairly large proof of concept for testing one of our front end applications. The only “difficulties” were things like persisting authentication which only took a few google searches to solve. I may have other posts around adding additional flexibility in the future.

The main benefit to me (other than the flexibility, speed, and the obvious) is that the syntax is flexible enough for a developer, but easy enough for somebody with less coding knowledge (QA, BA, etc).

The post ⚡lightning-fast testing of web applications with Cypress appeared first on yer.ac | Adventures of a developer, and other things..

Using Docker Containers for easy local WordPress development🐳

Rich — Mon, 06 Jan 2020 09:00:00 +0000

This will cover off setting up a Docker container for local WordPress development & mounting the container folder for easier development.

Major edit: there is a performance issue with this which I have mentioned in.more detail on the original article http://yer.ac/blog/2020/01/06/using-docker-containers-for-easy-local-wordpress-development🐳/

Why do this?

Typically when doing any kind of WordPress work (Which is exceptionally rare for me), I would spin up a new WP instance on my host, or use something like a local LAMP/XAMP server.

Whilst this works, leveraging things like Docker mean almost zero configuration and more time developing. Better still, these then become throwaway environments!

Requirements

The only requirement for this is the installation of Docker (https://www.docker.com)

Creating the WordPress instance

Firstly, create the folder where the configuration will live, for example C:\Docker\Wordpress In this folder we need to make a file named docker-compose.yml. This will be the YAML file detailing our wordpress installation – such as login information and MySQL setup.

In this file, copy and paste the content below:

version: '3.1'
services:
  wordpress:
    image: wordpress
    restart: always
    ports:
      - 1234:80
    environment:
      WORDPRESS_DB_HOST: db
      WORDPRESS_DB_USER: exampleuser
      WORDPRESS_DB_PASSWORD: examplepass
      WORDPRESS_DB_NAME: exampledb
  db:
    image: mysql:5.7
    restart: always
    environment:
      MYSQL_DATABASE: exampledb
      MYSQL_USER: exampleuser
      MYSQL_PASSWORD: examplepass
      MYSQL_RANDOM_ROOT_PASSWORD: '1'

This will tell Docker that we need to use the image name “WordPress” from the Docker Hub (https://hub.docker.com/_/wordpress), forward port 80 of the container (Which will be the exposed website) to port number 1234 of the parent host. This means going to http://localhost:1234 would go to port 80 of the docker container.

Finally to use MySQL v5.7 as our database. You shouldn’t need to change this information, but if you do, make sure that the Database information in the wordpress section matches the information in the db section.

Once this file is saved, we can run docker-compose up -d whilst in the same directory. This will take the YAML file, download any images that are not already on the local system and then set WordPress and MySQL up based on the YAML.

Note: : If you get the error no matching manifest for windows/amd64 in the manifest list entries when running this command, and you are on Windows 10, you will need to enable “Experimental Mode” in your Docker installation. To do this, right-click on Docker in the system tray and go to settings, checking the box named “Experimental mode”.

Note: : If you get an error like ERROR: for wordpress_db_1 Cannot create container for service db:, this is usually caused by there being a conflict in ports. Try changing the external port (in this example “1234” for something else)

We can confirm if the docker container is running by running docker ps to list all running containers, or by going to http://localhost:1234 (The port will be whatever you set it to in the YAML)

From here you can follow the standard WordPress installation steps.

Stopping and starting instances

You can stop and start instances by doing the following:

docker-compose stop and docker-compose up -d. Stop may take a little while.

Note that these commands (in this syntax) have to be ran at the same level as the YAML file.

Working with the instance

If you just wanted to play with WordPress this would be enough, but what if you wanted to copy files to the instance, or edit files?

Option 1: SSH/ FTP

Although not personally a fan of this method, we could SSH to the container with a tool like PuTTY, see this guide: https://phase2.github.io/devtools/common-tasks/ssh-into-a-container/ We could also configure the container to have an FTP server. This won’t be covered as I believe the next option to be better.

Option 2: Mount a windows folder

This option will mount a specific folder within the Docker container to a folder on your own system.

Before continuing, we need to make sure that the container is currently stopped (docker-compose stop)

We will make a change to the YAML file so that we add a volumes option, as shown below.

version: '3.1'

services:

  wordpress:
    image: wordpress
    restart: always
    ports:
      - 1234:80
    environment:
      WORDPRESS_DB_HOST: db
      WORDPRESS_DB_USER: exampleuser
      WORDPRESS_DB_PASSWORD: examplepass
      WORDPRESS_DB_NAME: exampledb
    volumes: 
    - "C:/Docker/Wordpress/Mounted:/var/www/html"

  db:
    image: mysql:5.7
    restart: always
    environment:
      MYSQL_DATABASE: exampledb
      MYSQL_USER: exampleuser
      MYSQL_PASSWORD: examplepass
      MYSQL_RANDOM_ROOT_PASSWORD: '1'

As we can see, the volumes setting contains a string which is made up of 2 parts, seperated by a colon :. These are:

C:/Docker/Wordpress/Mounted – This is the local path you want to mount the folder against.
/var/www/html” – This is the path within the container you want to mount.

Note that personally I like having the mount in the same directory as the configuration YAML for clarity, but it can live anywhere on your system.

Saving this file and re-running the docker-compose command will now map the specified volume.

Footnote

I am a bit of a newbie when it comes to Docker, and I don’t use WordPress an awful lot so do take with a pinch of salt, and do let me know if this can be improved!

The post Using Docker Containers for easy local WordPress development🐳 appeared first on yer.ac | Adventures of a developer, and other things..

Identifying Nuget package references which are using relative paths across whole solution

Rich — Wed, 20 Nov 2019 12:25:56 +0000

Keeping up with a recent binge upgrading projects, including upgrading all my projects in a solution to 4.8, I have been upgrading nuget packages. Whilst this is a relatively simple task, what irks me is that when you add or upgrade a nuget package in Visual Studio, it will often change the package path to be relative to the project file.

What does the issue look like?

When we do something like Install-Package EntityFramework -Version 6.3.0 we end up with something like this in the CSPROJ

As we can see, the <HintPath> is using a relative path based on the project file location to the solution file (As packages are kept under /Packages at the same level of the .sln)

Why is this a problem? Although it could “just work” for some scenarios, if there are shared libraries which are used across different solutions we can end up with a build failure due to missing packages! Ideally, the hint path should start with $(SolutionDir)packages\ so the correct package will be located regardless of the solution being built.

There are workarounds to stop this occurring (I had varied results), but this doesn’t really solve the issue of retrospectively locating all the places this has happened!

Identifying all references which use relative paths

My solution has close to a hundred projects nested within it, so a manual check could be problematic and prone to human error.

Below is a snippet that will take a Solution file, and for each project check for any hint path which does not contain $($SolutionDir) providing the directory and solution name are changed.

Once the script runs it will output any projects and hint paths that may be an issue. For me it was simply going over these one by one and changing everything before \Packages (i..e …....\somefolder\Packages) to use the solution directory variable – i.e. $(SolutionDir)\Packages

Creating a helper in Visual Studio

Although the snippet above will quickly identify any potential issues, it’s not very reusable as you have to change the paths for each solution.

One thing we can do is add a custom tool into Visual Studio to execute the Powershell on demand for the given solution.

To do this, first save the script in the snippet below to your own system. Note this is slightly different to the script above as it takes in a parameter.

Once this is done, go to Visual Studio -> Tools -> External Tools -> Add

In here we can make a new external tool called something like “Find Bad References”.

Command: Path to the system Powershell executable. C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe

Arguments: -file "C:\Path\To\File.ps1" $(SolutionFileName)

Initial Directory: $(SolutionDir)

Use Output Window : Checked

Now, once saved we should have a new option under tools with the same name as the provided title.

Once clicked, it will execute the script and provide results (if any) in the output tab.

Final thoughts

This isn’t great, and there are plugins that do similar things already, hell, some may even automatically fix these issues! This was designed to scan and generate the potential issues for a manual verification. Feel free to improve it and let me know!

The post Identifying Nuget package references which are using relative paths across whole solution appeared first on yer.ac | Adventures of a developer, and other things..

Pragmatically upgrading .net framework version for all projects with PowerShell

Rich — Wed, 06 Nov 2019 10:28:44 +0000

Do feel free to provide any comments/feedback to @TheRichCarey on Twitter

We had a situation where we needed to upgrade all the CSPROJ files in a solution to 4.8. The issue is that some of our solutions contain almost a hundred projects so a manual intervention would be prone to error. (plus we have multiple solutions to apply this against!)

Whilst there are a number of extensions that do this on the VS Marketplace, they seemed a little overkill for something that can surely be achieved in PowerShell? At the end of the day its a simple find-and-replace, right?

This will load the solution file, iterate over each CSPROJ referenced and then replace the current framework version with the one specified in $versionToUse. It will then overwrite the file.

Potential improvements

This met my needs fine, but could be improved for sure! Things that it could do better are:

Auto scan for .sln files.
Provide some form of report at the end
Auto-checkout for TFS or Git (Using git CLI or TFS CLI)

The post Pragmatically upgrading .net framework version for all projects with PowerShell appeared first on yer.ac | Adventures of a developer, and other things..

Ensuring “dotnet test” TRX & Coverage files end up in SonarQube

Rich — Wed, 16 Oct 2019 14:13:10 +0000

Do feel free to provide any comments/feedback to @TheRichCarey on Twitter

I have written beforeabout using SonarQube to do static analysis, but one issue I never came back to was ensuring that code coverage files generated via a build pipeline end up being picked up by the Sonar Scanner to assess code coverage.

Note that the following I am actually using the ‘dotnet test’ build step, rather than the ‘Vs Test’ one. Do let me know if you find a nice work around for the VS Test variant, as I couldn’t get it to drop coverage files!

The issue

The issue is that:

When using VSTest, TRX files are deleted automatically if using version 2+ of the VS Test task as per this stack overflow post.
When I switched back to ‘dotnet test’ the same thing appeared to be happening.
.coverage files are not output by default
TRX and Coverage files are placed in a temporary folder of the build agent rather than the executing agents working directory.
Even though SonarQube could detect the tests, it would still register as 0.0% code coverage!

Getting ‘dotnet test’ to collect coverage

The first step was to get the ‘dotnet test’ build step to collect the code coverage, and not just dump TRX files.

To do this, go to the “Arguments” field of the dotnet test build step and append --collect "Code Coverage", as well as ensuring that “Publish test results and code coverage” is enabled.

Ensure generated files are copied to the working directory

As the coverage files will end up in the /tmp folder of the build agent, SonarQube will not be able to scan them.

We will need to add a new build step of “copy files” with the correct filter set to get the .trxand .coverage files from the default temporary directory on the build agent, to the test results folder of the workspace. To do this we need to add the “Copy Files” task into the build and place it after the “VS Test” task. The source folder for the copy will be $(Agent.HomeDirectory)\_work\_temp and the target folder will be $(Common.TestResultsDirectory) – The contents can remain as ** but feel free to filter if required. Example below.

If we run a build now, we should now see files in the TestResults folder of the build agent’s working directory.

I didn’t have to make any changes to the configuration within SonarQube as it should just pick up the coverage files. If I follow the above I get the following (Lets just ignore the fact the number is low ;)

CSPROJ Changes to test projects

One thing I did notice in the console when attempting to fix this code coverage issue was that I got a lot of warnings like:

SonarQube.Integration.targets: warning : **The project does not have a valid ProjectGuid**. Analysis results for this project will not be uploaded to SonarQube.

As all my projects were .net core or .net standard the CSPROJ files do not contain a <ProjectGuid> tag by default. As also suggested in this stack overflow answer, I added a GUID to my test project file. I am not 100% if this is required, but it stopped warnings appearing in my console and does no harm.

Bonus

If you have multiple builds to update and you are using Azure Devops, you can take advantage of “Task Groups”. This allows you to create a single build step which in turn executes a series of other build steps. Using the steps above, you can create a new Task Group to create a single build step to run the test script and make sure the files are copied to the correct location for analysis. For example I have the single build step below:

Which means I can then just call this single build step in all my builds

The post Ensuring “dotnet test” TRX & Coverage files end up in SonarQube appeared first on yer.ac | Adventures of a developer, and other things..

Include both Nuget Package References and project reference DLL using “dotnet pack” 📦

Rich — Thu, 05 Sep 2019 09:04:05 +0000

Edit: Looks like at somepoint dev.to did a formatting change which broke some bits here. Check out the Original: http://yer.ac/blog/2019/09/05/dotnet-pack-project-reference-and-nuget-dependency/

Recently I have been trying to generate more Nuget packages for our dotnet core projects, utilizing the dotnet pack command. One issue I have been encountering is that the command was either referencing the required nuget packages, or the project reference DLLs, never both.

The current problem.

If you have Project A which has a project reference to Project B as well as including a nuget package called Package A you would expect the generated package to contain a link to both the required nuget package, and the DLL(s) for Project B, yes? This however is not how the dotnet pack command works.

This issue is widely reported on their repo (I.e. https://github.com/NuGet/Home/issues/3891 ) and unfortunately it seems the developers and the community are in a bit of a disagreement to what is “correct”. The official stance (as I understood it) is that the project references won’t be included as they should be their own packages. This however is not always practical or desired.

The workaround.

Plenty of workarounds have been suggested around Stack Overflow and Github including having a seperate nuspec file, using Powershell to inject things into the generated nupkg and so on…

The solution below worked for me, but of course, YMMV.

In the end I ditched having my own .nuspec file within my project (as per some SO posts) and instead used the CSPROJ (as recommended). Below you can see the required fields for the packaging (version, naming, etc), a reference to a nuget package, and a reference to another project within the solution.

Snippet of CSPROJ with basic package info filled in.

If you run dotnet pack now, it will generate an appropriately named package which will contain a nuget dependancy on SomeNugetPackage. This can be confirmed by opening the nupkg with an archive tool (7Zip,WinRar, WinZip…) and seeing that the only DLL in the lib folder will be the DLL of the project being packed.

The fix is as follows:

Alter the project reference to set the ReferenceOutputAssembly flag to true, and IncludeAssets to the DLL name

<ProjectReference Include="..\ProjectB.csproj">
  <ReferenceOutputAssembly>true</ReferenceOutputAssembly>
  <IncludeAssets>ProjectB.dll</IncludeAssets>
</ProjectReference>

Add the following line into the <PropertyGroup> element

<TargetsForTfmSpecificBuildOutput>$(TargetsForTfmSpecificBuildOutput);CopyProjectReferencesToPackage</TargetsForTfmSpecificBuildOutput>

Add new target between <project> tags

<Target DependsOnTargets="ResolveReferences" Name="CopyProjectReferencesToPackage">
    <ItemGroup>
      <BuildOutputInPackage Include="@(ReferenceCopyLocalPaths->WithMetadataValue('ReferenceSourceTarget', 'ProjectReference'))"/>
    </ItemGroup>
  </Target>

So now you end up with something that looks like this

netstandard2.0
1.0.9
MyProduct
MyProduct
MyProduct
Your name
Company Name
My library
Copyright © 2019 MyCompany
$(TargetsForTfmSpecificBuildOutput);CopyProjectReferencesToPackage

true
ProjectB.dll

End result CSPROJ. (Click to enlarge) "/>

Now if you run dotnet pack you should see any project reference DLL under the lib folder of the package, and if you inspect the nuspec file inside the package (or upload it to your package repo) you should see the nuget dependencies.

Hopefully this helps someone, as there is a lot of conflicting info around. Please let me know if this would cause any issues!

The post Include both Nuget Package References and project reference DLL using “dotnet pack” 📦 appeared first on yer.ac | Adventures of a developer, and other things..

💰Penny Pinching in Azure💰: Setting up a self-hosted build agent for Azure DevOps to save cash

Rich — Wed, 31 Jul 2019 19:41:51 +0000

Azure DevOps has brilliant build pipeline options and as easy as it is to get set up with their hosted build agents, it can get quite costly rather quick. In this post I cover off setting up a self-hosted build agent for use with Azure.

This post won’t cover setting up the build box, but can be covered in a later guide if required. I actually have my build box scripted out using Choco commands to allow building of .NET projects to make this step easier.

Pros/Cons

Pro: Full control over the build
Pro: Can have your builds build items or run services which simply aren’t available in the Hosted agents.
Pro: Low cost. If you already have the hardware, why pay for Azure VMs?
Con: Maintenance and redundancy. If the machine goes down or breaks it blocks your pipeline.
Con: Extra setup steps.

Prerequisites

Before starting you will need to make sure:

You are a collection/build admin
You have a server configured to build the appropriate software (i.e. Correct SDKs etc which won’t be covered in this post)

Personal Access Tokens

First of all, you will need a personal access token for your account. This is used to allow your build agent access to Azure without hard-coding your credentials into your build scripts. You can use your own account for this, or a specially created service account – Just note it will need permissions to access the collections it will be building.

To get this, log in to your Azure Devops portal, and navigate to your security page.

In here, select “Personal Access Tokens” and then “New”. A panel will be displayed to configure this PAT. Specify a friendly and unique name, select the organisation you are using this token for, and then set its security access.

For the security access, I recommend selecting Full Access under “Scopes” so you can use this PAT for general Dev Ops activities. You can fine-tune the control, but you must ensure it has read/execute on the build scope as an absolute minimum. For expiry I typically select the longest period which is 1 year.

Agent download and configuration

Next up you will need to navigate to the project settings > Pipelines > Agent Pools.

Create a new Agent Pool with an appropriate name (You don’t *have* to do this and can just use the default pool if you wish, but I like the separation). When your pool is created you will see the option to add a new agent to it.

Clicking “New Agent” will give you the instructions for the OS of your choice. As per the instructions, download the agent (A ~130 ZIP file) and then place somewhere sensible on the machine that will be acting as a build server. When extracted, run config.cmd in an elevated command window

When running the config.cmd command you will require the following information:

Server URL
- This will be https://dev.azure.com/{organisation name}
What type of authentication you will use (Just press return as it will default to PAT)
Your PAT to access the server, as set up in the first step.
The Pool to connect to. This will be the name of the agent pool created above.
The working folder. The folder to use for storing workspaces being built.
A name for this agent. Call it whatever you want, but I would personally always include the machine name as it makes it easier to work out which agents are running.

Providing all the above settings are specified correctly and there are no authentication issues, it should now attempt to start.

Confirming the agent is active

Going back to the Agent Pools configuration screen you should now see the agent listed in the appropriate agent pool.

If the agent is not displaying after a few minutes, something went wrong in setup.

If the agent is displaying offline, try running the “run.cmd” command in an elevated command window on your build server.

Now all you have to do is select your new agent pool when creating your next build!

The post Setting up a self-hosted build agent for Azure DevOps appeared first on yer.ac | Adventures of a developer, and other things..

DEV Community: Rich

From Acceptance Criteria to Playwright Tests with MCP

The problem with UI testing

Getting Started.

Getting setup

MCP setup

Custom agent instructions and test prep

Running our first test

Manually

Persistent prompt files

Test your tests!

Full Autonomy as an option

Agents and why we deliberately did not use all of them

But you should check them out anyway…

What isn’t covered here / Assumed knowledge and scope

Testers still need to think like someone writing automated tests.

Why markdown structure matters, aka. My examples are bad…

Final considerations

Vibing in Kiro to create a self-serve Portainer wrapper.

Vibing in Kiro to create a self-serve Portainer wrapper. - yer.ac | Adventures of a developer, and other things. Using an Agentic IDE to create a self-service Portainer dashboard yer.ac .

Vibing in Kiro to create a self-serve Portainer wrapper. - yer.ac | Adventures of a developer, and other things.

Validating with PowerShell first

Spec driven Vibe Coding with Kiro

Let the magic happen…

The end result

Final thoughts on the output

More importantly, Kiro’s priced themselves out.

Verbosity is suddenly an issue

What’s next?

Moving local workspaces between users VS2019/TFS [Self Hosted/ Azure Devops]

The low effort approach instead:

Prerequisites

Accessing your old workspace (Pending changes intact)

Migrating workspaces

Locating workspaces

Migrating (Workspace has NO pending changes)

Migrating (Workspace has Pending changes)

Alternatives

Common issues

TF14045: The identity [user] is not a recognized identity.

Adding VS Developer Command Prompt To Windows Terminal (VS 2019)

Editing Your Terminal Settings

Adding An Icon

Supporting multiple configurations in Cypress

Uses for environmental variables

Creating the different config files

Editing the plugin file

Consuming the configuration

⚡lightning-fast testing of web applications with Cypress

Don’t we already have tons of testing frameworks?

Setting up

Project structure

Diving into testing with a real example

Creating and running tests

Basic Interactions

Code sharing and re-use

Commands & Shared Code

Final thoughts

Using Docker Containers for easy local WordPress development🐳

Why do this?

Requirements

Creating the WordPress instance

Stopping and starting instances

Working with the instance

Option 1: SSH/ FTP

Option 2: Mount a windows folder

Footnote

Identifying Nuget package references which are using relative paths across whole solution

What does the issue look like?

Identifying all references which use relative paths

Creating a helper in Visual Studio

Final thoughts

Pragmatically upgrading .net framework version for all projects with PowerShell

Potential improvements

Ensuring “dotnet test” TRX & Coverage files end up in SonarQube

The issue

Getting ‘dotnet test’ to collect coverage

Ensure generated files are copied to the working directory

CSPROJ Changes to test projects

Bonus

Vibing in Kiro to create a self-serve Portainer wrapper. - yer.ac | Adventures of a developer, and other things.

Using an Agentic IDE to create a self-service Portainer dashboard

yer.ac

.

Include both Nuget Package References and project reference DLL using “dotnet pack” 📦