Running VS Code end-to-end tests on a Windows desktop nobody can see

My end-to-end tests press Ctrl+S in real VS Code windows. That can't happen on the screen I'm working on.

The extension under test is Virga, which runs Apex and SOQL scratches against an org with Ctrl+S and lets you close them without a "save changes?" dialog. Most of what can go wrong in it lives in VS Code itself: auto save, the dirty state of a tab, native dialogs, keymaps that also want Ctrl+S, tabs that come back after a restart. A fake of the VS Code API can imitate those, and Virga's unit tests do. At some point, though, you have to check the real thing, and that means real VS Code windows.

This post is about how Virga's end-to-end suite runs 221 checks in six VS Code sessions on a Windows desktop that is never shown.

Why not the official test runner

The usual way to test an extension is to start VS Code with --extensionTestsPath. Virga's suite deliberately doesn't. In extension test mode, VS Code answers its own "save changes?" dialog with "Don't Save" without ever showing it. For most extensions that's convenient. For Virga that dialog is the bug the tests exist to catch, so the suite starts VS Code as a normal application and drives it from outside.

A desktop nobody looks at

Windows has supported several desktops per window station for a long time. The one you see is called Default. CreateDesktop makes another, and a process started with that desktop's name in STARTUPINFO.lpDesktop puts its windows there. Child processes inherit it. Nothing on that desktop is ever displayed, so nothing there can take the focus.

The launcher, e2e/hidden-desktop.ps1, compiles a few lines of C# with Add-Type. Condensed, with the error handling left out:

IntPtr desktop = CreateDesktopW(desktopName, IntPtr.Zero, IntPtr.Zero, 0, GENERIC_ALL, IntPtr.Zero);
var si = new STARTUPINFO();
si.cb = Marshal.SizeOf(typeof(STARTUPINFO));
si.lpDesktop = desktopName;
PROCESS_INFORMATION pi;
CreateProcessW(null, new StringBuilder(commandLine), IntPtr.Zero, IntPtr.Zero, false,
    CREATE_NO_WINDOW, IntPtr.Zero, cwd, ref si, out pi);
WaitForSingleObject(pi.hProcess, (uint)timeoutMs);

On a timeout it kills the whole process tree with taskkill /T /F, so no VS Code is left running on a desktop I can't see.

Before anything else starts, the launcher runs a self-test. It starts a small PowerShell child on the hidden desktop that asks Windows which desktop its own thread is on (GetThreadDesktop plus GetUserObjectInformation) and writes the name to a file. If that name isn't the hidden desktop's, the run stops there and no VS Code is started. I wanted that guarantee checked every run instead of assumed.

The whole suite is started through it:

& $launcher -WorkingDirectory $root -CommandLine $cmd

where $cmd runs node e2e\run-e2e.mjs with its output going to a log file. npm run e2e does all of this.

An isolated VS Code

run-e2e.mjs starts VS Code with its own --user-data-dir and --extensions-dir under e2e/, so my own profile and extensions are never touched. Virga is loaded from the source folder, or installed from the packaged .vsix for npm run e2e:vsix. Helper extensions come along: one that registers the apex, apex-anon and soql language ids without language servers, a driver that runs the suite inside the window and quits, and in some passes stand-ins for other extensions.

The suite can use a copy of VS Code in e2e/vsc/, so a pending update of the installed one is never in the way. A copy has a few traps:

• Every VS Code start registers its urlProtocol as the Windows handler for vscode:// links. A copy has to get its own in product.json, or a sign-in callback could open the test copy with your real profile.
• It also needs its own win32MutexName, or a pending update of the installed VS Code keeps it from starting.
• A copy in a path longer than 260 characters can't load the tokenizer worker, so the test profile sets editor.experimental.asyncTokenization to false.

The base profile also sets window.dialogStyle to native and files.hotExit to off, so closing a window with a dirty file really asks, and only Virga's auto save can keep it from asking.

Keys through the DevTools protocol

VS Code is an Electron app, so with --remote-debugging-port it speaks the Chrome DevTools protocol. The suite connects to the workbench page and sends input the way a keyboard would (condensed):

const base = { modifiers: k.modifiers, key: k.key, code: k.code,
               windowsVirtualKeyCode: k.vk, nativeVirtualKeyCode: k.vk };
await s.send('Input.dispatchKeyEvent', { type: 'rawKeyDown', ...base });
await s.send('Input.dispatchKeyEvent', { type: 'keyUp', ...base });
// typed text
await s.send('Input.insertText', { text: step.text });

It also turns on Emulation.setFocusEmulationEnabled, so the page behaves as focused even though nobody is looking at it.

Going through the real input path matters. A command called directly from a test would skip the keymap. With key events, VS Code's own keybinding resolution decides what Ctrl+S does, which is exactly what I needed for the IntelliJ IDEA keymap, where Ctrl+S is Save All.

Catching dialogs instead of hanging on them

A native dialog on a hidden desktop would block the run forever. e2e/dialog-watch.ps1 polls EnumWindows every 50 ms for windows of class #32770, the native message box, that belong to the VS Code under test: a process started from the test executable whose parent is the test runner. A VS Code you have open yourself is never touched. Each dialog is logged as one JSON line with its title and buttons, then dismissed ("Don't Save", else Cancel, else its first button).

EnumWindows only sees the desktop the watcher itself runs on. That works out, because the watcher is started by the runner and inherits the hidden desktop.

The passes check that the dialog log stayed empty, so a "save changes?" anywhere fails the run.

Six runs, 221 checks

The suite starts VS Code six times:

1. Default keymap: the whole suite.
2. IntelliJ IDEA keymap: the key checks again, with Ctrl+S as Save All.
3. Restart, run A: three scratches open in two editor groups, one in a background tab, one as a preview tab, then VS Code quits. The runner then deletes the file of the scratch that was in front.
4. Restart, run B: VS Code starts again and restores those tabs.
5. Two windows, run A: the same project in two windows, with a Reload Window in one while the other closes a scratch.
6. Two windows, run B: the folder again, then a .code-workspace with it.

The last full run took 212 seconds from source and 317 seconds against the .vsix. The log reads like a list of claims. An excerpt:

ok   (4) that run had the text shown at Ctrl+S and nothing typed after it
ok   (4) every key typed after Ctrl+S is still in the editor, the 14 spaces too (no save participant trimmed them)
ok   [IntelliJ keymap] (f) Ctrl+S in a plain file saved every dirty file (Save All: the keymap is active)
ok   no VS Code dialog appeared during the whole run, closing the window included

221 passed, 0 failed; 6 VS Code run(s); 212 s

Two bugs worth the setup

The first Ctrl+S of a window. The first time VS Code dispatches a contributed command, it resolves the command's activation event with a round trip to the extension host. Keys typed meanwhile reach the document before the command does, so the first Ctrl+S could run a script that already included a key typed right after it. Virga now runs the command once at activation, as a no-op, so the event is resolved up front. The (4) checks above press Ctrl+S and keep typing, then compare what ran with what was on screen.

Restored tabs with missing files. After a restart, VS Code restored scratch tabs whose files were already gone, and other tools working on open files reported a file that "does not exist". Virga now closes restored scratch tabs at startup. Runs 3 and 4 reproduce exactly that situation, including a stand-in extension that reads files late, the way a code analyzer reading on open does.

Where the unit tests fit

Most of Virga's logic lives in two lib.js files that don't import vscode. They're tested in plain Node, together with a fake of the VS Code API (test/fake-vscode.js) that copies the behavior the extension depends on: auto save as a debounce, a short auto save counting as "being saved", failed saves, the "save changes?" prompt, restored tabs before activation, several windows on one project. CI on Ubuntu runs those tests and packages the extension on every push. The end-to-end suite is Windows only, and it's on the release checklist, against the source and against the packaged .vsix.

If you want to borrow it

Everything is in the Virga repository under MIT: e2e/hidden-desktop.ps1, e2e/dialog-watch.ps1 and e2e/run-e2e.mjs. The hidden desktop part is Windows-specific. On Linux, a virtual X server does a similar job.

• Repository: github.com/wiktormalyska/virga
• Extension: VS Code Marketplace and Open VSX

Salesforce, Apex and related marks are trademarks of Salesforce, Inc. Virga is an independent project and is not affiliated with or endorsed by Salesforce.