15 of 16 Mac UI tests failed: what SwiftUI on macOS taught us

Our notes app, MindPage, is one SwiftUI codebase for iPad, iPhone and Mac. The iPhone and iPad UI tests were green. Then we ran the Mac suite, which drives the real app with mouse and keyboard through XCUITest. 15 of 16 tests failed. By the end of the day it was 16 of 16, alongside 43 iPhone and 44 iPad tests.

Roughly half the failures were real Mac bugs, and half were the tests. Here is each one, with the cause, because most of them are SwiftUI-on-macOS behaviours that aren't obvious if you build iPad first.

The scoreboard

Run	Passing	What changed
1	1 / 16	first Mac run
2	6 / 16	sidebar width, a button label, a missing test ID
3	9 / 17	choose() skips disabled items, text queries narrowed, ⌘V fixed (+1 temporary probe test)
4	15 / 16	undo version, pen bar hit area, photo accessibility, Search/Summarize/Word lookups
5	16 / 16	last query narrowed

Real bug 1: toolbar items silently moved to the overflow menu

On macOS, .toolbar items attached to a NavigationSplitView sidebar render in the toolbar above the sidebar. Our sidebar was 148 pt wide. Three buttons plus the system "Hide Sidebar" button didn't fit next to the traffic lights, so macOS moved New folder, sync status and Settings into "more toolbar items". No warning, no layout error.

#if os(macOS)
.navigationSplitViewColumnWidth(min: 300, ideal: 300)
#endif

230 pt was not enough; the window controls take about 100 pt.

Real bug 2: a split button announced as "add"

A Menu with a primaryAction becomes a pop-up button on macOS. Its inner button took its accessibility label from the SF Symbol (plus), so VoiceOver said "add". .accessibilityLabel("New notebook") on the menu fixes VoiceOver; in XCUITest you then find it as app.popUpButtons["New notebook"], not app.buttons.

Real bug 3: undo history wiped after every stroke

Our canvas sends each edit to the model as a small value: the page, the strokes before and after, and the version of the page it was drawn on. That version field had a default of 0.

If an edit's version is older than the model's, the page was reloaded by sync in between, so the model redraws from the merged page and bumps its version. The canvas reacts to a version bump by clearing undo history, because older steps may not fit the merged page.

The iPad canvas passed version. The Mac canvas didn't, so every edit had the default 0. Every stroke looked like it came from before a reload, and the history was wiped after each one. ⌘Z never had anything to undo.

The fix stamps the version on edits, and on undo/redo steps as they are replayed. The lesson is about defaults: a field with a harmless-looking default of 0 compiled fine and failed silently.

Real bug 4: .buttonStyle(.plain) only hits the glyph

The pen bar uses .plain so icons keep their own colours. On macOS a plain button's hit area is the drawn content. The accessibility frame of our Undo button was 12×12 pt inside a 30×30 frame, and the centre of arrow.uturn.backward is empty space. XCUITest clicks the centre, so nothing happened. A human hits it some of the time.

Button(action: undo) {
    Image(systemName: "arrow.uturn.backward")
        .frame(width: 30, height: 30)
        .contentShape(.rect)   // the whole square is the target
}
.buttonStyle(.plain)

In the same change we also took the bar's decorative capsule border out of hit testing (.allowsHitTesting(false)), so we can't say which of the two mattered more. Both are cheap; do both.

Real bug 5: ⌘V never reached the notebook

We had a Paste button in an Insert menu with .keyboardShortcut("v"). On macOS the main menu gets key equivalents first, and Edit ▸ Paste found no responder that could paste, so our shortcut never fired. The Mac way is to answer the standard command:

.onPasteCommand(of: [.image, .pdf, .fileURL, .plainText]) { _ in pasteFromClipboard() }

That only works when something in the view has focus, so the canvas also takes focus when the page opens (.focusable(interactions: .edit) plus @FocusState).

Real bug 6: the sign-in sheet had nowhere to appear

Not found by the suite. Google sign-in on Mac showed nothing. ASWebAuthenticationSession asks its context provider for a presentation anchor, and ours returned ASPresentationAnchor(). On iOS that's a fresh UIWindow, which happened to work. On macOS it's a fresh NSWindow that is never on screen. Return the app's key window instead. (The fix is in TestFlight; we're waiting on the first real sign-in to confirm.)

The test bugs

Failure	Actual cause
Deleted notebook "still in the library"	app.menuItems["Delete"] matched the menu bar's disabled Edit ▸ Delete first. Filter with isEnabled == true.
Can't click "Create" in a dialog	The Touch Bar exposes a "Create" button too, and it can't be clicked. Query app.sheets.buttons or app.dialogs.buttons.
"Timed out while evaluating UI query"	descendants(matching: .any).matching(predicate) over a page canvas and menu bar. Query typed elements (staticTexts, textViews, buttons) instead.
Word import "didn't open"	It did. The page counter's text is in value, not label.
Stroke count unreadable	macOS doesn't expose .accessibilityValue on a generic element. We moved the count into the label: "Page, 2 strokes".

What helped most

1. Screen recordings. XCUITest attached an .mp4 per test to the .xcresult. One frame with ffmpeg -sseof -3 usually showed the answer. Search, Summarize and Word import were all visibly working while their tests failed.
2. A probe test. A throwaway test that logs app.debugDescription and the state before/after one action found the undo and pen bar bugs in two runs.
3. A separate bundle id for tests. The Mac suite launches com.mindscend.mindpage.uitest, ad-hoc signed with its own sandbox, so -resetLibrary never touches a real library.

About MindPage

MindPage is a local-first notes app for handwriting and PDFs: sync through your own Google Drive or iCloud Drive, AI on the device. Search runs on our own model, MindPage Search v2, fine-tuned from EmbeddingGemma 2 on what Apple Vision really reads from handwriting (R@1 85.9 vs 80.3 for v1 on 234 Vision-read handwritten pages). iPhone/iPad 1.1 is submitted to App Review, Mac is in TestFlight, and the web app is at mindscend.academy.

More: mindscendinc.com/mindpage

Comments

[Arhan Canli]

On bug 6: ASPresentationAnchor is NSWindow on macOS, and as far as I know NSApp.keyWindow can be nil when the app is not the active application, for example when the sign-in flow starts from a menu command while another window or the browser has focus. A fallback chain like keyWindow, then the first visible window in NSApp.windows, then mainWindow avoids returning an anchor that is nil or off screen.

On bug 4, you could separate the two changes the way you separate the failing tests: revert only .allowsHitTesting(false) on the capsule and rerun the Undo test. If it still passes with contentShape alone, you know which one the 12x12 accessibility frame needed, and the other can be dropped or kept for a stated reason.

Has the first real Google sign-in on the TestFlight build come back yet?