DEV Community

Cover image for You can't guard a value that doesn't exist yet
jaydeep bhatt
jaydeep bhatt

Posted on AI-assisted

You can't guard a value that doesn't exist yet

#ios #testing #debugging #opensource

A drag request with four ordinary numbers in it: fromX: 10, fromY: 700, toX: 10, toY: 300. It works in the first app. Switch to a second app, send the same request, and the phone goes sluggish, the call hangs, and eventually WebDriverAgent returns:

unknown error: Invalid parameter not satisfying: point.x != INFINITY && point.y != INFINITY
Enter fullscreen mode Exit fullscreen mode

None of those numbers is infinite.

That is appium/WebDriverAgent#1056, opened in September 2025 against WDA 9.15.3 on an iPhone 12 mini running iOS 18.5. The maintainers traced the error into Apple's own drag API and suggested reporting it to Apple or switching to W3C actions. The reporter filed it with Apple. Then it sat for a year.

I fixed it in PR #1270. It merged on 29 September 2026 and shipped in appium-webdriveragent 16.13.0 the next day. This is how, starting with the first fix I wrote, which was wrong.

fix: avoid infinite-point assertion and hang in coordinate drag gestures #1270

wilcoyay avatar
wilcoyay posted on

Summary

Fixes #1056: /wda/dragfromtoforduration (and /wda/pressAndDragWithVelocity) can hang for a long time and then fail with unknown error: Invalid parameter not satisfying: point.x != INFINITY && point.y != INFINITY when the target app is busy, e.g. right after switching applications.

Root cause

Both endpoints called -[XCUICoordinate pressForDuration:thenDragToCoordinate:(withVelocity:thenHoldForDuration:)]. XCTest resolves the coordinates' screen points lazily, inside its event dispatch (_dispatchEvent:eventBuilder:): it re-snapshots the application, waits for it to become idle, and only then calls screenPoint on both coordinates and passes them to its event builder (XCUIPressHoldDragWithVelocityAndHoldEvent), which asserts that neither point is infinite. When the app is busy, that snapshot is slow (the hang) and comes back unusable, so the resolved point is INFINITY (the assertion). WDA had no chance to validate anything, because the points only existed inside XCTest's own call.

Fix

A new -[XCUIApplication fb_pressAtCoordinate:forDuration:thenDragToCoordinate:withVelocity:thenHoldForDuration:error:] (in XCUIApplication+FBTouchAction), used by both endpoints:

  • Resolves both screen points once, up front, and returns a clear error if either is not finite ("Cannot resolve the on-screen position of the drag gesture … make sure it is idle and retry"). The points that are validated are exactly the ones that get sent.
  • Builds the same touch event path as XCTest's own builder: touch down, hold still for the press duration, move to the end point at the drag velocity, hold, lift. The velocity mapping comes from XCTest's own XCUIPixelsPerSecondForDragGestureVelocity (looked up at runtime), falling back to the values it currently returns (default 500, slow 250, fast 750).
  • Synthesizes the event directly via FBXCTestDaemonsProxy, the same way W3C actions do, so XCTest's second app snapshot and idle wait are skipped. After the gesture it waits up to animationCoolOffTimeout, like W3C actions.
  • Rejects zero/negative velocity and negative/non-finite durations with invalid argument (400). XCTest used to raise these as a 500 unknown error.

Behavior changes

  • There is no idle wait before the drag any more (that wait is where the hang came from); this matches W3C actions.
  • Invalid velocity or duration values now return 400 instead of 500.

Testing

  • New integration tests in FBElementSwipingTests.m (FBPressAndDragTests):
    • a drag between application coordinates scrolls the list;
    • a coordinate whose screenPoint is INFINITY fails cleanly with the new error and sends no gesture;
    • a velocity of 0 is rejected as invalid argument.
  • As a check, the old pressForDuration:thenDragToCoordinate: call with the same infinite coordinate raises exactly the error from the issue.
  • WebDriverAgentLib builds for iOS, tvOS and watchOS simulators; the new tests pass on an iPhone 18 Pro simulator (iOS 27.0).
  • Manual curl run on a simulator: drag in Settings, go to the home screen, open Photos, drag again; both succeed, and invalid velocity/duration return 400.

Not reproduced: the exact busy third-party app on a real iOS 18.5 device from the issue. It would be great if the reporter could try this branch.

View on GitHub

The first fix guarded the wrong place

WDA builds gesture coordinates in FBCoordinateWithAnchorOffset. It normalizes the client's offset against the element's wdFrame and returns an XCUICoordinate. An infinite value in a coordinate points straight at that function, so I hardened it with isfinite checks on the frame and on the computed offset.

It went nowhere, for three reasons:

  • The inputs in the issue are finite. The offset check never fires.
  • The frame was already guarded. wdFrame swaps an infinite frame for a sentinel rect (-1, -1, 0, 0), which the existing CGRectIsEmpty check rejects. My new check could only ever catch NaN.
  • The infinite point is created later, inside XCTest. A check that runs while the coordinate is being built cannot see a value that does not exist yet.

I dropped it. The third reason turned out to be the whole bug.

XCUICoordinate is a recipe, not a point

An XCUICoordinate stores "this element, plus this normalized offset". It has no screen position until something reads screenPoint, and reading it means resolving the element through an accessibility snapshot.

WDA handed XCTest two of these recipes and called:

[startCoordinate pressForDuration:duration thenDragToCoordinate:endCoordinate];
Enter fullscreen mode Exit fullscreen mode

Everything after that line is Apple's code, and the docs say nothing about what it does internally. So I disassembled XCUIAutomation.framework from Xcode 27.0 with otool -tV and followed the call chain. Inside _dispatchEvent:eventBuilder:, XCTest snapshots the app, waits for it to go idle, and only then resolves both points and hands them to its private event builder, XCUIPressHoldDragWithVelocityAndHoldEvent:

Sequence diagram of the old drag path. WebDriverAgent passes two lazy coordinates to XCTest. XCTest requests an accessibility snapshot from the busy app, gets a slow reply with no usable frame, and waits for idle, which is the hang. screenPoint then resolves to INFINITY, XCPointerEventPath asserts, and WebDriverAgent returns HTTP 500. testmanagerd is never reached.

After an app switch, the new app's main thread is saturated. The snapshot request crawls, which is the hang. The snapshot comes back without a usable frame, screenPoint resolves to {INFINITY, INFINITY}, the event builder asserts, and WDA's exception handler turns the NSInternalInconsistencyException into a 500.

WDA never saw a point. Both points were created and rejected inside a single XCTest call, after XCTest's own snapshot. There was no seam to validate them, time out sooner, or return a better error.

Three ways out, and why I took the expensive one

Option Fixes the hang Fixes the crash Cost
Catch XCTest's exception, return a friendly error No Cosmetically Users still sit through the hang
Pre-check screenPoint, then call XCTest No Partly XCTest re-resolves later, so it is a race, and every drag pays an extra snapshot
Resolve once, validate, synthesize the event directly Yes Yes, as a clean 400 Must match XCTest's gesture timing exactly

The first two are cheap and both keep the hang, which is the part users actually feel. The third has one real risk: timing drift from XCTest. Disassembly removed that risk, because I could copy Apple's builder line for line instead of approximating it.

The fix

A new method on XCUIApplication, fb_pressAtCoordinate:forDuration:thenDragToCoordinate:withVelocity:thenHoldForDuration:error:. Both /wda/dragfromtoforduration and /wda/pressAndDragWithVelocity now go through it.

Sequence diagram of the new drag path. WebDriverAgent resolves both screen points once through XCTest, which takes one snapshot. If either point is not finite, WebDriverAgent returns HTTP 400 immediately and sends nothing. Otherwise it builds the event with XCTest's exact touch path and the start element's display ID, sends it straight to testmanagerd with no re-snapshot or idle wait, and returns HTTP 200.

The core, trimmed from the merged code:

// Resolved once, here, so the points validated are exactly the points sent.
CGPoint startPoint = startCoordinate.screenPoint;
CGPoint endPoint = endCoordinate.screenPoint;
if (!FBIsFinitePoint(startPoint) || !FBIsFinitePoint(endPoint)) {
  return [[[FBErrorBuilder builder]
           withDescriptionFormat:@"Cannot resolve the on-screen position of the drag gesture ..."]
          buildError:error];
}
Enter fullscreen mode Exit fullscreen mode

And the event builder, mirroring XCTest's private one:

XCPointerEventPath *path = [[XCPointerEventPath alloc] initForTouchAtPoint:start offset:0];
if (pressDuration > 0) {
  [path moveToPoint:start atOffset:pressDuration];        // hold still
}
CGFloat distance = (CGFloat)hypot(end.x - start.x, end.y - start.y);
NSTimeInterval dragEnd = pressDuration + distance / pointsPerSecond;
[path moveToPoint:end atOffset:dragEnd];                  // drag
[path liftUpAtOffset:dragEnd + holdDuration];             // hold, lift
Enter fullscreen mode Exit fullscreen mode

The event goes out through FBXCTestDaemonsProxy, the same path W3C actions use. No second snapshot, no idle wait before the gesture. After the gesture it waits up to animationCoolOffTimeout, again matching W3C actions.

Two XCTest details found along the way shaped the code:

  • The velocity constants are not speeds. XCUIGestureVelocityDefault, Slow and Fast are sentinels: -10, -20, -30. XCTest maps them through an exported but undeclared function, XCUIPixelsPerSecondForDragGestureVelocity, to 500, 250 and 750. The fix looks that function up at runtime with dlsym, so a future Xcode change is picked up automatically, falls back to the measured values if it disappears, and logs once when it does.
  • "Pixels per second" is points per second. XCTest divides a distance measured in points by that value, with no scale factor.

What changed for clients:

Situation Before After
Busy app after a switch Long hang, then 500 unknown error Fast 400 invalid element state telling you the app was not idle, or the drag succeeds if the single resolution works
velocity: 0 500 unknown error, raised inside XCTest 400 invalid argument
Negative duration Passed straight through to XCTest 400 invalid argument
Normal drag Works Works

Making the test prove something

The simulator cannot reproduce a busy third-party app, so I built a stand-in for the broken state:

@interface FBUnresolvableCoordinate : XCUICoordinate
@end

@implementation FBUnresolvableCoordinate
- (CGPoint)screenPoint
{
  return CGPointMake(INFINITY, INFINITY);
}
@end
Enter fullscreen mode Exit fullscreen mode

A fake is only worth something if it breaks the old code the same way production did. So before writing the real test, I ran a throwaway one that fed this coordinate to the old XCTest API. It threw NSInternalInconsistencyException with the exact message from #1056. Only then did the fake become the regression test: the same coordinate through the new method returns NO, the expected error, and the list on screen has not moved, so no gesture was sent.

Four tests ended up in FBPressAndDragTests:

  • A real drag between app coordinates scrolls the list.
  • An unresolvable coordinate fails cleanly and sends nothing.
  • A velocity of 0 throws invalid argument.
  • The event builder keeps a non-default display ID and gets the timing right. This one came out of review.

On top of that: builds for iOS, tvOS and watchOS, and a manual curl replay of the issue on the simulator. Drag in Settings, go to the home screen, launch Photos, drag again. Both drags succeeded, and invalid velocity and duration returned 400.

Review: two maintainers, five comments

Round one, mykola-mokhnach, four comments. All code hygiene, in a codebase that builds with warnings as errors:

  • No float equality comparisons. A <= 0 check on a CGFloat had to go.
  • Throw a proper argument error where the value enters, instead of returning a 0 sentinel and relying on XCTest to complain.
  • Compute the preset comparisons once instead of twice.
  • Log when the dlsym lookup fails, so a renamed symbol does not silently change drag speeds.

Fixed in one commit, replied per comment, approved the same day.

Round two, KazuCocoa, one comment, and the one that mattered. XCTest's builder passes the start element's display ID into the event record. My version used the W3C path's initializer, which means display 0. On a secondary display, the gesture would land on the wrong screen. He had confirmed the difference by inspecting XCTest himself. The same release added multi-display screenshot support (#1267), so this would have been a real regression, not a theoretical one.

The fix kept the "resolve once" property intact. Disassembly showed -[XCUIElement displayID] just reads lastSnapshot.displayID without taking a new snapshot, and screenPoint had refreshed that snapshot a moment earlier. To test it without a second display, I pulled the event-record construction into its own class method and asserted on its output:

Gesture timeline used by the test. Press for 0.3 seconds, drag 400 points at 500 points per second for 0.8 seconds, hold for 0.1 seconds, lift at 1.2 seconds. The test asserts displayID equals 42, one event path, and maximumOffset equals 1.2.

Approved and merged on 29 September.

Six red CI jobs, none of them mine, each one proven

The final CI run: 51 jobs passed, 6 failed. From a first-time contributor, "unrelated failures" is the least credible sentence in open source, so I read every log before saying it:

  • Functional tests, Xcode 16.4. The simulator needed four boot attempts, the runner never answered /status, and the job was killed at the timeout. The same job passed on my first commit.
  • tvOS build and iPhone unit tests on the minimum Xcode. No device matched the destination specifier, exit 70, before any code ran.
  • One integration job on the maximum Xcode. The runner never bootstrapped. Zero tests ran.
  • Two int_test_3 jobs. Eight failures each, all in FBVoiceOverTests, which fail intermittently on unrelated branches too.

Every new test passed wherever the runner started, including Xcode 16.4 on iOS 18.5, the reporter's OS version.

What I did not test

I said this in the PR and it belongs here too:

  • The reporter's exact case. A busy third-party app on a real iOS 18.5 device. The fake coordinate covers it synthetically, not on hardware.
  • A real secondary display. Covered by the builder test only.
  • originalOffset. XCTest also sets it on the event record. The fix does not. No observed difference; still an open question.

And one known gap: /wda/element/:uuid/pressAndDragWithVelocity, the element-to-element variant, still calls XCTest directly and very likely has the same weakness. That is the next PR.

What carries over

None of this is specific to iOS.

  1. Find where the bad value is created before you guard against it. Validation at the edge of your code is useless when the value is born inside a dependency.
  2. Validate the thing you actually send. Resolve once, check it, send exactly that. Anything else is a race with a nicer error message.
  3. A fake proves nothing until it fails the old code the way production did. Reproduce the exact error first, then trust the test.
  4. When the docs stop, read the binary. A maintainer and I both settled design questions by disassembling XCTest instead of arguing about what it probably does.

Most of my work is mainframe-to-cloud migration testing, where rule 2 is the entire job: compare what the new system actually emits, not what you assume it emits. Same discipline, different stack.

Top comments (0)