Skip to content

fix(e2e): make CI green by retrying flaky Patrol native automation (Android) - #1097

Closed
Lyokone wants to merge 1 commit into
masterfrom
fix/e2e-android-patrol-retry
Closed

fix(e2e): make CI green by retrying flaky Patrol native automation (Android)#1097
Lyokone wants to merge 1 commit into
masterfrom
fix/e2e-android-patrol-retry

Conversation

@Lyokone

@Lyokone Lyokone commented Jul 22, 2026

Copy link
Copy Markdown
Owner

Summary

Gets the PR CI green. After tracing every red job on #1096, only one blocking job was actually failing the CI: the Android job in the e2e workflow.

The e2e run reported failure, but its iOS, Web, macOS and Windows jobs are all already continue-on-error: true (documented upstream Patrol / platform limitations) — their red ❌ don't fail the run. Linux passed. The location prepare workflow already reports success (its only red job, macOS, is likewise non-blocking). So the whole thing was red solely because the one remaining blocking mobile gate — Android — flaked.

Root cause

The Android job failed at:

grantPermissionWhenInUse() failed with Invalid response: 404
selector button to allow permission while using found nothing

That's Patrol's native UIAutomator not finding the Android system permission dialog button — a well-known transient emulator flake where the dialog hasn't finished rendering the instant Patrol polls for it. The app built and ran fine; the exact same test path passed on #1094 and failed on #1096 with no change to it. It's timing flake, not a regression.

Change

Wrap each patrol test invocation in the Android job's script in an inline retry (up to 3 attempts, 5s apart). A genuine failure still exit 1s after the retries, so real breakage stays red — this only absorbs the transient native-automation flake.

  • Validated the workflow YAML parses.
  • Validated the retry one-liner under sh -n and dry-ran the loop (3 attempts → exit 1 on persistent failure).
  • Kept each retry on a single physical line, per the existing per-line-split constraint of reactivecircus/android-emulator-runner documented in the job.

Not touched (deliberately)

The iOS / Web / macOS / Windows red ❌ are pre-existing, documented, and already non-blocking (continue-on-error), for genuine upstream/platform reasons (Patrol's macOS Package.swift bug, a missing iOS RunnerUITests target needing Xcode, a Windows headless-session permission limitation, an unidentified web patrol blocker). This PR intentionally leaves them as-is rather than papering over them.

The Android e2e job is the only *blocking* job in the e2e workflow (iOS,
Web, macOS and Windows are all already continue-on-error). It was the sole
reason the whole e2e run went red on #1096: Patrol's native UIAutomator
grantPermissionWhenInUse() intermittently returns "Invalid response: 404 --
selector button to allow permission while using found nothing" when the
Android system permission dialog hasn't finished rendering the instant
Patrol polls for it.

This is timing flake, not a regression: the identical test path passed on
#1094 and failed on #1096 with no change to it. Wrap each `patrol test`
invocation in an inline retry (up to 3 attempts, 5s apart). A genuine
failure still exits non-zero after the retries, so real breakage stays red.
@Lyokone
Lyokone force-pushed the fix/e2e-android-patrol-retry branch from c72d118 to 0a85baf Compare July 22, 2026 14:30
@Lyokone

Lyokone commented Jul 22, 2026

Copy link
Copy Markdown
Owner Author

Superseded: instead of retrying the flaky Patrol e2e suite, we're removing the slow e2e/patrol setup entirely and relying on the fast mockito unit tests + location-prepare build/analyze jobs. See the follow-up PR.

@Lyokone Lyokone closed this Jul 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant