[CI] Parity auto-trigger: collect scheduled architectures including Preview - #3397
Closed
ethanwee1 wants to merge 4 commits into
Closed
[CI] Parity auto-trigger: collect scheduled architectures including Preview#3397ethanwee1 wants to merge 4 commits into
ethanwee1 wants to merge 4 commits into
Conversation
Auto-parity was trunk-scoped (mi350 only). mi300/mi200/navi31 run in their own scheduled upstream workflows on their own SHAs at a different cadence, so the trunk-push scan never reached them. - Discover candidate SHAs from both trunk.yml pushes and the scheduled per-arch workflows (fetch_scheduled_commits / fetch_candidate_commits), deduped newest-first; a 40-hex guard drops stray non-row output. - Hold back SHAs newer than the newest scheduled run so a lagging mi300/mi200/navi batch can still join that SHA's report, yielding ONE combined parity report per SHA (parity.yml's matrix already emits a per-arch artifact plus a merged summary) once every arch that ran has finished. When no scheduled runs exist, nothing is held (mi350/trunk behaves exactly as before). - Expand ARCHS_IN to mi350 mi300 mi200 navi31. - Per-config CUDA-baseline gating: a SHA whose CUDA jobs for a test config did not run (e.g. a failed trunk run that never launched CUDA default) has no baseline for that config, so exclude just that config from the dispatch (via parity.yml's exclude_* inputs) instead of emitting a bogus all-MISSED column, and drop its ROCm shards from the completion gate. No parity_job_config.json change: mi300 has no fallback; its default/ distributed/inductor come from rocm-mi300 / periodic-rocm-mi300 / inductor-rocm-mi300, which run together on one scheduled SHA.
|
Jenkins build for 639f16e2d55b12f21a3892f7887fbfcda85c0306 commit finished as FAILURE |
Shard counts were hardcoded (config value or literals), so when an arch falls back to a different workflow whose sharding differs, the constructed "(config, i, N)" job/artifact keys miss the real jobs. Concretely mi200's default/inductor workflows are dormant upstream and fall back to trunk-rocm-sandbox, which shards default into 10 (config says 6) and inductor into 4 (config says 2) - so every key "DOES NOT EXIST IN JOBS" and the mi200 shard of a combined auto report fails. Add derive_shard_count(), which reads the actual shard total from the resolved run's job names, and use it for ROCm default/distributed/inductor (falling back to the config value only when no matching jobs are found). This unblocks mi200 in the multi-arch auto-trigger.
|
Jenkins build for 6b0eed6a8fa73fb298d5c6372b61ba858fb374dc commit finished as ABORTED |
Collaborator
|
I'd like to wait for the changes where it looks at the yaml file to get the job names and shard counts, so this one will be more robust. |
5 tasks
Treat the daily rocm-preview lane as a scheduled architecture source and remove the retired nightly exclusion.
Stack pytorch#3397 on pytorch#3406 so scheduled Preview discovery reads the matching preview config on its own branch.
ethanwee1
changed the title
ethanwee1
changed the base branch from
develop
to
ethanwee/parity-rocm-preview
August 11, 2026 14:59
Author
|
Superseded by clean current-develop Preview auto-trigger replacement #3555. The replacement keeps this PR's scheduled multi-arch behavior and adds daily rocm-preview discovery. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Expands auto-parity beyond trunk/MI350 and now includes the daily scheduled
rocm-previewlane. A SHA produces one combined report after every in-scope architecture that ran is ready.This PR is stacked on #3406 so its branch has the
previewconfig it consumes. After #3406 lands, retarget this PR todevelop.Changes
previewinARCHS_IN; remove the retirednightlyexclusion.Validation
parity-auto.ymlparses as YAMLrocm-previewLanding order
develop