[CI] Parity: discover job topology from workflow YAML

[ethanwee1]

Summary

• download workflow YAML from raw.githubusercontent.com at the exact tested commit and read CUDA/ROCm num_shards values from the authoritative test matrix
• follow nested reusable workflows and build-job matrix outputs, while retaining configured counts only as a failure fallback
• self-heal renamed job prefixes and redirect artifact downloads to the workflow run that actually hosted reusable-workflow shards

Why parity_job_config.json remains

The workflow YAML is now authoritative for shard counts, but the JSON still supplies information that cannot yet be discovered from one resolved workflow file: the ordered workflow candidates and architecture-specific fallbacks, seed job prefixes used to locate the right family before self-healing, CUDA test-job kinds, and check-run/workflow regexes used by parity-auto.yml before dispatch. Removing the file now would require moving those hardcoded values elsewhere or separately redesigning workflow discovery and gating; it would not actually remove the configuration. A follow-up can remove the JSON once those remaining consumers derive their topology dynamically.

Test plan

• Validate CUDA trunk default (14) and distributed (10) against live workflow source
• Validate ROCm mi300 default (8) and sandbox inductor (4)
• Validate nested CUDA inductor workflow resolution (2)
• Run Python syntax and whitespace checks

Preview topology update

This branch now uses preview only and validates topology discovery against the scheduled rocm-preview mi350 8/3/2 lane. #3406 owns the canonical Preview config; reconcile the duplicate config lines when rebasing after #3406 lands. This PR remains the prerequisite for #3536 and #3397.

Prefix safety update

Ported the fork #6 safety fix in 71e3f00d61f: check-runs are scoped to the selected workflow and nested reusable runs, candidates are ROCm-only and architecture-filtered, and Preview uses a distinct rocm-preview token.

Validation covers Python compilation, live trunk/Preview YAML matrix resolution at upstream SHA cf18dda7, scoped check-run selection, and rejection of a same-SHA MI350 candidate when resolving Preview.

Alignment landing order

Depends on #3554. Land before #3536 and before downloader resilience/automation PRs. Prefix safety includes nested reusable-run scoping and architecture filtering; validated against live trunk and Preview workflow YAML at cf18dda7.

[jithunnair-amd]

Can we confirm that the job_key will have the value linux-jammy-rocm-py3_10-build and not linux-jammy-rocm-py3.10-mi350 in trunk.yml?

[ethanwee1]

Confirmed. _matrix_for_job_prefix matches either the displayed name or YAML key, but it deliberately carries (key, job) and passes key into _literal_test_matrix. For trunk, that means job_key is the YAML identifier such as linux-jammy-rocm-py3_10-build, while linux-jammy-rocm-py3.10-mi350 is only used to select that job by its displayed name. I exercised the updated helper against trunk.yml at upstream SHA cf18dda7; it followed the build job output and resolved a non-empty default matrix. I also added workflow-run and architecture scoping in 71e3f00 so same-SHA workflows cannot contribute an unrelated prefix.