Adding DirectML fallback support in heuristics in EP - #84
Draft
urpetkov-amd wants to merge 1 commit into
Draft
Conversation
Zhaeong
approved these changes
Aug 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Move AMDGPU automatic DirectML fallback into the execution provider
Summary
This work moves the existing ModelBench-only automatic DirectML fallback policy
into the AMDGPU umbrella execution provider.
The new implementation:
automode.model_archprovider option.This is whole-session backend routing. It is not graph-level, node-level, or
per-operation fallback.
Motivation
The original fallback was implemented specifically in ModelBench:
list.
autotodirectml.Why the caller supplies the identity
The AMDGPU umbrella EP chooses and constructs one backend when the provider is
created. At that point ONNX Runtime has not yet exposed the model graph through
GetCapability().The EP therefore cannot inspect the complete graph before deciding whether its
kernel registry and implementation should come from MIGraphX or DirectML
without a substantially larger composite-EP redesign.
ModelBench already knows the model before session creation and owns the
registry containing its expected SHA256 checksum. It can supply a stable,
name-independent hint while the EP remains the owner of the actual routing
policy.
Model identity
For a registered ModelBench model:
ModelUrls.representing a 64-bit content signature.
automode, it sends that 16-character string through theexisting
model_archprovider option.model_arch_hash().(GPU architecture prefix, model_arch_hash)againstkArchModelBackend.The generated routing table performs the same FNV-1a calculation offline for
the same 16-character SHA prefix. Runtime and generated values therefore match
without adding a new provider option or a separate signature parser.
ModelBench changes
Remove name-based routing
The large
directMLFallbackModelstables and theshouldFallbackToDirectML(model_name, architecture)decision are removed fromthe runtime path.
ModelBench no longer changes
autotodirectmlbased on the model name.AMDGPU EP changes
Generated routing data
The generated header defines
kArchModelBackendas an array of:Each row contains:
gfxarchitecture prefix.model_arch.Profile::DirectX.Model names do not appear in the generated table or shipped EP binary.
Backend selection
select_backend()checks the generated table before applying generalarchitecture defaults. A matching row therefore reproduces the legacy
ModelBench fallback while keeping the decision inside the EP.