Add CPU-only MemcpyTransformer test with fake device EP#29568
Add CPU-only MemcpyTransformer test with fake device EP#29568GopalakrishnanN wants to merge 4 commits into
Conversation
Add a fake non-CPU execution provider to exercise MemcpyTransformer copy-node insertion without requiring CUDA. The test builds a CPU -> fake-device -> CPU graph, registers fake MatMul kernel metadata for graph rewrite, and asserts MemcpyFromHost and MemcpyToHost are inserted at the partition boundaries.
There was a problem hiding this comment.
Pull request overview
This PR adds a CPU-only unit test for MemcpyTransformer that validates copy-node insertion at mixed execution-provider boundaries without requiring CUDA. It does so by introducing a minimal fake non-CPU execution provider with a GPU-like OrtDevice, registering fake kernel metadata so MemcpyTransformer can query kernel placement info, and asserting that MemcpyFromHost/MemcpyToHost nodes are inserted at the CPU↔fake-EP boundaries.
Changes:
- Add a minimal
FakeDeviceExecutionProvider(GPU-like device) for testing partition-boundary copy insertion in CPU-only builds. - Add
TransformerTest.MemcpyTransformerTestWithFakeDeviceProviderto validateMemcpyFromHostandMemcpyToHostinsertion around a CPU → fake-EP → CPU graph. - Refactor
GetNotNullProviderPtrshelper so it’s available to the new non-CUDA-gated test.
Extend the fake-device MemcpyTransformer test helper so it can also run with the real DirectML execution provider when USE_DML is enabled. The DML variant uses the provider's registered kernel metadata and verifies MemcpyFromHost/MemcpyToHost insertion at CPU<->DML boundaries.
|
DirectML follow-up: after building a DML-enabled shared test tree on this Intel Arc machine, I extended this PR with a real DML variant in addition to the fake-device test. Validation: built onnxruntime_test_all in �uild\Windows_DML18_Shared and ran TransformerTest.MemcpyTransformerTestWithFakeDeviceProvider:TransformerTest.MemcpyTransformerTestWithDmlProvider — 2/2 passed. Also ran DML smoke QuantizeLinearOpTest.Uint8_5D_NoZeroPoint_Opset21_DML — passed. |
Description
Adds a CPU-only
MemcpyTransformertest that exercises copy-node insertion at mixed-EP partition boundaries without requiring CUDA.The new test introduces a tiny fake non-CPU execution provider whose default device is GPU-like, but it never executes kernels. The test:
MatMulkernel metadata for the fake provider soMemcpyTransformercan inspect kernel metadata during graph rewrite;MemcpyTransformer;MemcpyFromHostis inserted before the fake-device node and aMemcpyToHostis inserted before the downstream CPU node.This complements the existing CUDA-gated
MemcpyTransformertests with a locally runnable boundary-copy test.Motivation and Context
EP partitioning/copy behavior is a high-impact performance/design area. Previously, the direct
MemcpyTransformertests were guarded byUSE_CUDA, so CPU-only builds did not exercise this graph-rewrite logic. This test makes the core copy-insertion behavior verifiable without GPU hardware while still avoiding actual execution of fake-device kernels.Testing
Built
onnxruntime_test_all(Release, Windows) and ran:Result: