Skip to content

Add CPU-only MemcpyTransformer test with fake device EP#29568

Open
GopalakrishnanN wants to merge 4 commits into
mainfrom
GopalakrishnanN/memcpy-transformer-fake-ep-test
Open

Add CPU-only MemcpyTransformer test with fake device EP#29568
GopalakrishnanN wants to merge 4 commits into
mainfrom
GopalakrishnanN/memcpy-transformer-fake-ep-test

Conversation

@GopalakrishnanN

Copy link
Copy Markdown
Contributor

Description

Adds a CPU-only MemcpyTransformer test that exercises copy-node insertion at mixed-EP partition boundaries without requiring CUDA.

The new test introduces a tiny fake non-CPU execution provider whose default device is GPU-like, but it never executes kernels. The test:

  • builds a CPU -> fake-device -> CPU graph;
  • registers fake MatMul kernel metadata for the fake provider so MemcpyTransformer can inspect kernel metadata during graph rewrite;
  • applies MemcpyTransformer;
  • asserts a MemcpyFromHost is inserted before the fake-device node and a MemcpyToHost is inserted before the downstream CPU node.

This complements the existing CUDA-gated MemcpyTransformer tests with a locally runnable boundary-copy test.

Motivation and Context

EP partitioning/copy behavior is a high-impact performance/design area. Previously, the direct MemcpyTransformer tests were guarded by USE_CUDA, so CPU-only builds did not exercise this graph-rewrite logic. This test makes the core copy-insertion behavior verifiable without GPU hardware while still avoiding actual execution of fake-device kernels.

Testing

Built onnxruntime_test_all (Release, Windows) and ran:

.\onnxruntime_test_all.exe --gtest_filter="TransformerTest.MemcpyTransformerTestWithFakeDeviceProvider"

Result:

[==========] 1 test from 1 test suite ran.
[  PASSED  ] 1 test.

Add a fake non-CPU execution provider to exercise MemcpyTransformer copy-node insertion without requiring CUDA. The test builds a CPU -> fake-device -> CPU graph, registers fake MatMul kernel metadata for graph rewrite, and asserts MemcpyFromHost and MemcpyToHost are inserted at the partition boundaries.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a CPU-only unit test for MemcpyTransformer that validates copy-node insertion at mixed execution-provider boundaries without requiring CUDA. It does so by introducing a minimal fake non-CPU execution provider with a GPU-like OrtDevice, registering fake kernel metadata so MemcpyTransformer can query kernel placement info, and asserting that MemcpyFromHost/MemcpyToHost nodes are inserted at the CPU↔fake-EP boundaries.

Changes:

  • Add a minimal FakeDeviceExecutionProvider (GPU-like device) for testing partition-boundary copy insertion in CPU-only builds.
  • Add TransformerTest.MemcpyTransformerTestWithFakeDeviceProvider to validate MemcpyFromHost and MemcpyToHost insertion around a CPU → fake-EP → CPU graph.
  • Refactor GetNotNullProviderPtrs helper so it’s available to the new non-CUDA-gated test.

Extend the fake-device MemcpyTransformer test helper so it can also run with the real DirectML execution provider when USE_DML is enabled. The DML variant uses the provider's registered kernel metadata and verifies MemcpyFromHost/MemcpyToHost insertion at CPU<->DML boundaries.
@GopalakrishnanN

Copy link
Copy Markdown
Contributor Author

DirectML follow-up: after building a DML-enabled shared test tree on this Intel Arc machine, I extended this PR with a real DML variant in addition to the fake-device test. Validation: built onnxruntime_test_all in �uild\Windows_DML18_Shared and ran TransformerTest.MemcpyTransformerTestWithFakeDeviceProvider:TransformerTest.MemcpyTransformerTestWithDmlProvider — 2/2 passed. Also ran DML smoke QuantizeLinearOpTest.Uint8_5D_NoZeroPoint_Opset21_DML — passed.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can commit the suggested changes from lintrunner.

Comment thread onnxruntime/test/framework/memcpy_transformer_test.cc Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants