VitisAI EP: "Failed to create runner" from a plain C++ app using the ONNX Runtime C/C++ API, while the identical model works fine via Python `onnxruntime`

[Looong01]

Environment

• Ryzen AI / VitisAI SDK: 1.7.1 (C:\Program Files\RyzenAI\1.7.1)
• NPU: Strix (STX), detected correctly by pnputil / AMD's own quicktest.py
• OS: Windows 11, x64
• ONNX Runtime: 1.23.3.dev20260320 (both the SDK-bundled copy under onnxruntime\bin and the conda onnxruntime pip package — same version, different file hashes, see below)
• xrt-smi examine -r aie-partitions confirms the NPU device and driver (amdxdna) are healthy and otherwise in use by other Windows processes without issue

Summary

A model that AMD's own vaip_core compiler successfully compiles into 275 separate AIE-executable subgraphs (confirmed via compiled.AMD_AIE2P_4x8_CMC_Overlay.xmodel written to the on-disk compile cache, and via ONNX Runtime's VerifyEachNodeIsAssignedToAnEp verbose log showing 275 vitis_ai_ep_N fused nodes correctly assigned to VitisAIExecutionProvider) fails at runtime when the session is created from a plain C++ executable using the standard ONNX Runtime C/C++ API (Ort::SessionOptions::AppendExecutionProvider_VitisAI / the generic AppendExecutionProvider("VitisAI", ...)).

The identical .onnx file, with the identical provider_options (config_file, cache_dir, enable_cache_file_io_in_mem=0), using the identical on-disk compile cache (so no recompilation happens — this reproduces even on a cache hit), works perfectly when loaded via Python's onnxruntime.InferenceSession — including when explicitly matching every other variable we could think of between the two: passing model bytes in-memory instead of a file path, setting intra_op_num_threads=1/inter_op_num_threads=1, using the exact same DLL set (see below), etc.

Error

F<timestamp> <tid> runner_requests_queue.cpp:178] -- Error: Failed to create runner: Failed to open library 'C:\Windows\System32\DriverStore\FileRepository\kipudrv.inf_amd64_7b0051e064968f34\xrt_core.dll'

This happens once per compiled subgraph (275 times for our model, matching the 275 vitis_ai_ep_N nodes exactly). With session.disable_cpu_ep_fallback left at its default (not forced on), each failing subgraph silently falls back to CPU execution instead of hard-failing the session — so "the session was created successfully" is not evidence anything ran on the NPU.

The error message is misleading — the DLL loads fine

We used Process Monitor (Sysinternals) to trace file-system/loader activity for the failing process, filtered to its PID. For every single one of the 275 failures, the corresponding xrt_core.dll load actually succeeds at the OS level:

Load Image  C:\Windows\System32\DriverStore\FileRepository\kipudrv.inf_amd64_7b0051e064968f34\xrt_core.dll  SUCCESS

We also independently confirmed via a direct Win32 LoadLibraryEx call (outside of any ONNX Runtime context) that this exact file loads without error. So whatever is actually failing happens after the library is mapped into the process — the error text is not describing the real failure.

The most suspicious finding: 275 reloads vs. Python's 1

Comparing Process Monitor traces of the failing C++ process against a successful Python run of the same model with the same provider_options:

	xrt_core.dll "Load Image" events
Python (onnxruntime.InferenceSession)	1 (for the whole session, all 275 subgraphs)
C++ (Ort::Session via C/C++ API)	275 (one per compiled subgraph — genuinely reloaded, not just ref-counted)

This strongly suggests something in the C API session-creation path is not reusing/caching a VitisAI-EP-internal handle to xrt_core.dll across the 275 compiled-subgraph nodes within one session the way the Python binding's path does, and that repeated load/unload cycling of the driver library is what's actually failing intermittently (perhaps a device-handle race, or the library's own internal state not tolerating being torn down and reinitialized 275 times in quick succession).

What we ruled out (in case useful for narrowing this down)

• Not a DLL version/ABI mismatch. We found (and initially suspected) that this system has at least 4 different builds of onnxruntime.dll/vaiml.dll/etc. floating around (SDK's onnxruntime\bin, SDK's deployment\, a separately-prepared external ORT package, and the conda onnxruntime pip package's bundled copies) — all different SHA256 hashes despite matching file sizes in some cases. We tried all four as the complete DLL set next to our C++ executable, including byte-for-byte the exact DLLs Python's package uses (which we'd already proven work). All four produce the identical 275-failure pattern. This rules out DLL source/version entirely.
• Not RYZEN_AI_INSTALLATION_PATH (your quicktest.py requires it; setting/unsetting it for our repro made no difference either way).
• Not the provider_options values (config_file, cache_dir, enable_cache_file_io_in_mem) — reproduced with Python using the exact same dict, still succeeds.
• Not vaiml.dll presence (docs say it's only needed for runtime BF16 compilation; our model is XINT8; Python's working DLL set doesn't even ship it).
• Not a missing xrt.ini/sdaccel.ini next to the exe (both our C++ exe and Python fail to find one in their own directory via identical CreateFile ... NAME NOT FOUND, and Python still succeeds regardless).
• Not intra_op_num_threads/inter_op_num_threads (matched our C++'s SetIntraOpNumThreads(1) in the Python repro, still succeeds).
• Not Windows Smart App Control / Code Integrity / AppLocker (VerifiedAndReputablePolicyState=0; zero related events in Microsoft-Windows-CodeIntegrity/Operational, AppLocker/EXE and DLL, Application logs).
• Not model-bytes-vs-file-path (Python still succeeds loading from an in-memory byte buffer, matching what the C++ API does internally).
• Not the specific EP-registration call (tried both the dedicated SessionOptionsAppendExecutionProvider_VitisAI C API and the generic name-based SessionOptionsAppendExecutionProvider("VitisAI", ...) — both C++-side paths produce the identical failure).

Question

What does the Python onnxruntime binding do differently from the raw C/C++ Ort::Session/OrtApi::SessionOptionsAppendExecutionProvider_VitisAI path that would cause vaip_core/xrt_core.dll to be reused once per session in one case but reloaded 275 times (once per compiled subgraph) in the other? Is there a session option, environment variable, or EP-context-cache setting (we noticed EP Context cache enabled: 0 in the verbose log — is enabling this relevant here, e.g. via ep.context_enable?) that would make the C API path share a single VitisAI runtime instance across all compiled subgraphs the way the Python path apparently does?

Attached diagnostics

• scratch_procmon_katago_filtered.csv (~2MB) — Process Monitor trace (file-system/loader events only, Load Image/CreateFile/Process Start operations plus anything matching xrt_core/kipudrv/xrt.ini/sdaccel.ini/runner) for the failing C++ process. Confirms 275x Load Image ... xrt_core.dll ... SUCCESS.
• scratch_procmon_python_filtered.csv (~330KB) — same filtering, for the succeeding Python process loading the identical model with identical provider_options. Confirms exactly 1x Load Image ... xrt_core.dll ... SUCCESS.
• (Both trimmed from full multi-GB system-wide captures down to just the relevant PID's file/loader activity, for attachment-size reasons — full captures available on request.)

Here it is:
scratch_procmon_katago_filtered.csv
scratch_procmon_python_filtered.csv

Not attached: the model itself. It's a large (~235MB), privately-trained Go-playing neural network (KataGo project), not something we can share publicly, and not necessary to reproduce the underlying issue — the failure is generic to any model that compiles into multiple (N > 1) separate VitisAI-fused subgraphs when the session is created via the C/C++ API rather than Python. If a minimal public repro would help narrow this down, we're happy to try building one from a small openly-licensed ONNX model, just let us know.