Update Rust crate ort to v2.0.0-rc.13

[renovate]

This PR contains the following updates:

Package	Type	Update	Change
ort (source)	dependencies	patch	2.0.0-rc.10 → 2.0.0-rc.13

---

Release Notes

pykeio/ort (ort)

v2.0.0-rc.13

Compare Source

2.0.0-rc.13

💖 If you find ort useful, please consider sponsoring us on Open Collective 💖

🤔 Need help upgrading? Ask questions in GitHub Discussions or in the pyke.io Discord server!

---

🎯 Execution provider clarity

EP structs like ep::CUDA are now compile-time gated behind their respective Cargo feature flags. The EP feature flags only modifying runtime behavior was a common pain point, and this change brings ort in line with, like, every other Rust crate in existence.

⚠️ ort will now throw an error at link time if download-binaries is enabled and no binaries contain all of the requested EPs. Previously, if you enabled a nonsensical combination like cuda + coreml, ort would silently fall back to a CPU-only build. This caused great confusion as the only indicator of what went wrong was in log messages that don't show by default. A new lax-feature-matching Cargo feature will fallback to the closest fit instead of erroring.

🍎 ONNX Runtime 1.28

rc.13 skips ahead 4 ONNX Runtime versions to v1.28, bringing new operator support, bug & security fixes, and performance improvements.

✖️ No CUDA 12

rc.13 only ships CUDA 13 binaries, as ONNX Runtime has deprecated CUDA 12.

🧩 Custom operator ergonomics

Custom operators have been reworked to have a more Rusty interface:

 struct MyOperator;
 
 impl Operator for MyOperator {
+	type Kernel<'attr> = ort::operator::BoxedKernel<'attr>;
 
 	fn name(&self) -> &str {
 		"MyOperator"
 	}
 
-	fn inputs(&self) -> Vec<OperatorInput> {
-		vec![OperatorInput::required(TensorElementType::Float32), OperatorInput::required(TensorElementType::Float32)]
+	fn inputs(&self) -> impl IntoIterator<Item = OperatorInput> {
+		[OperatorInput::required(TensorElementType::Float32), OperatorInput::required(TensorElementType::Float32)]
 	}
 
-	fn outputs(&self) -> Vec<OperatorOutput> {
-		vec![OperatorOutput::required(TensorElementType::Float32)]
+	fn outputs(&self) -> impl IntoIterator<Item = OperatorOutput> {
+		[OperatorOutput::required(TensorElementType::Float32)]
 	}
 
-	fn create_kernel(&self, _: &KernelAttributes) -> ort::Result<Box<dyn Kernel>> {
-		Ok(Box::new(|ctx: &KernelContext| {
+	fn create_kernel<'attr>(&self, _: &KernelContext<'attr>) -> ort::Result<Self::Kernel<'attr>> {
+		Ok(Box::new(|ctx: &ComputeContext| {
 			...
 		}))
 	}
 }

ℹ️ Custom diagnostic messages

ort now uses the wonderful #[diagnostic] attributes to drastically improve the clarity of commonly encountered errors.

☑️ Miri-approved

Test coverage was expanded and ort was ran against Miri, and a few unsoundness bugs were fixed.

✨ Other features

• 8db51e7 Add VSINPU EP
• 5688cce (ort-web): Allow creating tensors from images.
• 726dc7c Expose the DirectML EP API.
• 5e669f6 Support string array operator attributes
• 5f99738 Support float16 tensor usage with the native f16 type on nightly Rust
• 62ad710 (ort-candle): Update candle to 0.11
• d5dc28f (ort-tract): Update tract to 0.23

🔧 Other fixes

• d5eb59f Committing a session which has a map input/output will no longer panic.
• d2838f8 Fixed the fallback behavior when ort-sys can't create a system-wide cache dir.
• 26f656b Allow DynTensor to be Cloned.
• e336d6b Allow 0 dimensions in tensors.
• 14e9d29 Fix tensor byte size calculation for 8-bit floating point types.
• 7bdabcc Allow try_extract_scalar to work on tensors with shape [1].
• 85f6074 Fixed retrieving string attributes in custom operator shape inference contexts.
• 17ed727 Don't deadlock when load-dynamic fails.
• a5ee3ad Don't force std for ndarray when std is enabled.
• d535605 Allow copy-dylibs to work independently of download-binaries.
• 49413ba Rerun build.rs on macOS when Xcode updates.
• bb20575 Warn when AVX-2 isn't supported when using pyke binaries.
• 9c840a3 Support FreeBSD.
• 01f377b Support Mac Catalyst.

---

❤️🧡💛💚💙💜

v2.0.0-rc.12

Compare Source

2.0.0-rc.12

💖 If you find ort useful, please consider sponsoring us on Open Collective 💖

🤔 Need help upgrading? Ask questions in GitHub Discussions or in the pyke.io Discord server!

---

This release was made possible by Rime.ai!

_{Authentic AI voice models for enterprise.}

---

📍 Multiversioning

🚨 If you used ort with default-features = false, enable the api-24 feature to use the latest features.

The big highlight of this release is multiversioning: ort can now use any minor version of ONNX Runtime from v1.17 to v1.24. New features are gated behind api-* feature flags, like api-20 or api-24. These flags will set the minimum version of ONNX Runtime required by ort.

More info 👉 https://ort.pyke.io/setup/multiversion

🪄 Automatic device selection

With ONNX Runtime 1.22 or later, ort will now automatically use an NPU if one is available for maximum efficiency & power savings! Setting your own execution providers will override this.

This is thanks to the super cool new SessionBuilder::with_auto_device API! There's also SessionBuilder::with_devices for finer control.

👁️ CUDA 13

ort now ships builds for both CUDA 12 & CUDA 13! It should automatically detect which CUDA you're using, but if it gets it wrong, you can override it by setting the ORT_CUDA_VERSION environment variable to 12 or 13.

🩹 SessionBuilder error recovery

You can now recover from errors when building a session by calling .recover() on the error type to get the SessionBuilder back.

🛡️ Build attestations

Prebuilt binaries are now attested via GitHub Actions, so you can verify that they are untampered builds of ONNX Runtime coming straight from pyke.io.

To verify, download your binary package of choice and use the gh CLI to verify:

➜  gh attestation verify --owner pykeio ./x86_64-pc-windows-msvc+cu13.tar.lzma2
Loaded digest sha256:e96616510082108be228ad6ea026246a31650b7d446b330c6b9671fcb9ae6267 for file://./x86_64-pc-windows-msvc+cu13.tar.lzma2
Loaded 1 attestation from GitHub API

The following policy criteria will be enforced:
- OIDC Issuer must match:................... https://token.actions.githubusercontent.com
- Source Repository Owner URI must match:... https://github.com/pykeio
- Predicate type must match:................ https://slsa.dev/provenance/v1
- Subject Alternative Name must match regex: (?i)^https://github.com/pykeio/

✓ Verification succeeded!

sha256:e96616510082108be228ad6ea026246a31650b7d446b330c6b9671fcb9ae6267 was attested by:
REPO                  PREDICATE_TYPE                  WORKFLOW
pykeio/ort-artifacts  https://slsa.dev/provenance/v1  .github/workflows/build-runner.yml@refs/heads/main

(Also note that the SHA-256 hash lines up with the one defined in dist.txt.)

---

Moving stuff around

• The ORT_LIB_LOCATION environment variable has been renamed to ORT_LIB_PATH.
  • Same with all other env vars ending in _LOCATION.
  • The old names will continue to work, but they won't make it into v2.0.0, so it's a good idea to change them now!
• Everything that used to be in ort::tensor is now in ort::value, because why have a tensor module if the Tensor<T> type actually comes from the value module?
• IoBinding and Adapter were moved from their own modules into ort::session. All sub-modules of ort::session besides builder were collapsed into ort::session.
• All sub-modules of ort::operator were collapsed into ort::operator.
• Session option changes:
  • with_denormal_as_zero -> with_flush_to_zero
  • with_device_allocator_for_initializers -> with_device_allocated_initializers

Fixes

• c52bd2a Fix MIGraphX registration.
  • #​512 missed a spot, thank you IntQuant =)
• 374a9d1 Fix global environment thread pools
• ff08428 Fix a segfault in Tensor::clone.
• 3d6c2a9 Use new API to load the DirectML EP.
• 5913ae0 Make vcpkg builds work again.
• 079ecb4 Fix issues with multiple environment registration.

---

❤️🧡💛💚💙💜

v2.0.0-rc.11

Compare Source

---

💖 If you find ort useful, please consider sponsoring us on Open Collective 💖

🤔 Need help upgrading? Ask questions in GitHub Discussions or in the pyke.io Discord server!

---

I'm sorry it took so long to get to this point, but the next big release of ort should be, finally, 2.0.0 🎉. I know I said that about one of the old alpha releases (if you can even remember those), but I mean it this time! Also, I would really like to not have to do another major release right after, so if you have any concerns about any APIs, please speak now or forever hold your peace!

A huge thank you to all the individuals who have contributed to the Collective over the years: Marius, Urban Pistek, Phu Tran, Haagen, Yunho Cho, Laco Skokan, Noah, Matouš Kučera, mush42, Thomas, Bartek, Kevin Lacker, & Okabintaro. You guys have made these past rc releases possible.

If you are a business using ort, please consider sponsoring me. Egress bandwidth from pyke.io has quadrupled in the last 4 months, and 90% of that comes from just a handful of businesses. I'm lucky enough that I don't have to pay for egress right now, but I don't expect that arrangement to last forever. pyke & ort have been funded entirely from my own personal savings for years, and (as I'm sure you're well aware 😂) everything is getting more expensive, so that definitely isn't sustainable.

Seeing companies that raise tens of millions in funding build large parts of their business on ort, ask for support, and then not give anything back just... seems kind of unfair, no?

---

ort-web

ort-web allows you to use the fully-featured ONNX Runtime on the Web! This time, it's hack-free and thus here to stay (it won't be removed, and then added back, and then removed again like last time!)

See the crate docs for info on how to port your application to ort-web; there is a little bit of work involved. For a very barebones sample application, see ort-web-sample.

Documentation for ort-web, like the rest of ort, will improve by the time 2.0.0 comes around. If you ever have any questions, you can always reach out via GitHub Discussions or Discord!

Features

• 5d85209 Add WebNN & WASM execution providers for ort-web.
• #430 (💖 @​jhonboy121) Support statically linking to iOS frameworks.
• #433 (💖 @​rMazeiks) Implement more traits for GraphOptimizationLevel.
• 6727c98 Make PrepackedWeights Send + Sync.
• 15bd15c Make the TLS backend configurable with new tls-* Cargo features.
• f3cd995 Allow overriding the cache dir with the ORT_CACHE_DIR environment variable.
• 🚨 8b3a1ed Load the dylib immediately when using ort::init_from.
  • You can now detect errors from dylib loading and let your program react accordingly.
• 🚨 #484 (💖 @​michael-p) Update ndarray to v0.17.
  • This means you'll need to upgrade your ndarray dependency to v0.17, too.
• 0084d08 New ort::lifetime tracing target tracks when objects are allocated/freed to aid in debugging leaks.

Fixes

• 2ee17aa Fix a memory leak in IoBinding.
• 317be20 Don't store Environment as a static.
  • This fixes a mutex lock failed: Invalid argument crash on macOS when exiting the process.
• 466025c Fix unexpected CPU usage when copying GPU tensors.
• ecca246 Fix UB when extracting empty tensors.
• 22f71ba Gate the ArrayExtensions trait behind the std feature, fixing #![no_std] builds.
• af63cea Fix an illegal memory access on no_std builds.
• #444 (💖 @​pembem22) Fix Android link.
• 1585268 Don't allow sessions to be created with non-CPU allocators
• #485 (💖 @​mayocream) Fix load order when using cuda::preload_dylibs.
• c5b68a1 Fix AsyncInferenceFut drop behavior.

Misc

• Update ONNX Runtime to v1.23.2.
• The MSRV is now Rust 1.88.
• Binaries are now compressed using LZMA2, which reduces bandwidth by 30% compared to gzip but may double the time it takes to download binaries for the first time.
  • If you use ort in CI, please cache the ~/.cache/ort.pyke.io directory between runs.
• ort's dependency tree has shrunk a little bit, so it should build a little faster!
• b68c928 Overhaul build.rs
  • Warnings should now appear when binaries aren't available, and errors should look a lot nicer.
  • pkg-config support now requires the pkg-config feature.
• 🚨 d269461 Make Metadata methods return Option<T> instead of Result<T>.
• 🚨 47e5667 Gate preload_dylib and cuda::preload_dylibs behind a new preload-dylibs feature flag instead of load-dynamic.
• 🚨 3b408b1 Shorten execution_providers to ep and XXXExecutionProvider to XXX.
  • They are still re-imported as their old names to avoid breakage, but these re-imports will be removed and thus broken in 2.0.0, so it's a good idea to change them now.
• 🚨 38573e0 Simplify ThreadManager trait.

ONNX Runtime binary changes

• Now shipping iOS & Android builds!!! Thank you Raphael Menges!!!
• Support for Intel macOS (x86_64-apple-darwin) has been dropped following upstream changes to ONNX Runtime & Rust.
  • Additionally, the macOS target has been raised to 13.4.
  • This means I can't debug macOS issues in my Hackintosh VM anymore, so expect little to no macOS support in general from now on.
• ONNX Runtime is now compiled with --client_package_build, meaning default options will optimize for low-resource edge inference rather than high throughput.
  • This currently only disables spinning by default. For server deployments, re-enable inter- and intra-op spinning for best throughput.
• Now shipping TensorRT RTX builds on Windows & Linux!
• x86_64 builds now target x86-64-v3, aka Intel Haswell/Broadwell and AMD Zen (any Ryzen) or later.
• Linux builds are now built with Clang instead of GCC.
• Various CUDA changes:
  • Kernels are now shipped compressed; this saves bandwidth & file size, but may slightly increase first-run latency. It will have no effect on subsequent runs.
  • Recently-added float/int matrix multiplication kernels aren't enabled. Quantized models will miss out on a bit of performance, but it was impossible to compile these kernels within the limitations of free GitHub Actions runners.

ort-tract

• Update tract to 0.22.
• 2d40e05 ort-tract no longer claims it is ort-candle in ort::info().

ort-candle

• Update candle to 0.9.

---

❤️🧡💛💚💙💜

---

Configuration

📅 Schedule: (UTC)

• Branch creation
  • At any time (no schedule defined)
• Automerge
  • At any time (no schedule defined)

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

---

• If you want to rebase/retry this PR, check this box

---

This PR was generated by Mend Renovate. View the repository job log.