Add POSIX telemetry

[bmehta001]

Summary

Adds cross-platform 1DS telemetry for ONNX Runtime on Linux, macOS, Android, and iOS while keeping the existing Windows ETW provider unchanged. WebAssembly builds remain excluded.

Behavior

• Standard native build scripts and official package builds enable telemetry with --use_telemetry; direct build.py/CMake callers may omit it to build telemetry out.
• CI and ORT unit-test binaries hard-suppress telemetry initialization.
• ORT_TELEMETRY_DISABLED and the telemetry APIs disable session, model, execution-provider, performance, and error events; a minimal ProcessInfo heartbeat may still be emitted outside CI/tests.
• 1DS supplies host application, OS, and device-class context. ORT adds library identity, process correlation, and caller-framework metadata.
• Linux/macOS use a product-salted hash of a locally generated per-user UUID. Android/iOS retain the 1DS platform device identity.

Build integration

• Uses the vcpkg cpp-client-telemetry port when available, with a pinned FetchContent fallback.
• Builds the SDK statically with size-reduced sqlite configuration and product-specific offline cache storage.
• Keeps the encoded in-repo ingestion token as the default; an optional CMake/environment override is written to a generated header, not a compiler command line or pipeline YAML.
• Registers dependencies in the component manifest and third-party notices.

Validation

• Telemetry-enabled Linux onnxruntime_common build with GCC 13.
• Android arm64 and Apple packaging paths validated during development/CI.
• VS2026 onnxruntime_test_all target build and lintrunner checks.
• Concurrent first-run device-id publication stress-tested with one stable winner across 32 processes.
• End-to-end event upload verified against the telemetry backend.

[github-actions]

You can commit the suggested changes from lintrunner.

[Copilot]

Pull request overview

This PR introduces cross-platform build support for telemetry by extending the existing Windows ETW telemetry to non-Windows platforms via the 1DS (cpp_client_telemetry) SDK, wiring it through the build scripts and CMake.

Changes:

• Adds a new POSIX telemetry provider implementation using the 1DS SDK and swaps it into the POSIX Env when enabled.
• Extends build and CMake plumbing to fetch/link cpp_client_telemetry on non-Windows and to expose a cross-platform --use_telemetry build flag.
• Updates top-level build wrappers and third-party notices for the new dependency.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 12 comments.

Show a summary per file

File	Description
tools/ci_build/build_args.py	Moves --use_telemetry to cross-platform args.
tools/ci_build/build.py	Always sets -Donnxruntime_USE_TELEMETRY=ON/OFF based on --use_telemetry.
onnxruntime/core/platform/windows/telemetry.cc	Include ordering tweak only.
onnxruntime/core/platform/posix/telemetry.h	Introduces POSIX telemetry provider interface (1DS).
onnxruntime/core/platform/posix/telemetry.cc	Implements POSIX telemetry provider using cpp_client_telemetry.
onnxruntime/core/platform/posix/env.cc	Uses PosixTelemetry provider when USE_1DS_TELEMETRY is defined.
cmake/onnxruntime_common.cmake	Adds POSIX telemetry sources/defines and links 1DS + system libs when enabled.
cmake/onnxruntime_1ds_telemetry.cmake	New helper module for enabling 1DS telemetry on non-Windows.
cmake/external/onnxruntime_external_deps.cmake	Fetches cpp_client_telemetry only when telemetry is enabled on non-Windows.
cmake/deps.txt	Adds cpp_client_telemetry dependency entry.
cmake/CMakeLists.txt	Includes the new 1DS telemetry CMake module.
build.sh	Now passes --use_telemetry by default.
build.bat	Now passes --use_telemetry by default.
ThirdPartyNotices.txt	Adds cpp_client_telemetry license text (and an additional KleidiAi block).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[github-actions]

You can commit the suggested changes from lintrunner.

[Copilot]

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 5 comments.

[Copilot]

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 5 comments.

[Copilot]

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 5 comments.

[Copilot]

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 6 comments.

[Copilot]

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 5 comments.

[Copilot]

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 7 comments.

[Copilot]

Pull request overview

Copilot reviewed 23 out of 23 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 23 out of 23 changed files in this pull request and generated 8 comments.

[ashrit-ms]

[tianleiwu]

Review: Add POSIX telemetry

Thanks for the thorough cross-platform telemetry work. The redaction and CI-suppression helpers are well-factored and well-tested, and the SDK lifetime handling (a shared_mutex around logger init/shutdown, register-count singleton, non-blocking teardown) is careful. A few items are worth addressing before merge, one of which is a consent/opt-out concern.

High priority

ORT_TELEMETRY_DISABLED / DisableTelemetryEvents() still transmit the ProcessInfo event. Initialize() sets enabled_ = false for the env-var opt-out but still creates the uploader and logger, and LogProcessInfo() gates only on logger_ != nullptr (not IsEnabled()). So a user who sets ORT_TELEMETRY_DISABLED=1 still uploads one ProcessInfo event carrying osDescription, processName, cpuModel, totalMemoryMB, and (on desktop) the hashed device id. README.md / docs/Privacy.md present this variable as a way to turn data collection off, which most users will read as "nothing is sent." Recommend either (a) fully suppressing ProcessInfo on the explicit opt-out (treat it like CI suppression and skip creating the uploader), or (b) documenting explicitly that a single non-PII ProcessInfo event is still emitted after opt-out. Inline note on the code path.

Suggestions

• UTF-8 truncation in ScrubStringForTelemetry: the final out.resize(kMaxTelemetryStringLength) can cut in the middle of a multi-byte UTF-8 sequence, leaving an invalid partial codepoint at the tail of the transmitted string. Consider backing up to a UTF-8 boundary.
• Schema drift vs. the Windows provider: on failure the POSIX helpers return "" / -1 while the new Windows helpers return "unknown" / 0 for the same fields (cpuModel, totalMemoryMB, processorCount); and ProcessInfo reports cpuCount via std::thread::hardware_concurrency() while the logger-context processorCount uses sysconf(_SC_NPROCESSORS_ONLN). Aligning sentinel values and sources keeps the backend schema consistent.
• Device-id write race across processes: open(..., O_WRONLY|O_CREAT|O_TRUNC) in DeviceId::InitializeInternal() lets two concurrent first-run processes each generate and truncate/write a different GUID. Both are valid, but the persisted id can flap on first run; an O_EXCL create-then-read-back (or write-temp + rename) would make first-run persistence deterministic.
• cmake/vcpkg.json drops the explicit utf8-range dependency. This looks unrelated to telemetry. Is it intentional (now pulled transitively by the new baseline's protobuf) or an accidental removal? Worth confirming it does not regress non-telemetry builds.

Nitpick

• friend class EventBuilder; in posix/telemetry.h appears unnecessary: EventBuilder does not reference projection_ (or any private member); projection_ is read directly in LogSessionCreation. It can likely be removed.

[github-actions]

You can commit the suggested changes from lintrunner.

[bmehta001]

@tianleiwu Thank you for the detailed review. I have addressed everything you mentioned.

Regarding utf8-range, yes, that was an intentional removal. As you said, pinning vcpkg/protobuf ensured that utf8-range will be pulled in transitively.

[tianleiwu]

Consolidated review — head 86d16ad

Re-reviewed the latest head. This PR has already been through several review rounds, so this round mostly re-confirms prior threads and adds only the genuinely new findings (inline below). Overall implementation quality is high: careful reader/writer locking around SDK teardown, defensive ORT_TRY/ORT_CATCH on every path, secure device-id file creation (open+fchmod before write, 0700 tree, hashed id only), and well-tested path redaction / CI-suppression helpers.

My earlier-round threads — status at this head

• ✅ UTF-8 truncation splitting codepoints (redaction) — fixed via TruncateUtf8AtBoundary + dedicated test.
• ✅ cpuCount vs processorCount mismatch — both now use GetProcessorCount() (sysconf).
• ✅ Unnecessary friend EventBuilder — removed.
• ⚠️ Opt-out asymmetry (ORT_TELEMETRY_DISABLED still writes a device-id file + uploads ProcessInfo) — replied and re-opened the existing thread: the code behavior is unchanged; only the docs were updated. Please confirm privacy sign-off or gate the on-disk write + init event behind the opt-out.

Already tracked by existing (resolved) threads — not re-posting to avoid duplicates, but flagging so they aren't lost before merge:

• Telemetry on-by-default via build.sh --use_telemetry for from-source builds (posture change vs. the old "no data collection for private builds").
• Hardcoded collector URL + obfuscated (base64+XOR) default tenant token embedded in every binary — worth a one-line comment stating it's a public, write-only ingestion key, not a secret.
• Repo-wide vcpkg baseline bump + new global overrides (mimalloc 2.1.1, benchmark 1.9.4, directx-headers 1.616.0) and utf8-range removal, which affect all builds (incl. Windows/DML). A vcpkg install --dry-run diff on a non-telemetry config would confirm no silent downgrade.

Nit (not worth an inline): ThirdPartyNotices.txt ends without a trailing newline, and a few YAML files carry whitespace-only churn unrelated to telemetry.

Verdict: COMMENT — no blocking correctness bugs; please close out the privacy-posture sign-off and the vcpkg blast-radius check before merge.

[tianleiwu]

Re-review — head 89f882a

Re-reviewed the Address telemetry review follow-ups commit against all my earlier threads. Every concern from my prior rounds is now resolved, and I verified each fix on the current head:

• RuntimePerf batch durations — AddBatchSizeDurations now serializes into a single bounded totalRunDurationPerBatchSize string, matching the Windows LogRuntimePerf format exactly (same <batch>: <duration>, ... layout and the same unordered_map source). Payload width is no longer unbounded.
• Device-ID persistence failure — LogProcessInfo() now emits a single ORT_TELEMETRY_WARN when DeviceIdStatus::Failed, correctly guarded to the desktop path (excluded on Android/iOS). Good for field diagnosability without adding noise.
• Windows memory width — GetTotalMemoryMB() and its local now use uint64_t, and the event uses TraceLoggingUInt64, matching the POSIX side.
• Local test suppression — centralized into onnxruntime::test::SuppressTelemetryForTests() and invoked at the top of every affected test entry point (unittest_main, global_thread_pools, onnx, perftest, ep_weight_sharing_ctx_gen, webgpu/external_dawn, fuzzing) before any Ort::Env is created, so local non-CI dev runs won't spin up the uploader.
• Opt-out asymmetry / on-by-default posture — accepting the author's rationale that the ProcessInfo heartbeat under ORT_TELEMETRY_DISABLED is intentional and has explicit privacy sign-off, as documented in Privacy.md.

The earlier UTF-8 truncation, cpuCount vs processorCount mismatch, and the stray friend declaration were also confirmed fixed in prior commits. No remaining code-level findings from my side; the concurrency/teardown design and privacy scrubbing remain solid.

[Copilot]

Pull request overview

Copilot reviewed 44 out of 44 changed files in this pull request and generated 3 comments.

[Copilot]

Pull request overview

Copilot reviewed 44 out of 44 changed files in this pull request and generated 3 comments.