feat: add OTLP metrics ingest endpoint (POST /api/public/otel/v1/metrics) - #2
Merged
Merged
Conversation
…ics) Co-Authored-By: pi li <songmqq@proton.me>
Contributor
Author
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
…useless_conversion) Co-Authored-By: pi li <songmqq@proton.me>
…lias conflicts (keep main's aliases) Co-Authored-By: pi li <songmqq@proton.me>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Roadmap item X0: add an OTLP metrics ingest endpoint so upstream services can push OTel metrics into xtrace over the same OTLP path they already use for traces (previously xtrace only accepted OTLP traces). New route
POST /api/public/otel/v1/metrics, symmetric topost_otel_traces, landing data into the existingmetricspoint table — no schema change / no migration.Transport mirrors the traces handler exactly (reuses
ungzip_if_needed, content-type branch, JSON via serde +application/x-protobufvia prostExportMetricsServiceRequest). Auth is symmetric too — the new path joins the Langfuse/open-compat allowance set alongside the traces endpoint.OTLP → point-model mapping (walk
resource_metrics → scope_metrics → metrics), reusing the existingmetrics_tx → metrics_worker → write_metrics_batcheswrite path (not the traceingest_tx):labelsmerge Resource + Scope (otel.scope.name/version) + DataPoint attributes; GenAI semantic-convention keys (gen_ai.*,service.name) are carried verbatim as label keys — semantics via naming convention only, no business-specific columns (perAGENTS.mdsubstrate boundary).Note on histograms: xtrace computes true quantiles over raw sample points (
percentile_cont), so the{name}_count/{name}_sumderived rows are an OTLP-input compatibility shim, not a source forp99— latency-type metrics should still be pushed as one raw point per request (X1 semantics note). A code comment records this.Also folds in a few preexisting clippy fixes required to get the
cargo clippy -- -D warningsgate green (a newer stable clippy, rust-1.92.0, promotedcollapsible_if/redundant_closure/type_complexityto errors in untouched files —src/http/metrics.rs,src/http/traces.rs,src/ingest/batch.rs); these are mechanical and behavior-preserving.Test
tests/integration_test.rs, mock + Postgres-backed viaoneshot): JSON OTLP metrics round-trip (Gauge + Sum) read back throughGET /api/public/metrics/query, protobuf round-trip, Histogram producing{name}_count/{name}_sum, and an auth-symmetry check. Plus a unit test locking the JSON mapping.cargo fmt --all --check,cargo clippy --all-targets --all-features -- -D warnings, andcargo test --allagainst Postgres 16 (DATABASE_URL=postgres://xtrace:xtrace@localhost:5432/xtrace).docs/project_status.mdsnapshot refreshed perAGENTS.md.Link to Devin session: https://app.devin.ai/sessions/0314a364b623468489f859563573e8e7
Requested by: @EeroEternal