Skip to content

remotewrite/receiver: propose programmatic Receiver harness (mirrors sender) - #260

Open
om7057 wants to merge 1 commit into
prometheus:feature/rw2senderfixfrom
om7057:receiver-programmatic-poc
Open

remotewrite/receiver: propose programmatic Receiver harness (mirrors sender)#260
om7057 wants to merge 1 commit into
prometheus:feature/rw2senderfixfrom
om7057:receiver-programmatic-poc

Conversation

@om7057

@om7057 om7057 commented Aug 11, 2026

Copy link
Copy Markdown

What

Progresses #208 for the receiver side: this proposes converting remotewrite/receiver from the current config-file + env-var driven package main test binary into an importable, programmatic package, mirroring the Sender/RunTests/ComplianceTests pattern already landed for remotewrite/sender (#212).

As a proof of concept, this converts metric_test.go's three test functions (84 generated sub-cases) into the new declarative []Test{...} shape, and adds a Receiver interface + RunTests harness + a self-test (prometheus_test.go) that downloads a Prometheus release binary and runs the converted suite against it end-to-end. Verified locally: go test ./remotewrite/receiver/next/... passes all 84 sub-cases against both a locally-built and a downloaded release binary.

Why staged at remotewrite/receiver/next instead of replacing remotewrite/receiver

The remaining 9 files (combined_test.go, exemplar_test.go, histograms_test.go, metadata_test.go, request_test.go, rw1_compat_test.go, plus retiring main_test.go/helpers_test.go) still need conversion — roughly 2,300 more lines. Rather than doing that speculatively, I'd like early feedback on the API shape first, so this intentionally sits alongside the existing suite (which keeps working) rather than replacing it. If the design is accepted, I'll convert the rest and move next/ up to replace remotewrite/receiver.

Design questions for reviewers

  1. Lifecycle mismatch with sender. sender.RunTests restarts a fresh sender subprocess per test case (cheap — the mock scrape target/receiver are in-process). A receiver under test (e.g. a full Prometheus binary) is too costly to restart per case, and since these tests only assert on the synchronous HTTP response to each write (status + X-Prometheus-Remote-Write-*-Written headers), never a follow-up query, I start the receiver once for the whole RunTests call and send all cases at it sequentially — matching the current suite's TestMain-once behaviour. Open to alternatives if this is wrong.
  2. Multi-receiver support. The current suite tests against multiple receivers configured via a YAML file in one run (e.g. Mimir + Cortex + Prometheus side by side). This proposal only supports one Receiver per RunTests call, matching sender's shape. Should the caller just call RunTests once per receiver instead (simplest), or does something depend on the single-run multi-receiver behaviour?
  3. Readiness timeout: added a PROMETHEUS_RW2_COMPLIANCE_READY_TIMEOUT env var (default 3m) since a receiver may need to download/build a binary before responding — happy to align naming with sender's PROMETHEUS_RW2_COMPLIANCE_TEST_TIMEOUT if preferred.

Known blocker unrelated to this PR

remotewrite/receiver (the existing, untouched suite) currently fails go vet/go build on this branch as-is: ts.CreatedTimestamp undefined, because the pinned github.com/prometheus/prometheus (v0.307.4) predates writev2.TimeSeries start-timestamp support. There's already a dependabot branch (dependabot/go_modules/remotewrite/receiver/github.com/prometheus/prometheus-0.312.0) that should fix this — flagging in case it wasn't noticed, since it also means CreatedTimestamp is stubbed out (no-op) in this PR's request.go until that lands.

NONE

@om7057
om7057 marked this pull request as ready for review August 11, 2026 18:13
@bwplotka
bwplotka force-pushed the feature/rw2senderfix branch 2 times, most recently from ebdbd1f to 0380477 Compare August 12, 2026 10:05
@bwplotka

bwplotka commented Aug 12, 2026

Copy link
Copy Markdown
Member

Nice!

I just updated the rw2sender branch, so you might want to update.

But also we probably have to finish and get #256 merged so we can agree on the framework first. Will try to progress there

…etheus#208)

Ports the sender suite's programmatic pattern (Sender/RunTests/ComplianceTests,
see remotewrite/sender) to the receiver side, so receiver compliance tests can
be imported and run against a target Receiver implementation instead of being
driven by an external config file + env vars.

This converts metric_test.go's cases as a proof of concept (84 generated
sub-cases) and adds a self-test that runs them against a downloaded Prometheus
release binary. It intentionally lives at remotewrite/receiver/next rather
than replacing remotewrite/receiver outright, to avoid breaking the existing
suite while the design gets reviewed; if accepted, the remaining 9 files
(combined, exemplar, histograms, metadata, request, rw1_compat) would be
converted the same way and this would move up to replace remotewrite/receiver.

One notable design deviation from sender: RunTests starts the receiver once
per whole run rather than once per test case, since real receivers are too
costly to restart per case and these tests only assert on the synchronous
HTTP response to each write.

Also note: remotewrite/receiver currently fails `go vet` on this branch
(ts.CreatedTimestamp undefined) because the pinned prometheus/prometheus
version predates writev2 start-timestamp support; the pending dependabot bump
to v0.312.0 is a prerequisite for CreatedTimestamp-based cases.

Signed-off-by: om7057 <kulkarniom7057@gmail.com>
@om7057
om7057 force-pushed the receiver-programmatic-poc branch from e3aa09b to 6ba4486 Compare August 12, 2026 20:09
@om7057

om7057 commented Aug 12, 2026

Copy link
Copy Markdown
Author

Thanks for the heads up! Rebased onto the updated feature/rw2senderfix (on top of 9471697). No conflicts, since this only touches the new remotewrite/receiver/next directory.

Makes sense to wait for #256 to land first. I'll park this for now and pick it back up once things settle.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants