feat(sinks): validate all sinks at config-compile time - #26048
feat(sinks): validate all sinks at config-compile time#26048thomasqueirozb wants to merge 40 commits into
Conversation
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
vector/src/sinks/azure_blob/config.rs
Line 287 in 19eef9d
For azure_blob configurations that provide a malformed connection_string, this branch just copies the secret into validated state, while ParsedConnectionString::parse and container URL parsing still run later in build_client. Because vector validate --no-environment skips that build path, these pure configuration errors continue to pass compile-time validation; parse and retain the validated connection/container URL state here, deferring only credential construction to build.
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0eddcfa6ff
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
…t validation stays pure
# Conflicts: # src/sinks/azure_blob/config.rs
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f7215b709c
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 346586c9cc
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c93366efea
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
…qs, gcp_chronicle
# Conflicts: # src/sinks/azure_logs_ingestion/tests.rs # src/sinks/azure_monitor_logs/tests.rs
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b8ca129e49
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1697d97b20
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| let (sink, healthcheck) = ElasticsearchConfig { | ||
| endpoints: vec![endpoint], | ||
| let index = | ||
| Template::try_from(self.token.inner()).expect("unable to parse token as Template"); |
There was a problem hiding this comment.
Return errors for malformed Sematext tokens
When sematext_logs.token contains invalid template syntax such as foo{{, this expect now runs during sink validation and panics the config compile/vector validate --no-environment path instead of reporting a validation error. Since Template::try_from already returns a Result, propagate that error before building the derived Elasticsearch config so bad user input cannot abort validation.
Useful? React with 👍 / 👎.
| // `ElasticsearchCommon::parse_many`). | ||
| self.common_mode()?; | ||
|
|
||
| let request_limits = self.request.tower.into_settings(); |
There was a problem hiding this comment.
Validate OpenSearch Serverless prerequisites
When elasticsearch uses opensearch_service_type = "serverless" without AWS auth/region, or with a non-auto api_version, this validation still retains success after only endpoint selection and routing-mode checks. ElasticsearchCommon::parse_config rejects those combinations before API-version probing or network I/O, but vector validate --no-environment now skips that build path; mirror the pure Serverless auth/region/API-version checks here.
Useful? React with 👍 / 👎.
| @@ -198,12 +236,41 @@ impl SinkConfig for RemoteWriteConfig { | |||
| .transpose()?; | |||
|
|
|||
| let endpoint = self.endpoint.parse::<Uri>().context(UriParseSnafu)?; | |||
There was a problem hiding this comment.
Require absolute remote-write endpoints
When prometheus_remote_write.endpoint is a relative URI such as /api/v1/write or lacks a scheme/host, http::Uri still parses successfully here, so validation retains an endpoint that the healthcheck/request path later hands to the HTTP client as a non-absolute target. Since the sink docs require the endpoint to include the scheme and write path, check the parsed URI has a scheme and authority during validation instead of accepting a config that deterministically fails when built or healthchecked.
Useful? React with 👍 / 👎.
Summary
Extends the validated sink lifecycle to all sinks and makes
DynValidatedSinka supertrait ofSinkConfig, removing the per-sinkas_dyn_validatedboilerplate. This also fixesvector validate --no-environmentto catch sink configuration errors (including template path-confinement violations) at config-compile time.Vector configuration
NA
How did you test this PR?
make check-clippyandcargo test -p vector --no-run --features sources-demo_logs.Is this a breaking change?
Does this PR include user facing changes?
no-changeloglabel to this PR.References