From a8539e36a993599412de4f3bd49a7cd7bbb7c1eb Mon Sep 17 00:00:00 2001 From: Christian Date: Thu, 28 May 2026 14:24:08 -0500 Subject: [PATCH 1/5] Simplify JS asset proxy spec --- .../specs/2026-04-01-js-asset-proxy-design.md | 285 ++++++++++++++++++ 1 file changed, 285 insertions(+) create mode 100644 docs/superpowers/specs/2026-04-01-js-asset-proxy-design.md diff --git a/docs/superpowers/specs/2026-04-01-js-asset-proxy-design.md b/docs/superpowers/specs/2026-04-01-js-asset-proxy-design.md new file mode 100644 index 000000000..1e9a4871b --- /dev/null +++ b/docs/superpowers/specs/2026-04-01-js-asset-proxy-design.md @@ -0,0 +1,285 @@ +# JS Asset Proxy — Engineering Spec + +**Date:** 2026-04-01 +**Updated:** 2026-05-28 +**Status:** Proposed + +--- + +## Context + +Publishers often need to load JavaScript from third-party ad tech or measurement vendors. Those scripts are usually referenced directly from vendor-controlled domains, which means the publisher page depends on external script hostnames at runtime. + +The JS Asset Proxy gives Trusted Server a small, explicit way to serve configured third-party JavaScript files from first-party paths. Each proxied asset is declared in `trusted-server.toml`; at request time Trusted Server fetches the configured upstream URL and streams the response back to the browser with controlled response headers. + +This spec intentionally follows existing integration proxy patterns already used in Trusted Server. The implementation should be a focused integration-level proxy, not a new storage, build, or asset management subsystem. + +--- + +## Goals + +- Serve allowlisted third-party JavaScript assets from configured first-party paths. +- Keep configuration in `trusted-server.toml` under the existing `[integrations.*]` configuration model. +- Fetch only explicitly configured upstream URLs. +- Stream upstream JavaScript responses without server-side body transformation. +- Apply predictable downstream cache headers controlled by Trusted Server configuration. +- Reuse the existing integration registry and proxy request infrastructure. + +--- + +## Configuration + +Add a new integration configuration block: + +```toml +[integrations.js_asset_proxy] +enabled = false +cache_ttl_seconds = 3600 + +[[integrations.js_asset_proxy.assets]] +id = "vendor-loader" +path = "/assets/vendor-loader.js" +origin_url = "https://js.vendor.example.com/loader.js" + +[[integrations.js_asset_proxy.assets]] +id = "measurement-sdk" +path = "/assets/measurement-sdk.js" +origin_url = "https://cdn.vendor.example.com/sdk/measurement.js" +cache_ttl_seconds = 900 +``` + +### Fields + +| Field | Required | Description | +| ---------------------------- | -------: | ---------------------------------------------------------------- | +| `enabled` | Yes | Enables or disables the integration. | +| `cache_ttl_seconds` | No | Default downstream cache TTL for all assets. Defaults to `3600`. | +| `assets` | Yes | List of JavaScript assets the proxy may serve. | +| `assets[].id` | Yes | Stable identifier for logs, tests, and response diagnostics. | +| `assets[].path` | Yes | Exact first-party request path handled by Trusted Server. | +| `assets[].origin_url` | Yes | Exact upstream JavaScript URL to fetch. | +| `assets[].cache_ttl_seconds` | No | Per-asset downstream cache TTL override. | + +### Validation + +Configuration validation must reject: + +- enabled integration with malformed configured assets; +- empty `assets` when the integration is enabled; +- duplicate asset IDs; +- duplicate asset paths; +- asset paths that do not start with `/`; +- asset paths containing `*`; +- asset paths containing `..` path segments; +- `origin_url` values without an `https://` scheme; +- `origin_url` values with fragments; +- `cache_ttl_seconds = 0`. + +The implementation may use stricter validation if it keeps the configuration contract simple and documented. + +--- + +## Routing + +The integration registers one exact `GET` route per configured asset path using `IntegrationProxy::routes()`. + +Example registration from the configuration above: + +| Method | Path | Asset ID | Upstream URL | +| ------ | ---------------------------- | ----------------- | --------------------------------------------------- | +| `GET` | `/assets/vendor-loader.js` | `vendor-loader` | `https://js.vendor.example.com/loader.js` | +| `GET` | `/assets/measurement-sdk.js` | `measurement-sdk` | `https://cdn.vendor.example.com/sdk/measurement.js` | + +Only exact configured paths are handled. Paths not registered by the integration continue through the existing request dispatch behavior. + +The integration should rely on the existing integration registry duplicate-route checks so that an asset path cannot silently shadow another integration endpoint. + +--- + +## Request Flow + +For a matching request: + +1. Identify the configured asset by exact request path. +2. Build an upstream `GET` request to the asset's configured `origin_url`. +3. Use the existing proxy request infrastructure with streaming passthrough enabled. +4. Do not append EC IDs or any other per-user identifiers to the upstream URL. +5. Do not perform server-side JavaScript rewriting. +6. Finalize the response with the header policy below. + +### Proxy request configuration + +The JS asset proxy should use the same shape as existing script proxy integrations: + +```rust +let mut config = ProxyRequestConfig::new(origin_url) + .with_streaming() + .without_forward_headers(); +config.follow_redirects = false; +config.forward_ec_id = false; +``` + +The integration may forward this small request header allowlist from the browser request to the upstream request: + +- `Accept` +- `Accept-Language` +- `Accept-Encoding` + +It must set a fixed `User-Agent` such as `TrustedServer/1.0`. + +TLS verification follows the existing Trusted Server proxy backend policy used by `proxy_request()`. + +--- + +## Response Behavior + +### Successful upstream response + +For upstream `2xx` responses, Trusted Server streams the upstream body to the browser and constructs a response with only the headers needed for JavaScript delivery and diagnostics. + +Preserve these upstream response headers when present: + +- `Content-Type` +- `Content-Encoding` +- `ETag` +- `Last-Modified` +- `Vary` + +Set or override these response headers: + +```http +Cache-Control: public, max-age= +X-TS-JS-Asset-Proxy: true +X-TS-JS-Asset-ID: +``` + +If the response includes `Content-Encoding`, ensure `Vary` includes `Accept-Encoding` unless the upstream response uses `Vary: *`. + +Do not forward upstream `Set-Cookie` headers. + +### Upstream fetch failure + +If the upstream request cannot be completed, return: + +```http +502 Bad Gateway +X-TS-Error: js-asset-origin-unreachable +X-TS-JS-Asset-ID: +``` + +Log the asset ID and origin host at `warn` level. + +### Upstream non-success response + +If the upstream responds with a non-`2xx` status, return: + +```http +502 Bad Gateway +X-TS-Error: js-asset-origin-status +X-TS-JS-Asset-ID: +``` + +Log the upstream status, asset ID, and origin host at `warn` level. + +--- + +## Security Requirements + +- Fetch only the exact `origin_url` values declared in configuration. +- Do not accept user-provided upstream URLs at request time. +- Do not construct upstream hosts from request path segments. +- Do not forward cookies to the upstream JavaScript host. +- Do not forward upstream `Set-Cookie` headers to the browser. +- Do not forward `Referer` or `X-Forwarded-For` to the upstream JavaScript host. +- Do not append EC IDs or other Trusted Server identity values to asset requests. +- Require `https://` upstream URLs. + +--- + +## Implementation Plan + +### 1. Add integration configuration types + +Add a new `js_asset_proxy` integration module with typed config: + +```rust +#[derive(Debug, Clone, Deserialize, Serialize, Validate)] +pub struct JsAssetProxyConfig { + #[serde(default)] + pub enabled: bool, + #[serde(default = "default_cache_ttl_seconds")] + pub cache_ttl_seconds: u32, + #[serde(default)] + pub assets: Vec, +} + +#[derive(Debug, Clone, Deserialize, Serialize, Validate)] +pub struct JsAssetProxyAsset { + pub id: String, + pub path: String, + #[validate(url)] + pub origin_url: String, + pub cache_ttl_seconds: Option, +} +``` + +Implement `IntegrationConfig` for `JsAssetProxyConfig` and add custom validation for the rules in this spec. + +### 2. Register integration routes + +Add `crates/trusted-server-core/src/integrations/js_asset_proxy.rs` and register it from the integration builders list. + +`routes()` returns one exact `GET` endpoint per configured asset path. + +### 3. Implement request handling + +In `handle()`: + +1. Match `req.get_path()` to a configured asset. +2. Build the streaming proxy config. +3. Fetch the upstream response through `proxy_request()`. +4. Reject non-success upstream status codes. +5. Return a finalized response with the response header policy in this spec. + +### 4. Add sample disabled configuration + +Add a disabled sample block to `trusted-server.toml` using only `example.com` domains. + +--- + +## Files + +Expected code changes: + +- `crates/trusted-server-core/src/integrations/js_asset_proxy.rs` +- `crates/trusted-server-core/src/integrations/mod.rs` +- `trusted-server.toml` + +No adapter entry-point changes are expected if the existing integration registry dispatch is sufficient. + +--- + +## Verification + +Run the standard Rust verification for the changed integration code: + +```bash +cargo fmt --all -- --check +cargo test --workspace +cargo clippy --workspace --all-targets --all-features -- -D warnings +``` + +Add unit tests covering: + +- disabled config does not register routes; +- enabled config requires at least one asset; +- duplicate asset IDs are rejected; +- duplicate asset paths are rejected; +- invalid paths are rejected; +- non-HTTPS origins are rejected; +- exact configured routes are registered; +- request path selects the correct asset; +- upstream `2xx` response streams body and sets expected headers; +- upstream fetch failure returns `502` with `X-TS-Error: js-asset-origin-unreachable`; +- upstream non-success response returns `502` with `X-TS-Error: js-asset-origin-status`; +- `Set-Cookie`, `Referer`, `X-Forwarded-For`, and EC values are not forwarded. From 0ee438ab8584d7fa394d18f7888a6a17a7e5b762 Mon Sep 17 00:00:00 2001 From: Christian Date: Fri, 5 Jun 2026 11:09:58 -0500 Subject: [PATCH 2/5] update spec --- .../specs/2026-04-01-js-asset-proxy-design.md | 158 ++++++++++++------ 1 file changed, 110 insertions(+), 48 deletions(-) diff --git a/docs/superpowers/specs/2026-04-01-js-asset-proxy-design.md b/docs/superpowers/specs/2026-04-01-js-asset-proxy-design.md index 1e9a4871b..1315a1b62 100644 --- a/docs/superpowers/specs/2026-04-01-js-asset-proxy-design.md +++ b/docs/superpowers/specs/2026-04-01-js-asset-proxy-design.md @@ -23,6 +23,7 @@ This spec intentionally follows existing integration proxy patterns already used - Fetch only explicitly configured upstream URLs. - Stream upstream JavaScript responses without server-side body transformation. - Apply predictable downstream cache headers controlled by Trusted Server configuration. +- Allow configured assets to be individually proxied, disabled, or blocked from publisher HTML. - Reuse the existing integration registry and proxy request infrastructure. --- @@ -34,18 +35,27 @@ Add a new integration configuration block: ```toml [integrations.js_asset_proxy] enabled = false -cache_ttl_seconds = 3600 [[integrations.js_asset_proxy.assets]] -id = "vendor-loader" path = "/assets/vendor-loader.js" origin_url = "https://js.vendor.example.com/loader.js" +proxy = "enabled" [[integrations.js_asset_proxy.assets]] -id = "measurement-sdk" path = "/assets/measurement-sdk.js" origin_url = "https://cdn.vendor.example.com/sdk/measurement.js" +proxy = "enabled" cache_ttl_seconds = 900 + +[[integrations.js_asset_proxy.assets]] +path = "/assets/blocked-sdk.js" +origin_url = "https://cdn.vendor.example.com/sdk/blocked.js" +proxy = "blocked" + +[[integrations.js_asset_proxy.assets]] +path = "/assets/inactive-sdk.js" +origin_url = "https://cdn.vendor.example.com/sdk/inactive.js" +proxy = "disabled" ``` ### Fields @@ -53,12 +63,12 @@ cache_ttl_seconds = 900 | Field | Required | Description | | ---------------------------- | -------: | ---------------------------------------------------------------- | | `enabled` | Yes | Enables or disables the integration. | -| `cache_ttl_seconds` | No | Default downstream cache TTL for all assets. Defaults to `3600`. | +| `cache_ttl_seconds` | No | Optional downstream cache TTL override for all assets. When unset, preserve the upstream cache policy. | | `assets` | Yes | List of JavaScript assets the proxy may serve. | -| `assets[].id` | Yes | Stable identifier for logs, tests, and response diagnostics. | -| `assets[].path` | Yes | Exact first-party request path handled by Trusted Server. | -| `assets[].origin_url` | Yes | Exact upstream JavaScript URL to fetch. | -| `assets[].cache_ttl_seconds` | No | Per-asset downstream cache TTL override. | +| `assets[].path` | Yes | Stable identifier for logs, tests, and response diagnostics; exact first-party request path handled by Trusted Server. | +| `assets[].origin_url` | Yes | Exact upstream JavaScript URL to fetch or match for page rewriting. | +| `assets[].proxy` | No | Per-asset proxy behavior: `enabled`, `disabled`, or `blocked`. Defaults to `enabled`. | +| `assets[].cache_ttl_seconds` | No | Per-asset downstream cache TTL override. Takes precedence over the integration-level value. | ### Validation @@ -66,31 +76,43 @@ Configuration validation must reject: - enabled integration with malformed configured assets; - empty `assets` when the integration is enabled; -- duplicate asset IDs; - duplicate asset paths; +- duplicate `origin_url` values; - asset paths that do not start with `/`; - asset paths containing `*`; - asset paths containing `..` path segments; -- `origin_url` values without an `https://` scheme; -- `origin_url` values with fragments; -- `cache_ttl_seconds = 0`. +- `proxy` values other than `enabled`, `disabled`, or `blocked`; The implementation may use stricter validation if it keeps the configuration contract simple and documented. --- +## Asset Proxy Behavior + +Each asset has a `proxy` setting that controls both page rewriting and route registration: + +| Value | Behavior | +| ---------- | -------- | +| `enabled` | Rewrite exact matching ` + + + + "#; + + let processed = process_html_with_integration(html, integration); + + assert!(processed.contains(r#""#)); + assert!(processed.contains(r#""#)); + assert!(!processed.contains("blocked()")); + assert!(processed.contains(r#""#)); + } + + #[test] + fn rejects_duplicate_asset_paths() { + let config = config_with_assets(vec![ + asset( + "/assets/vendor.js", + "https://cdn.example.com/vendor-a.js", + JsAssetProxyMode::Enabled, + ), + asset( + "/assets/vendor.js", + "https://cdn.example.com/vendor-b.js", + JsAssetProxyMode::Enabled, + ), + ]); + + assert!( + config.validate().is_err(), + "duplicate asset paths should be rejected" + ); + } + + #[test] + fn rejects_duplicate_origin_urls() { + let config = config_with_assets(vec![ + asset( + "/assets/vendor-a.js", + "https://cdn.example.com/vendor.js", + JsAssetProxyMode::Enabled, + ), + asset( + "/assets/vendor-b.js", + "https://cdn.example.com/vendor.js", + JsAssetProxyMode::Enabled, + ), + ]); + + assert!( + config.validate().is_err(), + "duplicate origin URLs should be rejected" + ); + } + + #[test] + fn rejects_invalid_paths() { + for invalid_path in [ + "assets/vendor.js", + "//cdn.example.com/vendor.js", + "/assets/*.js", + "/assets/../vendor.js", + "/assets/{vendor}.js", + "/assets/vendor.js?v=1", + "/assets/vendor.js#v1", + "/assets/vendor js", + "/assets/vendor\n.js", + ] { + let config = config_with_assets(vec![asset( + invalid_path, + "https://cdn.example.com/vendor.js", + JsAssetProxyMode::Enabled, + )]); + + assert!( + config.validate().is_err(), + "path {invalid_path} should be rejected" + ); + } + } + + #[test] + fn rejects_non_https_origins() { + let config = config_with_assets(vec![asset( + "/assets/vendor.js", + "http://cdn.example.com/vendor.js", + JsAssetProxyMode::Enabled, + )]); + + assert!( + config.validate().is_err(), + "non-HTTPS origin should be rejected" + ); + } + + #[test] + fn rejects_unknown_proxy_mode() { + let toml = r#" + [[handlers]] + path = "^/secure" + username = "user" + password = "pass" + + [[handlers]] + path = "^/_ts/admin" + username = "admin" + password = "admin-pass" + + [publisher] + domain = "test-publisher.com" + cookie_domain = ".test-publisher.com" + origin_url = "https://origin.test-publisher.com" + proxy_secret = "unit-test-proxy-secret" + + [ec] + passphrase = "test-secret-key-32-bytes-minimum" + + [request_signing] + config_store_id = "test-config-store-id" + secret_store_id = "test-secret-store-id" + + [integrations.js_asset_proxy] + enabled = true + + [[integrations.js_asset_proxy.assets]] + path = "/assets/vendor.js" + origin_url = "https://cdn.example.com/vendor.js" + proxy = "passthrough" + "#; + let settings = Settings::from_toml(toml).expect("should parse settings TOML"); + + assert!( + settings + .integration_config::(JS_ASSET_PROXY_INTEGRATION_ID) + .is_err(), + "unknown proxy mode should fail deserialization" + ); + } + + #[test] + fn exact_configured_routes_are_registered() { + let mut settings = create_test_settings(); + settings + .integrations + .insert_config( + JS_ASSET_PROXY_INTEGRATION_ID, + &json!({ + "enabled": true, + "assets": [ + { + "path": "/assets/vendor.js", + "origin_url": "https://cdn.example.com/vendor.js" + }, + { + "path": "/assets/blocked.js", + "origin_url": "https://cdn.example.com/blocked.js", + "proxy": "blocked" + } + ] + }), + ) + .expect("should insert integration config"); + + let registry = IntegrationRegistry::new(&settings).expect("should build registry"); + + assert!(registry.has_route(&Method::GET, "/assets/vendor.js")); + assert!(!registry.has_route(&Method::GET, "/assets/vendor.js/extra")); + assert!(!registry.has_route(&Method::POST, "/assets/vendor.js")); + assert!(!registry.has_route(&Method::GET, "/assets/blocked.js")); + } + + #[test] + fn request_path_selects_the_correct_asset() { + let integration = JsAssetProxyIntegration::new(config_with_assets(vec![ + asset( + "/assets/a.js", + "https://cdn.example.com/a.js", + JsAssetProxyMode::Enabled, + ), + asset( + "/assets/b.js", + "https://cdn.example.com/b.js", + JsAssetProxyMode::Enabled, + ), + ])); + + let selected = integration + .enabled_asset_for_path("/assets/b.js") + .expect("should select configured asset"); + + assert_eq!(selected.origin_url, "https://cdn.example.com/b.js"); + } + + #[test] + fn successful_response_preserves_body_and_expected_headers() { + let mut configured_asset = asset( + "/assets/vendor.js", + "https://cdn.example.com/vendor.js", + JsAssetProxyMode::Enabled, + ); + configured_asset.cache_ttl_seconds = Some(900); + let integration = + JsAssetProxyIntegration::new(config_with_assets(vec![configured_asset.clone()])); + let upstream = Response::builder() + .status(StatusCode::OK) + .header(header::CONTENT_TYPE, "application/javascript") + .header(header::CONTENT_ENCODING, "gzip") + .header(header::ETAG, "\"asset-etag\"") + .header(header::LAST_MODIFIED, "Tue, 10 Jun 2026 00:00:00 GMT") + .header(header::VARY, "Origin") + .header(header::CACHE_CONTROL, "private, max-age=1") + .header(header::SET_COOKIE, "session=1") + .body(EdgeBody::from("console.log('ok');")) + .expect("should build upstream JS asset response"); + + let response = integration.finalize_asset_response(&configured_asset, upstream); + + assert_eq!(response.status(), StatusCode::OK); + assert_eq!( + response + .headers() + .get(HEADER_X_TS_JS_ASSET_PROXY) + .and_then(|value| value.to_str().ok()), + Some("true") + ); + assert_eq!( + response + .headers() + .get(header::CONTENT_TYPE) + .and_then(|value| value.to_str().ok()), + Some("application/javascript") + ); + assert_eq!( + response + .headers() + .get(header::CONTENT_ENCODING) + .and_then(|value| value.to_str().ok()), + Some("gzip") + ); + assert_eq!( + response + .headers() + .get(header::ETAG) + .and_then(|value| value.to_str().ok()), + Some("\"asset-etag\"") + ); + assert_eq!( + response + .headers() + .get(header::LAST_MODIFIED) + .and_then(|value| value.to_str().ok()), + Some("Tue, 10 Jun 2026 00:00:00 GMT") + ); + assert_eq!( + response + .headers() + .get(header::VARY) + .and_then(|value| value.to_str().ok()), + Some("Origin, Accept-Encoding") + ); + assert_eq!( + response + .headers() + .get(header::CACHE_CONTROL) + .and_then(|value| value.to_str().ok()), + Some("public, max-age=900") + ); + assert!( + response.headers().get(header::SET_COOKIE).is_none(), + "Set-Cookie should not be forwarded" + ); + } + + #[test] + fn preserves_upstream_cache_control_without_ttl_override() { + let configured_asset = asset( + "/assets/vendor.js", + "https://cdn.example.com/vendor.js", + JsAssetProxyMode::Enabled, + ); + let integration = + JsAssetProxyIntegration::new(config_with_assets(vec![configured_asset.clone()])); + let upstream = Response::builder() + .status(StatusCode::OK) + .header(header::CACHE_CONTROL, "public, max-age=123") + .body(EdgeBody::from("body")) + .expect("should build upstream JS asset response"); + + let response = integration.finalize_asset_response(&configured_asset, upstream); + + assert_eq!( + response + .headers() + .get(header::CACHE_CONTROL) + .and_then(|value| value.to_str().ok()), + Some("public, max-age=123") + ); + } + + #[test] + fn integration_cache_ttl_overrides_upstream_cache_control() { + let configured_asset = asset( + "/assets/vendor.js", + "https://cdn.example.com/vendor.js", + JsAssetProxyMode::Enabled, + ); + let mut config = config_with_assets(vec![configured_asset.clone()]); + config.cache_ttl_seconds = Some(300); + let integration = JsAssetProxyIntegration::new(config); + let upstream = Response::builder() + .status(StatusCode::OK) + .header(header::CACHE_CONTROL, "private, max-age=1") + .body(EdgeBody::from("body")) + .expect("should build upstream JS asset response"); + + let response = integration.finalize_asset_response(&configured_asset, upstream); + + assert_eq!( + response + .headers() + .get(header::CACHE_CONTROL) + .and_then(|value| value.to_str().ok()), + Some("public, max-age=300") + ); + } + + #[test] + fn upstream_error_responses_have_expected_headers() { + let unreachable = JsAssetProxyIntegration::origin_unreachable_response(); + assert_eq!(unreachable.status(), StatusCode::BAD_GATEWAY); + assert_eq!( + unreachable + .headers() + .get(HEADER_X_TS_ERROR) + .and_then(|value| value.to_str().ok()), + Some(ERROR_ORIGIN_UNREACHABLE) + ); + + let origin_status = JsAssetProxyIntegration::origin_status_response(); + assert_eq!(origin_status.status(), StatusCode::BAD_GATEWAY); + assert_eq!( + origin_status + .headers() + .get(HEADER_X_TS_ERROR) + .and_then(|value| value.to_str().ok()), + Some(ERROR_ORIGIN_STATUS) + ); + } + + #[test] + fn build_proxy_config_forwards_only_asset_header_allowlist() { + let mut req = build_http_request( + Method::GET, + "https://publisher.example.com/assets/vendor.js", + ); + req.headers_mut().insert( + HEADER_ACCEPT.clone(), + http::HeaderValue::from_static("application/javascript"), + ); + req.headers_mut().insert( + HEADER_ACCEPT_LANGUAGE.clone(), + http::HeaderValue::from_static("en-US"), + ); + req.headers_mut().insert( + HEADER_ACCEPT_ENCODING.clone(), + http::HeaderValue::from_static("gzip, br"), + ); + req.headers_mut().insert( + HEADER_REFERER.clone(), + http::HeaderValue::from_static("https://publisher.example.com/page"), + ); + req.headers_mut().insert( + HEADER_X_FORWARDED_FOR.clone(), + http::HeaderValue::from_static("192.0.2.10"), + ); + req.headers_mut().insert( + HEADER_X_TS_EC.clone(), + http::HeaderValue::from_static("edge-cookie-id"), + ); + req.headers_mut() + .insert(header::COOKIE, http::HeaderValue::from_static("session=1")); + + let config = + JsAssetProxyIntegration::build_proxy_config("https://cdn.example.com/vendor.js", &req); + + assert!(!config.copy_request_headers); + assert!(!config.follow_redirects); + assert!(!config.forward_ec_id); + + let forwarded: Vec<(String, String)> = config + .headers + .iter() + .map(|(name, value)| { + ( + name.as_str().to_string(), + value + .to_str() + .expect("should expose header value in test") + .to_string(), + ) + }) + .collect(); + + assert_eq!( + forwarded, + vec![ + ("accept".to_string(), "application/javascript".to_string()), + ("accept-language".to_string(), "en-US".to_string()), + ("accept-encoding".to_string(), "gzip, br".to_string()), + ("user-agent".to_string(), "TrustedServer/1.0".to_string()), + ] + ); + } + + #[test] + fn vary_with_accept_encoding_preserves_wildcard_and_existing_value() { + assert_eq!( + JsAssetProxyIntegration::vary_with_accept_encoding(Some("*")), + "*" + ); + assert_eq!( + JsAssetProxyIntegration::vary_with_accept_encoding(Some("Accept-Encoding")), + "Accept-Encoding" + ); + assert_eq!( + JsAssetProxyIntegration::vary_with_accept_encoding(Some("Origin")), + "Origin, Accept-Encoding" + ); + assert_eq!( + JsAssetProxyIntegration::vary_with_accept_encoding(None), + "Accept-Encoding" + ); + } + + #[test] + fn proxy_mode_defaults_to_enabled() { + let parsed: JsAssetProxyAsset = serde_json::from_value(json!({ + "path": "/assets/vendor.js", + "origin_url": "https://cdn.example.com/vendor.js" + })) + .expect("should deserialize asset"); + + assert_eq!(parsed.proxy, JsAssetProxyMode::Enabled); + } +} diff --git a/crates/trusted-server-core/src/integrations/lockr.rs b/crates/trusted-server-core/src/integrations/lockr.rs index 687f02a27..76fe505b9 100644 --- a/crates/trusted-server-core/src/integrations/lockr.rs +++ b/crates/trusted-server-core/src/integrations/lockr.rs @@ -456,6 +456,7 @@ mod tests { fn test_context() -> IntegrationAttributeContext<'static> { IntegrationAttributeContext { attribute_name: "src", + element_name: "script", request_host: "edge.example.com", request_scheme: "https", origin_host: "origin.example.com", diff --git a/crates/trusted-server-core/src/integrations/mod.rs b/crates/trusted-server-core/src/integrations/mod.rs index 3678f949c..84b3315b7 100644 --- a/crates/trusted-server-core/src/integrations/mod.rs +++ b/crates/trusted-server-core/src/integrations/mod.rs @@ -17,6 +17,7 @@ pub mod datadome; pub mod didomi; pub mod google_tag_manager; pub mod gpt; +pub mod js_asset_proxy; pub mod lockr; pub mod nextjs; pub mod osano; @@ -289,5 +290,6 @@ pub(crate) fn builders() -> &'static [IntegrationBuilder] { google_tag_manager::register, datadome::register, gpt::register, + js_asset_proxy::register, ] } diff --git a/crates/trusted-server-core/src/integrations/permutive.rs b/crates/trusted-server-core/src/integrations/permutive.rs index 3721207f4..b8d2e5eee 100644 --- a/crates/trusted-server-core/src/integrations/permutive.rs +++ b/crates/trusted-server-core/src/integrations/permutive.rs @@ -539,6 +539,7 @@ mod tests { let ctx = IntegrationAttributeContext { attribute_name: "src", + element_name: "script", request_host: "edge.example.com", request_scheme: "https", origin_host: "origin.example.com", @@ -572,6 +573,7 @@ mod tests { let ctx = IntegrationAttributeContext { attribute_name: "src", + element_name: "script", request_host: "edge.example.com", request_scheme: "https", origin_host: "origin.example.com", diff --git a/crates/trusted-server-core/src/integrations/prebid.rs b/crates/trusted-server-core/src/integrations/prebid.rs index d2bceb43a..f8d4d4845 100644 --- a/crates/trusted-server-core/src/integrations/prebid.rs +++ b/crates/trusted-server-core/src/integrations/prebid.rs @@ -1952,6 +1952,7 @@ passphrase = "test-secret-key-32-bytes-minimum" let integration = PrebidIntegration::new(base_config()); let ctx = IntegrationAttributeContext { attribute_name: "src", + element_name: "script", request_host: "pub.example", request_scheme: "https", origin_host: "origin.example", @@ -1969,6 +1970,7 @@ passphrase = "test-secret-key-32-bytes-minimum" let integration = PrebidIntegration::new(base_config()); let ctx = IntegrationAttributeContext { attribute_name: "href", + element_name: "a", request_host: "pub.example", request_scheme: "https", origin_host: "origin.example", diff --git a/crates/trusted-server-core/src/integrations/registry.rs b/crates/trusted-server-core/src/integrations/registry.rs index 103657e3e..60b4f9611 100644 --- a/crates/trusted-server-core/src/integrations/registry.rs +++ b/crates/trusted-server-core/src/integrations/registry.rs @@ -82,6 +82,7 @@ impl ScriptRewriteAction { #[derive(Debug)] pub struct IntegrationAttributeContext<'a> { pub attribute_name: &'a str, + pub element_name: &'a str, pub request_host: &'a str, pub request_scheme: &'a str, pub origin_host: &'a str, diff --git a/crates/trusted-server-core/src/integrations/sourcepoint.rs b/crates/trusted-server-core/src/integrations/sourcepoint.rs index 644d9da5b..122d76864 100644 --- a/crates/trusted-server-core/src/integrations/sourcepoint.rs +++ b/crates/trusted-server-core/src/integrations/sourcepoint.rs @@ -1123,6 +1123,7 @@ mod tests { let integration = SourcepointIntegration::new(Arc::new(config(true))); let ctx = IntegrationAttributeContext { attribute_name: "src", + element_name: "script", request_host: "edge.example.com", request_scheme: "https", origin_host: "origin.example.com", @@ -1147,6 +1148,7 @@ mod tests { let integration = SourcepointIntegration::new(Arc::new(config(true))); let ctx = IntegrationAttributeContext { attribute_name: "src", + element_name: "script", request_host: "edge.example.com", request_scheme: "https", origin_host: "origin.example.com", diff --git a/crates/trusted-server-core/src/integrations/testlight.rs b/crates/trusted-server-core/src/integrations/testlight.rs index 5e32cc89c..14f0afdbd 100644 --- a/crates/trusted-server-core/src/integrations/testlight.rs +++ b/crates/trusted-server-core/src/integrations/testlight.rs @@ -309,6 +309,7 @@ mod tests { let ctx = IntegrationAttributeContext { attribute_name: "src", + element_name: "script", request_host: "edge.example.com", request_scheme: "https", origin_host: "origin.example.com", @@ -338,6 +339,7 @@ mod tests { let integration = TestlightIntegration::new(config); let ctx = IntegrationAttributeContext { attribute_name: "src", + element_name: "script", request_host: "edge.example.com", request_scheme: "https", origin_host: "origin.example.com", diff --git a/trusted-server.example.toml b/trusted-server.example.toml index 0e8226efb..bd67600c6 100644 --- a/trusted-server.example.toml +++ b/trusted-server.example.toml @@ -96,6 +96,15 @@ script_url = "https://ads.example.com/gpt.js" cache_ttl_seconds = 3600 rewrite_script = true +[integrations.js_asset_proxy] +enabled = false +cache_ttl_seconds = 3600 + +[[integrations.js_asset_proxy.assets]] +path = "/assets/example-vendor-loader.js" +origin_url = "https://cdn.example.com/vendor-loader.js" +proxy = "enabled" + [proxy] # certificate_check = true # allowed_domains = ["ads.example.com", "*.cdn.example.com"] From e9a67050889d639065b9471cb92873899b6d9617 Mon Sep 17 00:00:00 2001 From: Christian Date: Mon, 22 Jun 2026 13:08:58 -0500 Subject: [PATCH 4/5] Add audit-generated JS asset proxy config --- Cargo.lock | 1 + crates/trusted-server-cli/Cargo.toml | 1 + crates/trusted-server-cli/src/audit.rs | 553 +++++++++++++++++- crates/trusted-server-core/src/config.rs | 73 ++- docs/guide/cli.md | 8 + docs/guide/getting-started.md | 4 +- .../specs/2026-04-01-js-asset-proxy-design.md | 24 +- ...2-ts-audit-js-asset-proxy-config-design.md | 360 ++++++++++++ 8 files changed, 1005 insertions(+), 19 deletions(-) create mode 100644 docs/superpowers/specs/2026-06-22-ts-audit-js-asset-proxy-config-design.md diff --git a/Cargo.lock b/Cargo.lock index 6e757bb5f..1daa3f5c7 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4360,6 +4360,7 @@ dependencies = [ "edgezero-cli", "futures", "log", + "rand 0.8.6", "regex", "scraper", "serde", diff --git a/crates/trusted-server-cli/Cargo.toml b/crates/trusted-server-cli/Cargo.toml index 189c5405a..b78cbaf20 100644 --- a/crates/trusted-server-cli/Cargo.toml +++ b/crates/trusted-server-cli/Cargo.toml @@ -19,6 +19,7 @@ clap = { workspace = true } edgezero-cli = { workspace = true } futures = { workspace = true } log = { workspace = true } +rand = { workspace = true } regex = { workspace = true } scraper = { workspace = true } serde = { workspace = true } diff --git a/crates/trusted-server-cli/src/audit.rs b/crates/trusted-server-cli/src/audit.rs index 5870b0de8..7f2684bbe 100644 --- a/crates/trusted-server-cli/src/audit.rs +++ b/crates/trusted-server-cli/src/audit.rs @@ -3,10 +3,13 @@ pub(crate) mod browser_collector; pub(crate) mod collector; use std::collections::BTreeSet; +use std::fmt::Write as _; use std::fs; use std::io::Write; use std::path::{Path, PathBuf}; +use rand::RngCore as _; + use serde::Serialize; use url::Url; @@ -60,6 +63,49 @@ pub(crate) struct AuditOutputs { pub(crate) artifact: AuditArtifact, pub(crate) js_assets_toml: String, pub(crate) draft_config_toml: String, + pub(crate) js_asset_proxy_candidate_count: usize, +} + +#[derive(Debug, Clone)] +struct DraftConfig { + toml: String, + js_asset_proxy_candidate_count: usize, +} + +#[derive(Debug, Clone)] +struct JsAssetProxySection { + toml: String, + candidate_count: usize, +} + +#[derive(Debug, Default)] +struct JsAssetProxySkipCounts { + first_party: usize, + malformed_url: usize, + non_https: usize, + duplicate_url: usize, + non_script: usize, +} + +#[derive(Debug)] +struct JsAssetProxyCandidate<'a> { + origin_url: String, + integration: Option<&'a str>, +} + +trait OpaqueAssetPathGenerator { + fn next_path(&mut self) -> String; +} + +#[derive(Debug, Default)] +struct RandomOpaqueAssetPathGenerator; + +impl OpaqueAssetPathGenerator for RandomOpaqueAssetPathGenerator { + fn next_path(&mut self) -> String { + let mut bytes = [0_u8; 12]; + rand::rngs::OsRng.fill_bytes(&mut bytes); + format!("/assets/{}.js", lowercase_hex(&bytes)) + } } #[derive(Debug, Clone, PartialEq, Eq)] @@ -153,12 +199,15 @@ fn build_audit_outputs(collected: &collector::CollectedPage) -> CliResult 0 { + format!( + "{} disabled entries written to draft config", + outputs.js_asset_proxy_candidate_count + ) + } else if wrote_config { + "none".to_string() + } else { + "not written (--no-config)".to_string() + }; writeln!( out, - "Audited {}\nTitle: {}\nJS assets: {}\nThird-party assets: {}\nDetected integrations: {}\nWrote: {}{}", + "Audited {}\nTitle: {}\nJS assets: {}\nThird-party assets: {}\nDetected integrations: {}\nJS asset proxy candidates: {}\nWrote: {}{}", outputs.artifact.audited_url, outputs .artifact @@ -237,6 +296,7 @@ fn write_success_summary( } else { integrations.join(", ") }, + asset_proxy_note, if written.is_empty() { "none".to_string() } else { @@ -247,7 +307,17 @@ fn write_success_summary( .map_err(|error| report_error(format!("failed to write command output: {error}"))) } +#[cfg(test)] fn build_draft_config(target_url: &Url, artifact: &AuditArtifact) -> CliResult { + let mut path_generator = RandomOpaqueAssetPathGenerator; + Ok(build_draft_config_with_generator(target_url, artifact, &mut path_generator)?.toml) +} + +fn build_draft_config_with_generator( + target_url: &Url, + artifact: &AuditArtifact, + path_generator: &mut dyn OpaqueAssetPathGenerator, +) -> CliResult { let host = target_url .host_str() .ok_or_else(|| report_error("audited URL is missing a host"))?; @@ -290,6 +360,9 @@ fn build_draft_config(target_url: &Url, artifact: &AuditArtifact) -> CliResult CliResult CliResult { + let (candidates, skipped) = select_js_asset_proxy_candidates(artifact); + let mut used_paths = BTreeSet::new(); + let mut toml = String::new(); + + toml.push_str("[integrations.js_asset_proxy]\n"); + toml.push_str("enabled = false\n"); + toml.push_str("cache_ttl_seconds = 3600\n\n"); + toml.push_str("# Generated by `ts audit`; review before enabling.\n"); + toml.push_str( + "# Audit note: some discovered scripts may be runtime-injected and may not appear\n", + ); + toml.push_str( + "# in origin HTML. JS Asset Proxy rewrites only exact script src values present in\n", + ); + toml.push_str("# HTML processed by Trusted Server.\n"); + + if candidates.is_empty() { + toml.push_str( + "# No eligible third-party HTTPS script assets were detected by `ts audit`.\n", + ); + } + + for candidate in &candidates { + let generated_path = generate_unique_asset_path(path_generator, &mut used_paths)?; + toml.push('\n'); + toml.push_str("# Generated by `ts audit`; review before enabling.\n"); + if let Some(integration) = candidate.integration { + let integration = sanitized_comment_value(integration); + toml.push_str(&format!("# Detected integration: {integration}\n")); + toml.push_str(&format!( + "# Native integration may be preferable: [integrations.{integration}]\n" + )); + } + toml.push_str("[[integrations.js_asset_proxy.assets]]\n"); + toml.push_str(&format!("path = {}\n", toml_quoted_string(&generated_path))); + toml.push_str(&format!( + "origin_url = {}\n", + toml_quoted_string(&candidate.origin_url) + )); + toml.push_str("proxy = \"disabled\"\n"); + } + + append_js_asset_proxy_skip_comments(&mut toml, &skipped); + toml.push('\n'); + + Ok(JsAssetProxySection { + toml, + candidate_count: candidates.len(), + }) +} + +fn select_js_asset_proxy_candidates( + artifact: &AuditArtifact, +) -> (Vec>, JsAssetProxySkipCounts) { + let mut candidates = Vec::new(); + let mut skipped = JsAssetProxySkipCounts::default(); + let mut seen_origin_urls = BTreeSet::new(); + + for asset in &artifact.assets { + if asset.kind != "script" { + skipped.non_script += 1; + continue; + } + if asset.party != AssetParty::ThirdParty { + skipped.first_party += 1; + continue; + } + + let Ok(url) = Url::parse(&asset.url) else { + skipped.malformed_url += 1; + continue; + }; + if url.host_str().is_none() { + skipped.malformed_url += 1; + continue; + } + if url.scheme() != "https" { + skipped.non_https += 1; + continue; + } + + let origin_url = url.to_string(); + if !seen_origin_urls.insert(origin_url.clone()) { + skipped.duplicate_url += 1; + continue; + } + + candidates.push(JsAssetProxyCandidate { + origin_url, + integration: asset.integration.as_deref(), + }); + } + + (candidates, skipped) +} + +fn generate_unique_asset_path( + path_generator: &mut dyn OpaqueAssetPathGenerator, + used_paths: &mut BTreeSet, +) -> CliResult { + for _ in 0..128 { + let path = path_generator.next_path(); + if !is_valid_generated_asset_path(&path) { + return cli_error(format!( + "generated JS asset proxy path `{path}` is invalid; expected /assets/.js" + )); + } + if used_paths.insert(path.clone()) { + return Ok(path); + } + } + + cli_error("failed to generate a unique JS asset proxy path after 128 attempts") +} + +fn is_valid_generated_asset_path(path: &str) -> bool { + let Some(opaque_id) = path + .strip_prefix("/assets/") + .and_then(|value| value.strip_suffix(".js")) + else { + return false; + }; + + !opaque_id.is_empty() + && opaque_id + .chars() + .all(|ch| ch.is_ascii_hexdigit() && !ch.is_ascii_uppercase()) +} + +fn replace_js_asset_proxy_section(document: &str, replacement: &str) -> CliResult { + let lines = document.lines().collect::>(); + let start = lines + .iter() + .position(|line| line.trim() == "[integrations.js_asset_proxy]") + .ok_or_else(|| { + report_error( + "failed to update starter config because section `[integrations.js_asset_proxy]` was not found", + ) + })?; + let mut end = start + 1; + + while end < lines.len() { + let trimmed = lines[end].trim(); + if trimmed.starts_with('[') + && trimmed.ends_with(']') + && trimmed != "[[integrations.js_asset_proxy.assets]]" + { + break; + } + end += 1; + } + + let mut output_lines = Vec::new(); + output_lines.extend_from_slice(&lines[..start]); + output_lines.extend(replacement.trim_end_matches('\n').lines()); + if end < lines.len() { + output_lines.push(""); + } + output_lines.extend_from_slice(&lines[end..]); + + let mut output = output_lines.join("\n"); + if document.ends_with('\n') { + output.push('\n'); + } + Ok(output) +} + +fn append_js_asset_proxy_skip_comments(toml: &mut String, skipped: &JsAssetProxySkipCounts) { + if skipped.first_party == 0 + && skipped.malformed_url == 0 + && skipped.non_https == 0 + && skipped.duplicate_url == 0 + && skipped.non_script == 0 + { + return; + } + + toml.push('\n'); + toml.push_str("# Skipped JS Asset Proxy audit candidates:\n"); + append_skip_count(toml, skipped.first_party, "first-party script"); + append_skip_count(toml, skipped.malformed_url, "malformed script URL"); + append_skip_count(toml, skipped.non_https, "non-HTTPS third-party script"); + append_skip_count(toml, skipped.duplicate_url, "duplicate script URL"); + append_skip_count(toml, skipped.non_script, "non-script asset"); +} + +fn append_skip_count(toml: &mut String, count: usize, label: &str) { + if count == 0 { + return; + } + + let plural = if count == 1 { "" } else { "s" }; + toml.push_str(&format!("# - {count} {label}{plural}\n")); +} + +fn sanitized_comment_value(value: &str) -> String { + value + .chars() + .map(|ch| if ch.is_control() { ' ' } else { ch }) + .collect() +} + +fn toml_quoted_string(value: &str) -> String { + let mut quoted = String::from("\""); + for ch in value.chars() { + match ch { + '\\' => quoted.push_str("\\\\"), + '"' => quoted.push_str("\\\""), + '\n' => quoted.push_str("\\n"), + '\r' => quoted.push_str("\\r"), + '\t' => quoted.push_str("\\t"), + ch if ch.is_control() => { + write!(&mut quoted, "\\u{:04X}", ch as u32).expect("should write to string"); + } + ch => quoted.push(ch), + } + } + quoted.push('"'); + quoted +} + +fn lowercase_hex(bytes: &[u8]) -> String { + const HEX: &[u8; 16] = b"0123456789abcdef"; + let mut encoded = String::with_capacity(bytes.len() * 2); + for byte in bytes { + encoded.push(HEX[(byte >> 4) as usize] as char); + encoded.push(HEX[(byte & 0x0f) as usize] as char); + } + encoded } fn replace_key_in_section( @@ -389,6 +700,7 @@ fn is_key_line(trimmed_line: &str, key: &str) -> bool { #[cfg(test)] mod tests { use std::cell::Cell; + use std::collections::VecDeque; use tempfile::TempDir; @@ -400,6 +712,26 @@ mod tests { calls: Cell, } + struct FixedPathGenerator { + paths: VecDeque, + } + + impl FixedPathGenerator { + fn new(paths: &[&str]) -> Self { + Self { + paths: paths.iter().map(|path| (*path).to_string()).collect(), + } + } + } + + impl OpaqueAssetPathGenerator for FixedPathGenerator { + fn next_path(&mut self) -> String { + self.paths + .pop_front() + .expect("should have a fixed generated asset path") + } + } + impl FakeCollector { fn new(collected: CollectedPage) -> Self { Self { @@ -451,6 +783,19 @@ mod tests { } } + fn audited_asset(url: &str, party: AssetParty, integration: Option<&str>) -> AuditedAsset { + AuditedAsset { + kind: "script".to_string(), + url: url.to_string(), + host: Url::parse(url) + .ok() + .and_then(|parsed| parsed.host_str().map(str::to_string)) + .unwrap_or_default(), + party, + integration: integration.map(str::to_string), + } + } + #[test] fn parse_audit_url_accepts_http_and_https() { assert!(parse_audit_url("http://publisher.example").is_ok()); @@ -627,6 +972,206 @@ mod tests { ); } + #[test] + fn build_draft_config_writes_disabled_js_asset_proxy_candidates() { + let url = Url::parse("https://publisher.example/page").expect("should parse URL"); + let artifact = AuditArtifact { + audited_url: url.to_string(), + page_title: Some("Example".to_string()), + js_asset_count: 2, + third_party_asset_count: 2, + detected_integrations: vec![DetectedIntegration { + id: "gpt".to_string(), + evidence: "https://securepubads.g.doubleclick.net/tag/js/gpt.js".to_string(), + }], + assets: vec![ + audited_asset( + "https://cdn.vendor.example/sdk.js", + AssetParty::ThirdParty, + None, + ), + audited_asset( + "https://securepubads.g.doubleclick.net/tag/js/gpt.js", + AssetParty::ThirdParty, + Some("gpt"), + ), + ], + warnings: Vec::new(), + }; + let mut generator = FixedPathGenerator::new(&[ + "/assets/aaaaaaaaaaaaaaaaaaaaaaaa.js", + "/assets/bbbbbbbbbbbbbbbbbbbbbbbb.js", + ]); + + let draft = build_draft_config_with_generator(&url, &artifact, &mut generator) + .expect("should build draft config"); + + assert_eq!( + draft.js_asset_proxy_candidate_count, 2, + "should report generated disabled entries" + ); + assert!(draft + .toml + .contains("[integrations.js_asset_proxy]\nenabled = false")); + assert!(draft.toml.contains("/assets/aaaaaaaaaaaaaaaaaaaaaaaa.js")); + assert!(draft.toml.contains("/assets/bbbbbbbbbbbbbbbbbbbbbbbb.js")); + assert!(draft + .toml + .contains("origin_url = \"https://cdn.vendor.example/sdk.js\"")); + assert!(draft.toml.contains("proxy = \"disabled\"")); + assert!(draft.toml.contains("Detected integration: gpt")); + assert!(draft + .toml + .contains("Native integration may be preferable: [integrations.gpt]")); + assert!( + !draft.toml.contains("example-vendor-loader"), + "should remove starter-template placeholder asset" + ); + toml::from_str::(&draft.toml).expect("draft should parse as TOML"); + } + + #[test] + fn generated_asset_proxy_paths_are_opaque() { + let url = Url::parse("https://publisher.example/page").expect("should parse URL"); + let artifact = AuditArtifact { + audited_url: url.to_string(), + page_title: None, + js_asset_count: 1, + third_party_asset_count: 1, + detected_integrations: Vec::new(), + assets: vec![audited_asset( + "https://cdn.vendor.example/vendor-loader.js", + AssetParty::ThirdParty, + None, + )], + warnings: Vec::new(), + }; + let mut generator = FixedPathGenerator::new(&["/assets/0123456789abcdef01234567.js"]); + + let draft = build_draft_config_with_generator(&url, &artifact, &mut generator) + .expect("should build draft config"); + let path_line = draft + .toml + .lines() + .find(|line| line.starts_with("path = ") && line.contains("0123456789abcdef")) + .expect("should include generated path"); + + assert!(path_line.contains("/assets/0123456789abcdef01234567.js")); + assert!( + !path_line.contains("vendor") + && !path_line.contains("cdn") + && !path_line.contains("loader"), + "generated path should not include vendor, domain, or filename semantics" + ); + } + + #[test] + fn asset_proxy_generation_deduplicates_and_summarizes_skips() { + let url = Url::parse("https://publisher.example/page").expect("should parse URL"); + let artifact = AuditArtifact { + audited_url: url.to_string(), + page_title: None, + js_asset_count: 4, + third_party_asset_count: 3, + detected_integrations: Vec::new(), + assets: vec![ + audited_asset( + "https://cdn.vendor.example/sdk.js", + AssetParty::ThirdParty, + None, + ), + audited_asset( + "https://cdn.vendor.example/sdk.js", + AssetParty::ThirdParty, + None, + ), + audited_asset( + "https://publisher.example/app.js", + AssetParty::FirstParty, + None, + ), + audited_asset( + "http://cdn.vendor.example/insecure.js", + AssetParty::ThirdParty, + None, + ), + ], + warnings: Vec::new(), + }; + let mut generator = FixedPathGenerator::new(&["/assets/111111111111111111111111.js"]); + + let draft = build_draft_config_with_generator(&url, &artifact, &mut generator) + .expect("should build draft config"); + + assert_eq!(draft.js_asset_proxy_candidate_count, 1); + assert_eq!( + draft + .toml + .matches("[[integrations.js_asset_proxy.assets]]") + .count(), + 1, + "should only emit one candidate entry" + ); + assert!(draft.toml.contains("# - 1 first-party script")); + assert!(draft.toml.contains("# - 1 non-HTTPS third-party script")); + assert!(draft.toml.contains("# - 1 duplicate script URL")); + } + + #[test] + fn asset_proxy_generation_with_no_candidates_removes_placeholder_asset() { + let url = Url::parse("https://publisher.example/page").expect("should parse URL"); + let artifact = AuditArtifact { + audited_url: url.to_string(), + page_title: None, + js_asset_count: 1, + third_party_asset_count: 0, + detected_integrations: Vec::new(), + assets: vec![audited_asset( + "https://publisher.example/app.js", + AssetParty::FirstParty, + None, + )], + warnings: Vec::new(), + }; + let mut generator = FixedPathGenerator::new(&[]); + + let draft = build_draft_config_with_generator(&url, &artifact, &mut generator) + .expect("should build draft config"); + + assert_eq!(draft.js_asset_proxy_candidate_count, 0); + assert!(draft + .toml + .contains("No eligible third-party HTTPS script assets")); + assert!( + !draft + .toml + .contains("[[integrations.js_asset_proxy.assets]]"), + "should not emit asset array entries without candidates" + ); + assert!( + !draft.toml.contains("example-vendor-loader"), + "should remove starter-template placeholder asset" + ); + toml::from_str::(&draft.toml).expect("draft should parse as TOML"); + } + + #[test] + fn run_audit_summary_reports_written_asset_proxy_candidates() { + let temp = TempDir::new().expect("should create temp dir"); + let config = temp.path().join("trusted-server.toml"); + let mut args = audit_args("https://publisher.example/page"); + args.config = Some(config); + args.no_js_assets = true; + let collector = FakeCollector::new(collected_page()); + let mut out = Vec::new(); + + run_audit(&args, &collector, &mut out).expect("should run audit"); + + let summary = String::from_utf8(out).expect("summary should be UTF-8"); + assert!(summary.contains("JS asset proxy candidates:")); + assert!(summary.contains("disabled entries written to draft config")); + } + #[test] fn build_draft_config_uses_final_url_and_detected_integrations() { let url = Url::parse("https://www.publisher.example:8443/path").expect("should parse URL"); diff --git a/crates/trusted-server-core/src/config.rs b/crates/trusted-server-core/src/config.rs index 4499db7e3..4ef2824bf 100644 --- a/crates/trusted-server-core/src/config.rs +++ b/crates/trusted-server-core/src/config.rs @@ -18,8 +18,8 @@ use crate::error::TrustedServerError; use crate::integrations::{ adserver_mock::AdServerMockConfig, aps::ApsConfig, datadome::DataDomeConfig, didomi::DidomiIntegrationConfig, google_tag_manager::GoogleTagManagerConfig, gpt::GptConfig, - lockr::LockrConfig, nextjs::NextJsIntegrationConfig, permutive::PermutiveConfig, prebid, - sourcepoint::SourcepointConfig, testlight::TestlightConfig, + js_asset_proxy::JsAssetProxyConfig, lockr::LockrConfig, nextjs::NextJsIntegrationConfig, + permutive::PermutiveConfig, prebid, sourcepoint::SourcepointConfig, testlight::TestlightConfig, }; use crate::settings::{IntegrationConfig, Settings}; @@ -109,12 +109,32 @@ impl edgezero_core::app_config::AppConfigMeta for TrustedServerAppConfig { /// Returns [`TrustedServerError`] when the config should not be deployed. pub fn validate_settings_for_deploy(settings: &Settings) -> Result<(), Report> { settings.reject_placeholder_secrets()?; + validate_js_asset_proxy_config(settings)?; let enabled_auction_providers = validate_enabled_integrations(settings)?; validate_auction_provider_names(settings, &enabled_auction_providers)?; PartnerRegistry::from_config(&settings.ec.partners).map(|_| ())?; Ok(()) } +fn validate_js_asset_proxy_config(settings: &Settings) -> Result<(), Report> { + let Some(raw_config) = settings.integrations.get("js_asset_proxy") else { + return Ok(()); + }; + + let config: JsAssetProxyConfig = serde_json::from_value(raw_config.clone()).map_err(|error| { + Report::new(TrustedServerError::Configuration { + message: format!( + "integration startup failed for `js_asset_proxy`: configuration could not be parsed: {error}" + ), + }) + })?; + config.validate().map_err(|error| { + Report::new(TrustedServerError::Configuration { + message: format!("integration startup failed for `js_asset_proxy`: {error}"), + }) + }) +} + fn validate_enabled_integrations( settings: &Settings, ) -> Result, Report> { @@ -261,6 +281,55 @@ password = "production-admin-password-32-bytes" ); } + #[test] + fn validate_rejects_invalid_enabled_js_asset_proxy_config() { + let mut settings = valid_settings(); + settings.integrations.insert( + "js_asset_proxy".to_string(), + serde_json::json!({ "enabled": true }), + ); + + let err = validate_settings_for_deploy(&settings) + .expect_err("should reject invalid JS asset proxy config"); + let message = err.to_string(); + assert!( + message.contains("js_asset_proxy"), + "error should mention JS asset proxy validation" + ); + assert!( + message.contains("empty_assets") || message.contains("assets"), + "error should mention the missing assets" + ); + } + + #[test] + fn validate_rejects_invalid_disabled_js_asset_proxy_assets() { + let mut settings = valid_settings(); + settings.integrations.insert( + "js_asset_proxy".to_string(), + serde_json::json!({ + "enabled": false, + "assets": [{ + "path": "bad path", + "origin_url": "not-a-url", + "proxy": "disabled" + }] + }), + ); + + let err = validate_settings_for_deploy(&settings) + .expect_err("should reject invalid disabled asset inventory"); + let message = err.to_string(); + assert!( + message.contains("js_asset_proxy"), + "error should mention JS asset proxy validation" + ); + assert!( + message.contains("path") || message.contains("origin_url"), + "error should mention the invalid asset fields" + ); + } + #[test] fn validate_trait_reports_deploy_errors() { let mut settings = valid_settings(); diff --git a/docs/guide/cli.md b/docs/guide/cli.md index d745c89d0..654bf1055 100644 --- a/docs/guide/cli.md +++ b/docs/guide/cli.md @@ -96,6 +96,14 @@ publisher-specific settings, then run: ts config validate ``` +The draft also fills `[integrations.js_asset_proxy]` with disabled third-party +script candidates from the audit. These entries are inventory only: they do not +register routes or rewrite HTML until you set +`integrations.js_asset_proxy.enabled = true` and change individual +`assets[].proxy` values to `"enabled"` or `"blocked"`. Some candidates may be +runtime-injected scripts; JS Asset Proxy only rewrites exact script `src` values +present in HTML processed by Trusted Server. + If a config already exists, avoid overwriting it: ```bash diff --git a/docs/guide/getting-started.md b/docs/guide/getting-started.md index 893c5b18f..c11858baa 100644 --- a/docs/guide/getting-started.md +++ b/docs/guide/getting-started.md @@ -117,7 +117,9 @@ ts audit https://publisher.example ``` The audit command writes `js-assets.toml` plus a draft `trusted-server.toml`. -Review the draft, replace placeholders/secrets, then validate it. +The draft includes disabled JS Asset Proxy candidates for detected third-party +scripts. Review the draft, replace placeholders/secrets, and enable only the asset +proxy entries you want to serve or block. Edit `trusted-server.toml` to configure: diff --git a/docs/superpowers/specs/2026-04-01-js-asset-proxy-design.md b/docs/superpowers/specs/2026-04-01-js-asset-proxy-design.md index 1315a1b62..06e115317 100644 --- a/docs/superpowers/specs/2026-04-01-js-asset-proxy-design.md +++ b/docs/superpowers/specs/2026-04-01-js-asset-proxy-design.md @@ -60,15 +60,15 @@ proxy = "disabled" ### Fields -| Field | Required | Description | -| ---------------------------- | -------: | ---------------------------------------------------------------- | -| `enabled` | Yes | Enables or disables the integration. | -| `cache_ttl_seconds` | No | Optional downstream cache TTL override for all assets. When unset, preserve the upstream cache policy. | -| `assets` | Yes | List of JavaScript assets the proxy may serve. | +| Field | Required | Description | +| ---------------------------- | -------: | ---------------------------------------------------------------------------------------------------------------------- | +| `enabled` | Yes | Enables or disables the integration. | +| `cache_ttl_seconds` | No | Optional downstream cache TTL override for all assets. When unset, preserve the upstream cache policy. | +| `assets` | Yes | List of JavaScript assets the proxy may serve. | | `assets[].path` | Yes | Stable identifier for logs, tests, and response diagnostics; exact first-party request path handled by Trusted Server. | -| `assets[].origin_url` | Yes | Exact upstream JavaScript URL to fetch or match for page rewriting. | -| `assets[].proxy` | No | Per-asset proxy behavior: `enabled`, `disabled`, or `blocked`. Defaults to `enabled`. | -| `assets[].cache_ttl_seconds` | No | Per-asset downstream cache TTL override. Takes precedence over the integration-level value. | +| `assets[].origin_url` | Yes | Exact upstream JavaScript URL to fetch or match for page rewriting. | +| `assets[].proxy` | No | Per-asset proxy behavior: `enabled`, `disabled`, or `blocked`. Defaults to `enabled`. | +| `assets[].cache_ttl_seconds` | No | Per-asset downstream cache TTL override. Takes precedence over the integration-level value. | ### Validation @@ -91,11 +91,11 @@ The implementation may use stricter validation if it keeps the configuration con Each asset has a `proxy` setting that controls both page rewriting and route registration: -| Value | Behavior | -| ---------- | -------- | +| Value | Behavior | +| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `enabled` | Rewrite exact matching `` matches. + +Therefore, some generated candidates may be runtime-only scripts injected by tag +managers or other JavaScript. Those entries are still useful as inventory, but +enabling them may not cause rewriting unless the exact `origin_url` appears in +origin HTML processed by Trusted Server. + +The generated config should include a warning comment near the asset proxy block: + +```toml +# Audit note: some discovered scripts may be runtime-injected and may not appear +# in origin HTML. JS Asset Proxy rewrites only exact script src values present in +# HTML processed by Trusted Server. +``` + +--- + +## 9. Validation requirements + +Generated draft config should pass the same structural validation as any manually +written config, except for the existing starter-template placeholder secret caveat +that already applies to audit-generated configs. + +Implementation must also update CLI runtime-startup validation so +`ts config validate` checks `JsAssetProxyConfig` alongside the other integrations. + +Specifically, `crates/trusted-server-cli/src/config_command.rs` should validate: + +```rust +validate_integration::(settings, "js_asset_proxy")?; +``` + +This catches invalid operator edits before `ts config push`. + +--- + +## 10. Implementation notes + +Suggested code changes: + +- Import `trusted_server_core::integrations::js_asset_proxy::JsAssetProxyConfig` + in `crates/trusted-server-cli/src/config_command.rs` and include it in enabled + integration validation. +- Add an asset-proxy draft builder in `crates/trusted-server-cli/src/audit.rs`, + near `build_draft_config`. +- Reuse `AuditArtifact.assets` for candidate selection. +- Add helper functions for: + - filtering eligible candidates; + - generating opaque paths; + - replacing the JS asset proxy block in the draft TOML; + - formatting TOML comments and asset entries. +- Keep the implementation browser-independent and unit-testable. + +Possible helper shape: + +```rust +fn build_js_asset_proxy_section( + artifact: &AuditArtifact, + id_generator: &mut dyn OpaqueAssetIdGenerator, +) -> CliResult; +``` + +The production generator can use randomness; tests can use fixed IDs. + +--- + +## 11. Tests + +Add focused unit tests for: + +1. `build_draft_config` replaces the sample JS asset proxy block with audited + disabled entries. +2. Generated entries use `proxy = "disabled"`. +3. Generated integration-level `enabled` remains `false`. +4. Generated paths are opaque `/assets/*.js` paths and do not include vendor + names, hosts, or source filenames. +5. Duplicate discovered script URLs produce one config entry. +6. First-party scripts are skipped. +7. Non-HTTPS third-party scripts are skipped with a warning/comment. +8. Known integrations are included but commented as candidates for native + integration review. +9. No eligible candidates removes the example placeholder asset and emits no + invalid asset entries. +10. `ts config validate` invokes JS Asset Proxy startup validation. + +Run at minimum: + +```bash +cargo test --workspace +cargo fmt --all -- --check +cargo clippy --workspace --all-targets --all-features -- -D warnings +``` + +--- + +## 12. Documentation updates + +Update `docs/guide/cli.md` and `docs/guide/getting-started.md` to say: + +- `ts audit` now fills `[integrations.js_asset_proxy]` with disabled candidates; +- candidates are review inventory, not active proxy routes; +- enabling requires setting both `integrations.js_asset_proxy.enabled = true` and + individual `assets[].proxy = "enabled"` or `"blocked"`; +- runtime-injected scripts may appear in audit output but may not be rewritten + unless they appear as exact script URLs in origin HTML. From 7730c4fba0f5c4000ee65e434ad2c22d900f5107 Mon Sep 17 00:00:00 2001 From: Christian Date: Wed, 1 Jul 2026 16:39:59 -0500 Subject: [PATCH 5/5] Fix JS asset proxy review findings --- .../trusted-server-adapter-fastly/src/main.rs | 60 +++++- .../src/route_tests.rs | 78 ++++++++ crates/trusted-server-cli/src/audit.rs | 2 +- .../src/integrations/js_asset_proxy.rs | 176 +++++++++++++++++- .../src/integrations/mod.rs | 2 +- crates/trusted-server-core/src/proxy.rs | 65 ++++++- docs/guide/cli.md | 2 +- .../specs/2026-04-01-js-asset-proxy-design.md | 21 ++- ...2-ts-audit-js-asset-proxy-config-design.md | 12 +- 9 files changed, 393 insertions(+), 25 deletions(-) diff --git a/crates/trusted-server-adapter-fastly/src/main.rs b/crates/trusted-server-adapter-fastly/src/main.rs index 489c4982d..50ab39d2e 100644 --- a/crates/trusted-server-adapter-fastly/src/main.rs +++ b/crates/trusted-server-adapter-fastly/src/main.rs @@ -100,6 +100,10 @@ enum HandlerOutcome { response: HttpResponse, body: EdgeBody, }, + RawStreaming { + response: HttpResponse, + body: EdgeBody, + }, } impl HandlerOutcome { @@ -108,11 +112,23 @@ impl HandlerOutcome { match self { HandlerOutcome::Buffered(resp) | HandlerOutcome::AuthChallenge(resp) => resp.status(), HandlerOutcome::Streaming { response, .. } - | HandlerOutcome::AssetStreaming { response, .. } => response.status(), + | HandlerOutcome::AssetStreaming { response, .. } + | HandlerOutcome::RawStreaming { response, .. } => response.status(), } } } +fn response_to_legacy_outcome(response: HttpResponse) -> HandlerOutcome { + let (parts, body) = response.into_parts(); + match body { + EdgeBody::Stream(_) => HandlerOutcome::RawStreaming { + response: HttpResponse::from_parts(parts, EdgeBody::empty()), + body, + }, + body => HandlerOutcome::Buffered(HttpResponse::from_parts(parts, body)), + } +} + /// Returns `true` if the raw config-store value represents an enabled flag. /// /// Accepted values (after whitespace trimming): `"1"` or `"true"` in any ASCII case. @@ -912,6 +928,46 @@ fn legacy_main(mut req: FastlyRequest) { log::error!("failed to finish asset streaming body: {e}"); } } + HandlerOutcome::RawStreaming { mut response, body } => { + finalize_response(&state.settings, geo_info.as_ref(), &mut response); + asset_cache_policy.apply_after_route_finalization(&mut response); + if should_finalize_ec { + ec_finalize_response( + &state.settings, + &ec_context, + finalize_kv_graph.as_ref(), + &partner_registry, + eids_cookie.as_deref(), + sharedid_cookie.as_deref(), + &mut response, + ); + } + request_filter_effects.apply_to_response(&mut response); + let fastly_resp = compat::to_fastly_response_skeleton(response); + let mut streaming_body = fastly_resp.stream_to_client(); + let mut stream_succeeded = false; + if let Err(e) = + futures::executor::block_on(stream_asset_body(body, &mut streaming_body)) + { + log::error!("raw response streaming failed: {e:?}"); + drop(streaming_body); + } else if let Err(e) = streaming_body.finish() { + log::error!("failed to finish raw streaming body: {e}"); + } else { + stream_succeeded = true; + } + + if is_real_browser && stream_succeeded { + if let Some(context) = build_pull_sync_context(&ec_context) { + run_pull_sync_after_send( + &state.settings, + &partner_registry, + &context, + &runtime_services, + ); + } + } + } } } @@ -1359,7 +1415,7 @@ async fn route_request( let _ = organic_route; let outcome = result - .map(HandlerOutcome::Buffered) + .map(response_to_legacy_outcome) .unwrap_or_else(|e| HandlerOutcome::Buffered(http_error_response(&e))); Ok(RouteResult { diff --git a/crates/trusted-server-adapter-fastly/src/route_tests.rs b/crates/trusted-server-adapter-fastly/src/route_tests.rs index ce542a7c0..9e6754cae 100644 --- a/crates/trusted-server-adapter-fastly/src/route_tests.rs +++ b/crates/trusted-server-adapter-fastly/src/route_tests.rs @@ -1551,6 +1551,84 @@ fn asset_routes_stream_asset_responses_directly() { // take the `response: None` direct-send path with EC finalization skipped. } +#[test] +fn integration_routes_preserve_streaming_proxy_responses() { + let mut settings = create_test_settings(); + settings + .integrations + .insert_config( + "js_asset_proxy", + &json!({ + "enabled": true, + "assets": [{ + "path": "/assets/vendor.js", + "origin_url": "https://cdn.example.com/vendor.js", + "proxy": "enabled" + }] + }), + ) + .expect("should insert JS asset proxy config"); + let orchestrator = build_orchestrator(&settings).expect("should build auction orchestrator"); + let integration_registry = + IntegrationRegistry::new(&settings).expect("should create integration registry"); + let partner_registry = test_partner_registry(&settings); + + let fastly_req = Request::get("https://test.com/assets/vendor.js"); + let http_client = Arc::new(StreamingRecordingHttpClient::new()); + let services = test_runtime_services_with_secret_http_client_and_geo( + &fastly_req, + Arc::new(FixedBackend), + Arc::new(NoopSecretStore), + Arc::clone(&http_client) as Arc, + Arc::new(FixedGeo(us_california_geo())), + ); + let req = compat::from_fastly_request(fastly_req); + + let outcome = futures::executor::block_on(route_request( + &settings, + &orchestrator, + &integration_registry, + &partner_registry, + &services, + req, + browser_device_signals(), + )) + .expect("should route streaming integration request"); + + let (response, body) = match outcome.outcome { + HandlerOutcome::RawStreaming { response, body } => Some((response, body)), + _ => None, + } + .expect("should preserve streaming integration response"); + assert_eq!(response.status(), StatusCode::OK); + assert!( + matches!(body, EdgeBody::Stream(_)), + "should preserve the integration response stream" + ); + + let calls = http_client + .calls + .lock() + .expect("should lock recorded calls"); + assert_eq!( + calls.len(), + 1, + "should send exactly one integration request" + ); + assert!( + calls[0].stream_response, + "JS asset proxy should request a streaming origin response from the platform" + ); + assert_eq!( + calls[0].backend_name, "https-cdn.example.com", + "should resolve the configured JS asset origin backend" + ); + assert_eq!( + calls[0].uri, "https://cdn.example.com/vendor.js", + "should send the request to the configured JS asset origin URL" + ); +} + #[test] fn asset_origin_failure_does_not_fall_back_to_publisher_origin() { let mut settings = create_test_settings(); diff --git a/crates/trusted-server-cli/src/audit.rs b/crates/trusted-server-cli/src/audit.rs index 7f2684bbe..15b856b60 100644 --- a/crates/trusted-server-cli/src/audit.rs +++ b/crates/trusted-server-cli/src/audit.rs @@ -426,7 +426,7 @@ fn build_js_asset_proxy_section( "# Audit note: some discovered scripts may be runtime-injected and may not appear\n", ); toml.push_str( - "# in origin HTML. JS Asset Proxy rewrites only exact script src values present in\n", + "# in origin HTML. JS Asset Proxy rewrites only matching script src URLs present in\n", ); toml.push_str("# HTML processed by Trusted Server.\n"); diff --git a/crates/trusted-server-core/src/integrations/js_asset_proxy.rs b/crates/trusted-server-core/src/integrations/js_asset_proxy.rs index f2359c16e..78b4bda25 100644 --- a/crates/trusted-server-core/src/integrations/js_asset_proxy.rs +++ b/crates/trusted-server-core/src/integrations/js_asset_proxy.rs @@ -190,6 +190,29 @@ fn path_contains_parent_segment(path: &str) -> bool { path.split('/').any(|segment| segment == "..") } +fn normalize_script_src(script_src: &str, request_scheme: &str) -> Option { + let candidate = if script_src.starts_with("//") { + let request_scheme = request_scheme.to_ascii_lowercase(); + if !matches!(request_scheme.as_str(), "http" | "https") { + return None; + } + format!("{request_scheme}:{script_src}") + } else { + script_src.to_string() + }; + + let mut url = Url::parse(&candidate).ok()?; + let has_default_port = matches!( + (url.scheme(), url.port()), + ("http", Some(80)) | ("https", Some(443)) + ); + if has_default_port { + url.set_port(None).ok()?; + } + + Some(url.to_string()) +} + /// JavaScript asset proxy integration implementation. pub struct JsAssetProxyIntegration { config: JsAssetProxyConfig, @@ -221,12 +244,24 @@ impl JsAssetProxyIntegration { .find(|asset| asset.origin_url == origin_url) } + fn asset_for_script_src( + &self, + script_src: &str, + ctx: &IntegrationAttributeContext<'_>, + ) -> Option<&JsAssetProxyAsset> { + self.asset_for_origin_url(script_src).or_else(|| { + let normalized_src = normalize_script_src(script_src, ctx.request_scheme)?; + self.asset_for_origin_url(&normalized_src) + }) + } + fn build_proxy_config<'a>( origin_url: &'a str, req: &Request, ) -> ProxyRequestConfig<'a> { let mut config = ProxyRequestConfig::new(origin_url) .with_streaming() + .with_stream_response() .without_forward_headers(); config.follow_redirects = false; config.forward_ec_id = false; @@ -472,7 +507,7 @@ impl IntegrationAttributeRewriter for JsAssetProxyIntegration { return AttributeRewriteAction::keep(); } - let Some(asset) = self.asset_for_origin_url(attr_value) else { + let Some(asset) = self.asset_for_script_src(attr_value, ctx) else { return AttributeRewriteAction::keep(); }; @@ -540,11 +575,18 @@ mod tests { integration: Arc, ) -> String { let rewriter: Arc = integration; + process_html_with_registry( + html, + IntegrationRegistry::from_rewriters(vec![rewriter], Vec::new()), + ) + } + + fn process_html_with_registry(html: &str, integrations: IntegrationRegistry) -> String { let processor = create_html_processor(HtmlProcessorConfig { origin_host: "origin.example.com".to_string(), request_host: "publisher.example.com".to_string(), request_scheme: "https".to_string(), - integrations: IntegrationRegistry::from_rewriters(vec![rewriter], Vec::new()), + integrations, max_buffered_body_bytes: 16 * 1024 * 1024, }); let pipeline_config = PipelineConfig { @@ -724,6 +766,134 @@ mod tests { assert!(processed.contains(r#""#)); } + #[test] + fn script_src_matching_normalizes_common_browser_url_forms() { + let integration = JsAssetProxyIntegration::new(config_with_assets(vec![asset( + "/assets/vendor.js", + "https://cdn.example.com/vendor.js", + JsAssetProxyMode::Enabled, + )])); + let ctx = rewrite_context(); + + for script_src in [ + "//cdn.example.com/vendor.js", + "HTTPS://CDN.EXAMPLE.COM/vendor.js", + "https://cdn.example.com:443/vendor.js", + ] { + assert!( + matches!( + integration.rewrite("src", script_src, &ctx), + AttributeRewriteAction::Replace(ref value) if value == "/assets/vendor.js" + ), + "script src {script_src} should normalize to the configured origin URL" + ); + } + } + + #[test] + fn js_asset_proxy_rewriter_takes_precedence_over_native_rewriters() { + let mut settings = create_test_settings(); + settings + .integrations + .insert_config("gpt", &json!({ "enabled": true })) + .expect("should insert GPT config"); + settings + .integrations + .insert_config( + JS_ASSET_PROXY_INTEGRATION_ID, + &json!({ + "enabled": true, + "assets": [{ + "path": "/assets/gpt.js", + "origin_url": "https://securepubads.g.doubleclick.net/tag/js/gpt.js", + "proxy": "enabled" + }] + }), + ) + .expect("should insert JS asset proxy config"); + let registry = IntegrationRegistry::new(&settings).expect("should build registry"); + let html = r#""#; + + let processed = process_html_with_registry(html, registry); + + assert!( + processed.contains(r#""#), + "JS asset proxy should rewrite before GPT native rewriter: {processed}" + ); + assert!( + !processed.contains("/integrations/gpt/script"), + "GPT native rewrite should not override JS asset proxy" + ); + } + + #[test] + fn js_asset_proxy_blocking_takes_precedence_over_native_rewriters() { + let mut settings = create_test_settings(); + settings + .integrations + .insert_config("gpt", &json!({ "enabled": true })) + .expect("should insert GPT config"); + settings + .integrations + .insert_config( + JS_ASSET_PROXY_INTEGRATION_ID, + &json!({ + "enabled": true, + "assets": [{ + "path": "/assets/gpt.js", + "origin_url": "https://securepubads.g.doubleclick.net/tag/js/gpt.js", + "proxy": "blocked" + }] + }), + ) + .expect("should insert JS asset proxy config"); + let registry = IntegrationRegistry::new(&settings).expect("should build registry"); + let html = r#""#; + + let processed = process_html_with_registry(html, registry); + + assert!( + !processed.contains("googletag.cmd"), + "blocked JS asset should remove the script element before GPT can rewrite it" + ); + assert!( + !processed.contains("/integrations/gpt/script"), + "GPT native rewrite should not keep a blocked script" + ); + } + + #[test] + fn disabled_js_asset_proxy_candidate_allows_native_rewriters() { + let mut settings = create_test_settings(); + settings + .integrations + .insert_config("gpt", &json!({ "enabled": true })) + .expect("should insert GPT config"); + settings + .integrations + .insert_config( + JS_ASSET_PROXY_INTEGRATION_ID, + &json!({ + "enabled": true, + "assets": [{ + "path": "/assets/gpt.js", + "origin_url": "https://securepubads.g.doubleclick.net/tag/js/gpt.js", + "proxy": "disabled" + }] + }), + ) + .expect("should insert JS asset proxy config"); + let registry = IntegrationRegistry::new(&settings).expect("should build registry"); + let html = r#""#; + + let processed = process_html_with_registry(html, registry); + + assert!( + processed.contains(r#""#), + "disabled JS asset proxy entries should not suppress native integration rewrites" + ); + } + #[test] fn rejects_duplicate_asset_paths() { let config = config_with_assets(vec![ @@ -1099,6 +1269,8 @@ mod tests { assert!(!config.copy_request_headers); assert!(!config.follow_redirects); assert!(!config.forward_ec_id); + assert!(config.stream_passthrough); + assert!(config.stream_response); let forwarded: Vec<(String, String)> = config .headers diff --git a/crates/trusted-server-core/src/integrations/mod.rs b/crates/trusted-server-core/src/integrations/mod.rs index 84b3315b7..cf39a9fec 100644 --- a/crates/trusted-server-core/src/integrations/mod.rs +++ b/crates/trusted-server-core/src/integrations/mod.rs @@ -279,6 +279,7 @@ type IntegrationBuilder = pub(crate) fn builders() -> &'static [IntegrationBuilder] { &[ + js_asset_proxy::register, prebid::register, testlight::register, nextjs::register, @@ -290,6 +291,5 @@ pub(crate) fn builders() -> &'static [IntegrationBuilder] { google_tag_manager::register, datadome::register, gpt::register, - js_asset_proxy::register, ] } diff --git a/crates/trusted-server-core/src/proxy.rs b/crates/trusted-server-core/src/proxy.rs index 9bf58e630..703f5e79d 100644 --- a/crates/trusted-server-core/src/proxy.rs +++ b/crates/trusted-server-core/src/proxy.rs @@ -311,6 +311,8 @@ pub struct ProxyRequestConfig<'a> { pub copy_request_headers: bool, /// When true, stream the origin response without HTML/CSS rewrites. pub stream_passthrough: bool, + /// When true, ask the platform adapter to preserve the upstream response body as a stream. + pub stream_response: bool, /// Domains allowed for the initial request and any redirects. /// /// **Open mode** (`&[]`): every host is permitted. Integration proxies pass `&[]` @@ -337,6 +339,7 @@ impl<'a> ProxyRequestConfig<'a> { headers: Vec::new(), copy_request_headers: true, stream_passthrough: false, + stream_response: false, allowed_domains: &[], } } @@ -368,6 +371,13 @@ impl<'a> ProxyRequestConfig<'a> { self.stream_passthrough = true; self } + + /// Ask the platform adapter to preserve the upstream response body as a stream. + #[must_use] + pub fn with_stream_response(mut self) -> Self { + self.stream_response = true; + self + } } /// Encodings we support decompressing in `finalize_proxied_response`. @@ -645,6 +655,7 @@ struct ProxyRequestHeaders<'a> { struct ProxyRedirectPolicy<'a> { follow_redirects: bool, stream_passthrough: bool, + stream_response: bool, allowed_domains: &'a [String], } @@ -672,6 +683,7 @@ pub async fn proxy_request( headers, copy_request_headers, stream_passthrough, + stream_response, allowed_domains, } = config; @@ -698,6 +710,7 @@ pub async fn proxy_request( ProxyRedirectPolicy { follow_redirects, stream_passthrough, + stream_response, allowed_domains, }, ) @@ -1296,10 +1309,15 @@ async fn proxy_with_redirects( message: "failed to build proxy request".to_string(), })?; + let mut platform_request = PlatformHttpRequest::new(edge_req, backend_name); + if redirect_policy.stream_response { + platform_request = platform_request.with_stream_response(); + } + let platform_resp = request_headers .services .http_client() - .send(PlatformHttpRequest::new(edge_req, backend_name)) + .send(platform_request) .await .change_context(TrustedServerError::Proxy { message: "Failed to proxy".to_string(), @@ -1451,6 +1469,7 @@ pub async fn handle_first_party_proxy( headers: Vec::new(), copy_request_headers: true, stream_passthrough: false, + stream_response: false, allowed_domains: &settings.proxy.allowed_domains, }, services, @@ -2380,7 +2399,8 @@ mod tests { HeaderValue::from_static("application/octet-stream"), ) .without_forward_headers() - .with_streaming(); + .with_streaming() + .with_stream_response(); assert_eq!(cfg.target_url, "https://example.com/asset"); assert!(cfg.follow_redirects, "should follow redirects by default"); @@ -2395,6 +2415,10 @@ mod tests { cfg.stream_passthrough, "should enable streaming passthrough" ); + assert!( + cfg.stream_response, + "should request streaming platform responses" + ); } #[test] @@ -3105,6 +3129,7 @@ mod tests { headers: Vec::new(), copy_request_headers: false, stream_passthrough: false, + stream_response: false, allowed_domains: &[], }, &services, @@ -3145,6 +3170,7 @@ mod tests { headers: Vec::new(), copy_request_headers: false, stream_passthrough: false, + stream_response: false, allowed_domains: &[], }, &services, @@ -3190,6 +3216,7 @@ mod tests { headers: Vec::new(), copy_request_headers: false, stream_passthrough: false, + stream_response: false, allowed_domains: &[], }, &services, @@ -3202,6 +3229,38 @@ mod tests { }); } + #[test] + fn proxy_request_forwards_stream_response_flag_to_platform_request() { + futures::executor::block_on(async { + use crate::platform::test_support::StubHttpClient; + + let stub = Arc::new(StubHttpClient::new()); + stub.push_response(200, b"ok".to_vec()); + let services = build_services_with_http_client( + Arc::clone(&stub) as Arc + ); + let settings = create_test_settings(); + let req = build_http_request(Method::GET, "https://example.com/"); + + proxy_request( + &settings, + req, + ProxyRequestConfig::new("https://example.com/resource") + .without_forward_headers() + .with_stream_response(), + &services, + ) + .await + .expect("should proxy successfully"); + + assert_eq!( + stub.recorded_stream_response_flags(), + vec![true], + "should request a streaming platform response" + ); + }); + } + #[test] fn proxy_request_forwards_curated_headers_when_copy_request_headers_is_true() { futures::executor::block_on(async { @@ -3238,6 +3297,7 @@ mod tests { headers: Vec::new(), copy_request_headers: true, stream_passthrough: false, + stream_response: false, allowed_domains: &[], }, &services, @@ -3302,6 +3362,7 @@ mod tests { headers: Vec::new(), copy_request_headers: false, stream_passthrough: false, + stream_response: false, allowed_domains: &[], }, &services, diff --git a/docs/guide/cli.md b/docs/guide/cli.md index 654bf1055..866fc278c 100644 --- a/docs/guide/cli.md +++ b/docs/guide/cli.md @@ -101,7 +101,7 @@ script candidates from the audit. These entries are inventory only: they do not register routes or rewrite HTML until you set `integrations.js_asset_proxy.enabled = true` and change individual `assets[].proxy` values to `"enabled"` or `"blocked"`. Some candidates may be -runtime-injected scripts; JS Asset Proxy only rewrites exact script `src` values +runtime-injected scripts; JS Asset Proxy only rewrites matching script `src` URLs present in HTML processed by Trusted Server. If a config already exists, avoid overwriting it: diff --git a/docs/superpowers/specs/2026-04-01-js-asset-proxy-design.md b/docs/superpowers/specs/2026-04-01-js-asset-proxy-design.md index 06e115317..1f48e0e4c 100644 --- a/docs/superpowers/specs/2026-04-01-js-asset-proxy-design.md +++ b/docs/superpowers/specs/2026-04-01-js-asset-proxy-design.md @@ -91,13 +91,13 @@ The implementation may use stricter validation if it keeps the configuration con Each asset has a `proxy` setting that controls both page rewriting and route registration: -| Value | Behavior | -| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `enabled` | Rewrite exact matching `` matches. +rewrites server-returned HTML by matching `` URLs. Therefore, some generated candidates may be runtime-only scripts injected by tag managers or other JavaScript. Those entries are still useful as inventory, but -enabling them may not cause rewriting unless the exact `origin_url` appears in +enabling them may not cause rewriting unless a matching `src` URL appears in origin HTML processed by Trusted Server. The generated config should include a warning comment near the asset proxy block: ```toml # Audit note: some discovered scripts may be runtime-injected and may not appear -# in origin HTML. JS Asset Proxy rewrites only exact script src values present in +# in origin HTML. JS Asset Proxy rewrites only matching script src URLs present in # HTML processed by Trusted Server. ``` @@ -357,4 +357,4 @@ Update `docs/guide/cli.md` and `docs/guide/getting-started.md` to say: - enabling requires setting both `integrations.js_asset_proxy.enabled = true` and individual `assets[].proxy = "enabled"` or `"blocked"`; - runtime-injected scripts may appear in audit output but may not be rewritten - unless they appear as exact script URLs in origin HTML. + unless matching script URLs appear in origin HTML.