Skip to content

Commit 64fc6d5

Browse files
refactor(spec)!: retire connector.rateLimitConfig and the outbound rate-limit shape (#4911) (#4947)
ADR-0049 enforce-or-remove: `ConnectorSchema.rateLimitConfig` declared an outbound throttle that no engine ever applied. The platform's only token bucket (runtime security/rate-limit.ts) is INBOUND; no connector provider reads the key and no seam exists that could. Removed rather than kept — the vocabulary returns with an implementation (#4834 / PR #4878 ruling). - `retiredKey()` tombstone on `ConnectorSchema.rateLimitConfig` (non-strict schema — a plain delete would be an ADR-0104 silent strip) - `ConnectorRateLimitConfigSchema`/`ConnectorRateLimitConfig` and the orphaned `RateLimitStrategySchema`/`RateLimitStrategy` removed with it - D2 conversion `connector-rate-limit-config-removed` (retiredFromLoadPath) + D3 chain step at major 17 - #4684's RENAMED_DEFS entry absorbed: rename-then-delete in the same unreleased major is a delete - baselines updated deliberately (manifest -2 defs, authorable -6 lines +2 [RETIRED], api-surface -4 exports); docs/spec-changes/upgrade guide regenerated Claude-Session: https://claude.ai/code/session_01Ehu85kbvMcrNTUJjwxvLJ9 Co-authored-by: Claude <noreply@anthropic.com>
1 parent ff39e63 commit 64fc6d5

15 files changed

Lines changed: 409 additions & 227 deletions
Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
---
2+
"@objectstack/spec": major
3+
---
4+
5+
refactor(spec)!: remove `connector.rateLimitConfig` and the whole outbound rate-limit shape — the engine never existed (#4911, ADR-0049)
6+
7+
`ConnectorSchema.rateLimitConfig` let an author declare an outbound throttle for
8+
their connector — `strategy`, `maxRequests`, `windowSeconds`, `burstCapacity`,
9+
`respectUpstreamLimits`, `rateLimitHeaders` — and nothing anywhere applied it.
10+
This is not the ordinary declared-but-unread case; it is a step worse:
11+
**there is no outbound rate-limiting engine to wire it to.** The only token
12+
bucket the platform owns is `packages/runtime/src/security/rate-limit.ts`, and it
13+
is INBOUND — the dispatcher calls `consume(key)` on a request fingerprint and
14+
answers 429. No connector provider (`connector-rest`, `connector-openapi`,
15+
`connector-mcp`, `connector-slack`) reads the key, and no seam exists that could.
16+
17+
So a well-formed, schema-validated block told the author they had capped their
18+
call rate against a third party's quota, and capped nothing — the false-compliance
19+
class ADR-0049 exists for. With no implementation and no committed roadmap,
20+
`experimental` would be a promise nobody made; **absent** is the honest
21+
disposition. The vocabulary comes back *with* the engine, in one change
22+
(implementation-first — the #4834 / PR #4878 ruling for the plugin-runtime family).
23+
24+
FROM → TO:
25+
26+
| Removed | Replacement |
27+
| :--- | :--- |
28+
| `connector.rateLimitConfig` (key) | **none** — delete it; throttle at the connector provider or upstream gateway |
29+
| `ConnectorRateLimitConfigSchema` / `ConnectorRateLimitConfig` | **none** — importing either is TS2305 in v17 |
30+
| `RateLimitStrategySchema` / `RateLimitStrategy` | **none** — the enum had no other consumer |
31+
32+
**Do NOT substitute `shared`'s `RateLimitConfig`.** That is the INBOUND limiter
33+
(`enabled` / `windowMs` / `maxRequests`) and caps the calls others make to *us*
34+
the opposite direction. #4684 split the two names for exactly this confusion; the
35+
conversion deliberately does not rewrite one into the other, because that would
36+
silently change behaviour rather than losing a no-op.
37+
38+
The retirement kit:
39+
40+
- **Tombstone.** `ConnectorSchema` is not `.strict()`, so a plain delete would be
41+
a silent strip (ADR-0104). `retiredKey()` makes the removal audible in the two
42+
channels an upgrading author hits — `tsc` (the key types `never`) and the parse
43+
(the prescription itself). It reaches `stack.connectors[]` and
44+
`DeclarativeConnectorEntry`, which is `ConnectorSchema.superRefine(…)`.
45+
- **ADR-0087 D2 conversion + D3 chain step** (`connector-rate-limit-config-removed`,
46+
`retiredFromLoadPath`): `os migrate meta --from 16` deletes the key from author
47+
sources and stored rows replay clean. A lossless delete — the block never had an
48+
effect to lose.
49+
- **The shape goes with the key.** `ConnectorRateLimitConfigSchema` and the
50+
`RateLimitStrategySchema` enum it embedded had no other consumer, and an
51+
exported schema with no consumer reads as a capability to whoever finds it
52+
(#3950).
53+
- **#4684's rename is absorbed.** `integration/RateLimitConfig`
54+
`integration/ConnectorRateLimitConfig` and this retirement landed in the same
55+
unreleased major; composed they are a plain delete, so the `RENAMED_DEFS` entry
56+
is removed rather than pointing at a def this build no longer emits.
57+
- Baselines updated deliberately: `json-schema.manifest.json` (−2 defs),
58+
`authorable-surface.json` (−6 def lines; `Connector` /
59+
`DeclarativeConnectorEntry` gain `… [RETIRED]`), `api-surface.json` (−4
60+
exports). `api-surface-signatures.json` is unchanged by construction — it hashes
61+
each `defineX` parameter as TypeScript *prints* it, a reference
62+
(`z.input<typeof ConnectorSchema >`), so key-level narrowing never reaches it.
63+
64+
No runtime behaviour changes — that impossibility is the reason for the removal.

content/docs/references/integration/connector.mdx

Lines changed: 4 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -134,8 +134,8 @@ a dead end of the same class in #4738.)
134134
## TypeScript Usage
135135

136136
```typescript
137-
import { CircuitBreakerConfigSchema, ConnectorSchema, ConnectorActionSchema, ConnectorConflictResolutionSchema, ConnectorErrorCategorySchema, ConnectorFieldMappingSchema, ConnectorHealthSchema, ConnectorRateLimitConfigSchema, ConnectorRetryStrategySchema, ConnectorStatusSchema, ConnectorTriggerSchema, ConnectorTypeSchema, DataSyncConfigSchema, DeclarativeConnectorEntrySchema, ErrorMappingConfigSchema, ErrorMappingRuleSchema, HealthCheckConfigSchema, RateLimitStrategySchema, RetryConfigSchema, SyncStrategySchema, WebhookConfigSchema, WebhookEventSchema, WebhookSignatureAlgorithmSchema } from '@objectstack/spec/integration';
138-
import type { CircuitBreakerConfig, Connector, ConnectorConflictResolution, ConnectorErrorCategory, ConnectorFieldMapping, ConnectorHealth, ConnectorRateLimitConfig, ConnectorRetryStrategy, ConnectorStatus, ConnectorType, DataSyncConfig, DeclarativeConnectorEntry, ErrorMappingConfig, ErrorMappingRule, HealthCheckConfig, RateLimitStrategy, RetryConfig, SyncStrategy, WebhookConfig, WebhookEvent, WebhookSignatureAlgorithm } from '@objectstack/spec/integration';
137+
import { CircuitBreakerConfigSchema, ConnectorSchema, ConnectorActionSchema, ConnectorConflictResolutionSchema, ConnectorErrorCategorySchema, ConnectorFieldMappingSchema, ConnectorHealthSchema, ConnectorRetryStrategySchema, ConnectorStatusSchema, ConnectorTriggerSchema, ConnectorTypeSchema, DataSyncConfigSchema, DeclarativeConnectorEntrySchema, ErrorMappingConfigSchema, ErrorMappingRuleSchema, HealthCheckConfigSchema, RetryConfigSchema, SyncStrategySchema, WebhookConfigSchema, WebhookEventSchema, WebhookSignatureAlgorithmSchema } from '@objectstack/spec/integration';
138+
import type { CircuitBreakerConfig, Connector, ConnectorConflictResolution, ConnectorErrorCategory, ConnectorFieldMapping, ConnectorHealth, ConnectorRetryStrategy, ConnectorStatus, ConnectorType, DataSyncConfig, DeclarativeConnectorEntry, ErrorMappingConfig, ErrorMappingRule, HealthCheckConfig, RetryConfig, SyncStrategy, WebhookConfig, WebhookEvent, WebhookSignatureAlgorithm } from '@objectstack/spec/integration';
139139

140140
// Validate data
141141
const result = CircuitBreakerConfigSchema.parse(data);
@@ -181,7 +181,7 @@ Circuit breaker configuration
181181
| **syncConfig** | `{ strategy?: Enum<'full' \| 'incremental' \| 'upsert' \| 'append_only'>; direction?: Enum<'import' \| 'export' \| 'bidirectional'>; schedule?: string \| { dialect: Enum<'cel' \| 'cron' \| 'template'>; source?: string; ast?: any; meta?: object }; realtimeSync?: boolean; … }` | optional | Data sync configuration |
182182
| **fieldMappings** | `{ source: string; target: string; transform?: { type: 'constant'; value: any } \| { type: 'cast'; targetType: Enum<'string' \| 'number' \| 'boolean' \| 'date'> } \| { type: 'lookup'; table: string; keyField: string; valueField: string } \| { type: 'javascript'; expression: string \| { dialect: Enum<'cel' \| 'cron' \| 'template'>; source?: string; ast?: any; meta?: object } } \| { type: 'map'; mappings: Record<string, any> }; defaultValue?: any; … }[]` | optional | Field mapping rules |
183183
| **webhooks** | `{ name: string; label?: string; object?: string; triggers?: Enum<'create' \| 'update' \| 'delete' \| 'bulk_update' \| 'bulk_delete'>[]; … }[]` | optional | Webhook configurations (not yet enforced — never read at registration; see #3197) |
184-
| **rateLimitConfig** | `{ strategy?: Enum<'fixed_window' \| 'sliding_window' \| 'token_bucket' \| 'leaky_bucket'>; maxRequests: number; windowSeconds: number; burstCapacity?: number; … }` | optional | Rate limiting configuration |
184+
| **rateLimitConfig** | `any` | optional | [REMOVED] `connector.rateLimitConfig` was removed in @objectstack/spec 17.0.0 (#4911, ADR-0049 D2) — the entire shape is gone, not just this key: `ConnectorRateLimitConfig` and its `RateLimitStrategy` enum were removed with it, because no outbound rate-limiting engine ever existed. The platform's only token bucket (runtime `security/rate-limit.ts`) throttles INBOUND requests to us; nothing throttled the calls a connector makes out, so every knob here was inert while reading like a configured cap. Delete the key. Do NOT substitute `shared` `RateLimitConfig` — that is the inbound limiter and would cap the wrong direction; until an outbound throttle exists, rate-limit at the connector provider or upstream gateway. Run `os migrate meta --from 16` to rewrite it automatically. |
185185
| **retryConfig** | `{ strategy?: Enum<'exponential_backoff' \| 'linear_backoff' \| 'fixed_delay' \| 'no_retry'>; maxAttempts?: number; initialDelayMs?: number; maxDelayMs?: number; … }` | optional | Retry configuration |
186186
| **connectionTimeoutMs** | `number` | optional | Connection timeout in ms |
187187
| **requestTimeoutMs** | `number` | optional | Request timeout in ms |
@@ -270,22 +270,6 @@ Connector health configuration
270270
| **circuitBreaker** | `{ enabled: boolean; failureThreshold: number; resetTimeoutMs: number; halfOpenMaxRequests: number; … }` | optional | Circuit breaker configuration |
271271

272272

273-
---
274-
275-
## ConnectorRateLimitConfig
276-
277-
### Properties
278-
279-
| Property | Type | Required | Description |
280-
| :--- | :--- | :--- | :--- |
281-
| **strategy** | `Enum<'fixed_window' \| 'sliding_window' \| 'token_bucket' \| 'leaky_bucket'>` || Rate limiting strategy |
282-
| **maxRequests** | `number` || Maximum requests per window |
283-
| **windowSeconds** | `number` || Time window in seconds |
284-
| **burstCapacity** | `number` | optional | Burst capacity |
285-
| **respectUpstreamLimits** | `boolean` || Respect external rate limit headers |
286-
| **rateLimitHeaders** | `{ remaining: string; limit: string; reset: string }` | optional | Custom rate limit headers |
287-
288-
289273
---
290274

291275
## ConnectorRetryStrategy
@@ -386,7 +370,7 @@ Connector type
386370
| **syncConfig** | `{ strategy?: Enum<'full' \| 'incremental' \| 'upsert' \| 'append_only'>; direction?: Enum<'import' \| 'export' \| 'bidirectional'>; schedule?: string \| { dialect: Enum<'cel' \| 'cron' \| 'template'>; source?: string; ast?: any; meta?: object }; realtimeSync?: boolean; … }` | optional | Data sync configuration |
387371
| **fieldMappings** | `{ source: string; target: string; transform?: { type: 'constant'; value: any } \| { type: 'cast'; targetType: Enum<'string' \| 'number' \| 'boolean' \| 'date'> } \| { type: 'lookup'; table: string; keyField: string; valueField: string } \| { type: 'javascript'; expression: string \| { dialect: Enum<'cel' \| 'cron' \| 'template'>; source?: string; ast?: any; meta?: object } } \| { type: 'map'; mappings: Record<string, any> }; defaultValue?: any; … }[]` | optional | Field mapping rules |
388372
| **webhooks** | `{ name: string; label?: string; object?: string; triggers?: Enum<'create' \| 'update' \| 'delete' \| 'bulk_update' \| 'bulk_delete'>[]; … }[]` | optional | Webhook configurations (not yet enforced — never read at registration; see #3197) |
389-
| **rateLimitConfig** | `{ strategy?: Enum<'fixed_window' \| 'sliding_window' \| 'token_bucket' \| 'leaky_bucket'>; maxRequests: number; windowSeconds: number; burstCapacity?: number; … }` | optional | Rate limiting configuration |
373+
| **rateLimitConfig** | `any` | optional | [REMOVED] `connector.rateLimitConfig` was removed in @objectstack/spec 17.0.0 (#4911, ADR-0049 D2) — the entire shape is gone, not just this key: `ConnectorRateLimitConfig` and its `RateLimitStrategy` enum were removed with it, because no outbound rate-limiting engine ever existed. The platform's only token bucket (runtime `security/rate-limit.ts`) throttles INBOUND requests to us; nothing throttled the calls a connector makes out, so every knob here was inert while reading like a configured cap. Delete the key. Do NOT substitute `shared` `RateLimitConfig` — that is the inbound limiter and would cap the wrong direction; until an outbound throttle exists, rate-limit at the connector provider or upstream gateway. Run `os migrate meta --from 16` to rewrite it automatically. |
390374
| **retryConfig** | `{ strategy?: Enum<'exponential_backoff' \| 'linear_backoff' \| 'fixed_delay' \| 'no_retry'>; maxAttempts?: number; initialDelayMs?: number; maxDelayMs?: number; … }` | optional | Retry configuration |
391375
| **connectionTimeoutMs** | `number` | optional | Connection timeout in ms |
392376
| **requestTimeoutMs** | `number` | optional | Request timeout in ms |
@@ -452,20 +436,6 @@ Health check configuration
452436
| **healthyThreshold** | `number` || Consecutive successes before marking healthy |
453437

454438

455-
---
456-
457-
## RateLimitStrategy
458-
459-
Rate limiting strategy
460-
461-
### Allowed Values
462-
463-
* `fixed_window`
464-
* `sliding_window`
465-
* `token_bucket`
466-
* `leaky_bucket`
467-
468-
469439
---
470440

471441
## RetryConfig

docs/protocol-upgrade-guide.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -188,6 +188,8 @@ That kernel-side tombstone was then SUPERSEDED inside the same unreleased major
188188

189189
Finally it removes the script-body capability token 'crypto.hash' (#4391). Four layers declared it — the `HookBodyCapability` enum, the doc table beside it, the CLI extractor and `ScriptContext.crypto.hash` — and none implemented it: `installCtx` wired only `randomUUID`, so the one call the token authorised threw inside the VM every time. The build-time inference made it worse than an ordinary declared-but-unenforced key: writing `ctx.crypto.hash(...)` made the CLI ADD the capability for you, so `os build` went green on the body that was guaranteed to fail at the first record write. Removed rather than implemented (ADR-0049) — hashing inside the sandbox widens its capability and security-review surface, and a capability that throws on every use yet drew zero complaints in its whole life is its own liveness verdict. This is an enum VALUE, not a key, so there is no `retiredKey()` tombstone: the enum error map carries the prescription, keyed on the received value so that only the spelling which used to be legal is told it "was removed". The conversion strips the dead token from `body.capabilities` on hooks and actions; it deliberately does NOT touch the `ctx.crypto.hash(...)` call the body made under it, which never returned a value and which the author must delete. Hashing returns only WITH an implementation, through the capability admission process.
190190

191+
It also removes `connector.rateLimitConfig` and its whole shape (#4911). This one is not "declared but unread" — it is declared but UNIMPLEMENTED, one step worse. The only token bucket the platform owns (runtime `security/rate-limit.ts`) is INBOUND: the dispatcher calls `consume(key)` on a request fingerprint and answers 429. Nothing anywhere throttles the calls a connector makes OUT, and no provider — `connector-rest`, `connector-openapi`, `connector-mcp`, `connector-slack` — reads the key or has a seam that could. So `strategy`, `maxRequests`, `windowSeconds`, `burstCapacity`, `respectUpstreamLimits` and `rateLimitHeaders` parsed cleanly and capped nothing, on a surface where the author believed they had bounded their spend against a third party's quota. `ConnectorRateLimitConfig` and the `RateLimitStrategy` enum it embedded had no other consumer and are removed with the key, so importing either is TS2305 in v17 — the #4834 shape, and the same implementation-first ruling: the vocabulary comes back WITH the engine, in one change. It is deliberately NOT converted to `shared` `RateLimitConfig`, which limits the calls others make to US; #4684 split their names for precisely this confusion, and rewriting an outbound cap into an inbound one would throttle the wrong direction. Delete the key and rate-limit where the calls are actually made — the connector provider or upstream gateway.
192+
191193
### Mechanical (applied for you)
192194

193195
| Conversion | Surface | Change | Load window |
@@ -229,6 +231,7 @@ Finally it removes the script-body capability token 'crypto.hash' (#4391). Four
229231
| `object-managed-by-system-to-system-data` | `object.managedBy` | object managedBy 'system' → 'system-data' (#3355 — ADR-0103's residual bucket named the engine-owned half v16 had already moved out to `engine-owned`; the rename leaves the name describing what the bucket actually holds: admin/user-writable platform data) | retired — `migrate meta` only |
230232
| `object-enable-trash-mru-removed` | `object.enable.trash / object.enable.mru` | object capability flags 'enable.trash'/'enable.mru' removed (#3207, #2377 close-out — no recycle bin and no MRU tracking ever ran; both default-true flags gated nothing) | retired — `migrate meta` only |
231233
| `hook-body-crypto-hash-removed` | `hook.body.capabilities / action.body.capabilities` | script-body capability token 'crypto.hash' removed (#4391 — the sandbox never installed ctx.crypto.hash, so the token granted a call that always threw; the CLI inferred it too) | retired — `migrate meta` only |
234+
| `connector-rate-limit-config-removed` | `connector.rateLimitConfig` | connector key 'rateLimitConfig' removed (#4911 — no outbound rate-limiting engine exists; the runtime's only token bucket limits INBOUND requests, so every knob here was inert while reading like a configured cap. The whole ConnectorRateLimitConfig shape went with it) | retired — `migrate meta` only |
232235

233236
### Semantic (delegated to you, with acceptance criteria)
234237

packages/spec/api-surface.json

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3865,8 +3865,6 @@
38653865
"ConnectorOrigin (type)",
38663866
"ConnectorProviderContext (interface)",
38673867
"ConnectorProviderFactory (type)",
3868-
"ConnectorRateLimitConfig (type)",
3869-
"ConnectorRateLimitConfigSchema (const)",
38703868
"ConnectorRetryStrategy (type)",
38713869
"ConnectorRetryStrategySchema (const)",
38723870
"ConnectorSchema (const)",
@@ -3887,8 +3885,6 @@
38873885
"ErrorMappingRuleSchema (const)",
38883886
"HealthCheckConfig (type)",
38893887
"HealthCheckConfigSchema (const)",
3890-
"RateLimitStrategy (type)",
3891-
"RateLimitStrategySchema (const)",
38923888
"ResolvedConnectorAuth (type)",
38933889
"RetryConfig (type)",
38943890
"RetryConfigSchema (const)",

packages/spec/authorable-surface.json

Lines changed: 2 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -4073,7 +4073,7 @@
40734073
"integration/Connector:name",
40744074
"integration/Connector:provider",
40754075
"integration/Connector:providerConfig",
4076-
"integration/Connector:rateLimitConfig",
4076+
"integration/Connector:rateLimitConfig [RETIRED]",
40774077
"integration/Connector:requestTimeoutMs",
40784078
"integration/Connector:retryConfig",
40794079
"integration/Connector:status",
@@ -4105,12 +4105,6 @@
41054105
"integration/ConnectorInstanceBearerAuth:credentialRef",
41064106
"integration/ConnectorInstanceBearerAuth:type",
41074107
"integration/ConnectorInstanceNoAuth:type",
4108-
"integration/ConnectorRateLimitConfig:burstCapacity",
4109-
"integration/ConnectorRateLimitConfig:maxRequests",
4110-
"integration/ConnectorRateLimitConfig:rateLimitHeaders",
4111-
"integration/ConnectorRateLimitConfig:respectUpstreamLimits",
4112-
"integration/ConnectorRateLimitConfig:strategy",
4113-
"integration/ConnectorRateLimitConfig:windowSeconds",
41144108
"integration/ConnectorTrigger:description",
41154109
"integration/ConnectorTrigger:interval",
41164110
"integration/ConnectorTrigger:key",
@@ -4140,7 +4134,7 @@
41404134
"integration/DeclarativeConnectorEntry:name",
41414135
"integration/DeclarativeConnectorEntry:provider",
41424136
"integration/DeclarativeConnectorEntry:providerConfig",
4143-
"integration/DeclarativeConnectorEntry:rateLimitConfig",
4137+
"integration/DeclarativeConnectorEntry:rateLimitConfig [RETIRED]",
41444138
"integration/DeclarativeConnectorEntry:requestTimeoutMs",
41454139
"integration/DeclarativeConnectorEntry:retryConfig",
41464140
"integration/DeclarativeConnectorEntry:status",

packages/spec/docs/SYNC_ARCHITECTURE.md

Lines changed: 3 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -264,14 +264,8 @@ const sapConnector: Connector = {
264264
}
265265
],
266266

267-
// Rate Limiting
268-
rateLimitConfig: {
269-
strategy: 'token_bucket',
270-
maxRequests: 100,
271-
windowSeconds: 60,
272-
burstCapacity: 150,
273-
respectUpstreamLimits: true
274-
},
267+
// (`rateLimitConfig` sat here until #4911 retired it — no outbound
268+
// rate-limiting engine ever existed. Throttle at the provider/gateway.)
275269

276270
// Retry Configuration
277271
retryConfig: {
@@ -404,7 +398,7 @@ const pipeline: ETLPipeline = {
404398
const connector: Connector = {
405399
authentication: { type: 'oauth2', ... },
406400
webhooks: [...],
407-
rateLimitConfig: { ... }
401+
retryConfig: { ... }
408402
};
409403
```
410404

0 commit comments

Comments
 (0)