Update docs for paced scheduled portfolio analysis

docs/concepts/architecture/design/vulnerability-analysis.md

@@ -27,11 +27,11 @@ Three triggers can start vulnerability analysis. Each trigger creates a run
 of the outer `analyze-project` workflow, which calls the `vuln-analysis` workflow
 described in this document, followed by project policy evaluation and metrics update.
 
-| Trigger    | Workflow instance ID                              | Concurrency key                 | Priority    |
-|:-----------|:--------------------------------------------------|:--------------------------------|:------------|
-| Scheduled  | `analyze-project-scheduled:<projectUuid>`         | `analyze-project:<projectUuid>` | 0 (default) |
-| BOM upload | `analyze-project:bom-upload:<bomUploadToken>`     | `analyze-project:<projectUuid>` | 50          |
-| Manual     | `analyze-project-manual:<projectUuid>`            | `analyze-project:<projectUuid>` | 75          |
+| Trigger    | Workflow instance ID                          | Concurrency key                 | Priority    |
+|:-----------|:----------------------------------------------|:--------------------------------|:------------|
+| Scheduled  | `analyze-project-scheduled:<projectUuid>`     | `analyze-project:<projectUuid>` | 0 (default) |
+| BOM upload | `analyze-project:bom-upload:<bomUploadToken>` | `analyze-project:<projectUuid>` | 50          |
+| Manual     | `analyze-project-manual:<projectUuid>`        | `analyze-project:<projectUuid>` | 75          |
 
 All triggers share the same concurrency key pattern, which serializes analysis runs per project
 regardless of the trigger. Only one run per project can be active at a time.
@@ -43,7 +43,10 @@ Scheduled and manual triggers use project-scoped instance IDs to deduplicate con
 requests for the same project.
 
 The engine processes higher priority values first, so manual triggers (75) take precedence
-over BOM uploads (50), which take precedence over scheduled runs (0).
+over BOM uploads (50), which take precedence over scheduled runs (0). Priority governs admission,
+not preemption. A run that already holds a project's concurrency key keeps it until it completes,
+so a higher priority run for the same project waits.
+[Pacing of the scheduled trigger](#scheduled-trigger-pacing) keeps that wait bounded.
 
 When `analyze-project` invokes `vuln-analysis`, it sets the nested workflow's concurrency
 key to `vuln-analysis:<projectUuid>`. This isolates the vulnerability analysis stage from
@@ -52,6 +55,20 @@ the surrounding policy and metrics stages.
 Refer to the [durable execution](durable-execution.md) documentation for details
 on how the engine enforces concurrency keys, instance IDs, and priorities.
 
+### Scheduled trigger pacing
+
+The scheduled trigger does not start the whole portfolio at a fixed time.
+Instead, each project carries the time of its last analysis.
+Every 60 seconds the task looks for projects whose last analysis started longer ago than
+[`dt.task.portfolio-analysis.max-analysis-age-ms`](../../../reference/configuration/properties.md#dttaskportfolio-analysismax-analysis-age-ms)
+(24 hours by default), and starts the ones with the oldest attempt first.
+It stops once [`dt.task.portfolio-analysis.max-in-flight-analyses`](../../../reference/configuration/properties.md#dttaskportfolio-analysismax-in-flight-analyses)
+of its own runs are active (50 by default).
+
+All three triggers update that timestamp, so a project that clients analyze often never comes due
+on the schedule. This makes the scheduled trigger a floor on analysis frequency rather than a second
+source of load.
+
 ## Workflow execution
 
 The `vuln-analysis` workflow orchestrates the full analysis lifecycle:

docs/guides/upgrading/v5.1.0.md

@@ -1,9 +1,23 @@
 # Upgrading to v5.1.0
 
 !!! warning "The database migration runs longer than usual"
-    This release rebuilds the global portfolio metrics view and populates a new table tracking package
-    metadata resolution state. Both scale with portfolio size. Plan for extra startup time, and do not
-    interrupt the API server while the migration runs.
+    This release rebuilds the global portfolio metrics view and populates new tables tracking package
+    metadata resolution state and per-project analysis timestamps. Each scales with portfolio size.
+    Plan for extra startup time, and do not interrupt the API server while the migration runs.
+
+* **The portfolio analysis no longer runs at a fixed time**. Earlier versions started the analysis of the entire
+  portfolio once a day, which created an analysis run for every project within minutes, and delayed analyses that
+  BOM uploads triggered in the meantime. Each project now carries the time of its last analysis attempt.
+  The task runs continuously and picks up projects whose last analysis attempt is longer ago than
+  [`dt.task.portfolio-analysis.max-analysis-age-ms`](../../reference/configuration/properties.md#dttaskportfolio-analysismax-analysis-age-ms),
+  24 hours by default. It keeps at most [`dt.task.portfolio-analysis.max-in-flight-analyses`](../../reference/configuration/properties.md#dttaskportfolio-analysismax-in-flight-analyses)
+  of its own analyses running at a time, 50 by default. The limit applies to the task alone. BOM uploads and API
+  calls start analyses regardless of it, and every analysis updates that timestamp no matter what started it.
+  A project that gets analysed often never becomes due for scheduled analysis.
+
+    !!! warning
+        The API server refuses to start when your configuration still contains `dt.task.portfolio-analysis.cron`.
+        Remove the property, and set the two replacements to match the throughput your deployment can sustain.
 
 * **Database queries now time out**. The API server aborts any single database query that runs longer than 60 seconds.
   In earlier versions, queries ran unbounded, so a stuck or runaway query could occupy a connection indefinitely and

docs/reference/configuration/task-scheduler.md

@@ -51,7 +51,6 @@ minute.
 |-----------------------------------|-------------------------------------------------------------------------------------------------------------------------|------------------|-----------------------------------------------------------------------------------------------|
 | Portfolio metrics update          | [`dt.task.portfolio-metrics-update.cron`](properties.md#dttaskportfolio-metrics-updatecron)                             | `10 * * * *`     | Refreshes per-project and portfolio time series metrics.                                      |
 | Vulnerability metrics update      | [`dt.task.vuln-metrics-update.cron`](properties.md#dttaskvuln-metrics-updatecron)                                       | `0 */6 * * *`    | Refreshes counters used by the vulnerability dashboard.                                       |
-| Portfolio vulnerability analysis  | [`dt.task.portfolio-analysis.cron`](properties.md#dttaskportfolio-analysiscron)                                         | `0 6 * * *`      | Re-analyzes every component in the portfolio against current vulnerability data.              |
 | NVD mirror[^1]                    | [`dt.task.nvd-vuln-data-source-mirror.cron`](properties.md#dttasknvd-vuln-data-source-mirrorcron)                       | `0 4 * * *`      | Mirrors the NIST National Vulnerability Database.                                             |
 | GitHub Advisories mirror[^1]      | [`dt.task.github-advisory-vuln-data-source-mirror.cron`](properties.md#dttaskgithub-advisory-vuln-data-source-mirrorcron) | `0 2 * * *`    | Mirrors the GitHub Advisory Database.                                                         |
 | OSV mirror[^1]                    | [`dt.task.osv-vuln-data-source-mirror.cron`](properties.md#dttaskosv-vuln-data-source-mirrorcron)                       | `0 3 * * *`      | Mirrors the OSV vulnerability database.                                                       |