Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,8 @@ Every fork edit inside a shared upstream file is wrapped in sentinel comments:
Find them all: `grep -rn "OPENFRAME(" --include='*.go' --include='*.yaml' --include='*.tpl' .`
Net-new fork-only code lives under `openframe/`, `server/service/openframe/`,
`server/datastore/mysql/migrations/openframe/`, `server/datastore/redis/keyprefix.go`,
and `server/fleet/openframe.go`.
`server/fleet/openframe.go`, `server/datastore/mysql/openframe.go`, and
`server/service/openframe_middleware.go`.

## Syncing from upstream (the important workflow)

Expand Down
13 changes: 13 additions & 0 deletions charts/fleet/templates/configmap.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -22,3 +22,16 @@ data:
{{ .Values.database.databaseKey }}: {{ .Values.database.database | quote }}
{{ .Values.database.usernameKey }}: {{ .Values.database.username | quote }}
{{- end }}
---
{{- if not .Values.fleet.openframe.multiTenancy.existingConfigMap }}
# >>> OPENFRAME(mysql-multitenancy): chart-managed ConfigMap holding the tenant UUID — openframe/docs/helm-chart.md
apiVersion: v1
kind: ConfigMap
metadata:
name: fleet-openframe-tenant
labels:
{{- include "fleet.labels" . | nindent 4 }}
data:
{{ .Values.fleet.openframe.multiTenancy.tenantUuidKey }}: {{ .Values.fleet.openframe.multiTenancy.tenantUuid | quote }}
{{- end }}
# <<< OPENFRAME(mysql-multitenancy)
16 changes: 16 additions & 0 deletions charts/fleet/templates/deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -209,6 +209,22 @@ spec:
- name: FLEET_OPENFRAME_MODE
value: {{ .Values.fleet.setup.openframeMode | quote }}
# <<< OPENFRAME(helm)
# >>> OPENFRAME(mysql-multitenancy): shared-DB multitenancy feature envs — openframe/docs/helm-chart.md
# Master switch (maps openframe.fleet.multi-tenancy.enabled). "false" ⇒ pre-feature
# fork behavior. "true" + tenant UUID ⇒ pinned mode (one Fleet per tenant);
# "true" + no tenant UUID ⇒ shared per-request mode (one Fleet per cluster, fail closed).
- name: FLEET_OPENFRAME_MULTI_TENANCY_ENABLED
value: {{ .Values.fleet.openframe.multiTenancy.enabled | quote }}
- name: FLEET_OPENFRAME_TENANT_UUID
valueFrom:
configMapKeyRef:
name: {{ default "fleet-openframe-tenant" .Values.fleet.openframe.multiTenancy.existingConfigMap }}
key: {{ .Values.fleet.openframe.multiTenancy.tenantUuidKey }}
{{- if .Values.fleet.openframe.multiTenancy.teamId }}
- name: FLEET_OPENFRAME_TEAM_ID
value: {{ .Values.fleet.openframe.multiTenancy.teamId | quote }}
{{- end }}
Comment thread
ivan-flamingo marked this conversation as resolved.
# <<< OPENFRAME(mysql-multitenancy)
## END FLEET SECTION
## BEGIN MYSQL SECTION
# >>> OPENFRAME(helm): fork-externalized DB connection from ConfigMap/Secret refs — openframe/docs/helm-chart.md
Expand Down
6 changes: 6 additions & 0 deletions charts/fleet/templates/job-migration.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,12 @@ spec:
- name: FLEET_SERVER_KEY
value: "/secrets/tls/{{ .Values.fleet.tls.keySecretKey }}"
{{- end }}
# >>> OPENFRAME(mysql-multitenancy): with multitenancy on, `fleet prepare db`
# serializes concurrent migration runs against the shared MySQL via a named lock
# (GET_LOCK) — the flag must reach the migration Job, not only the server.
- name: FLEET_OPENFRAME_MULTI_TENANCY_ENABLED
value: {{ .Values.fleet.openframe.multiTenancy.enabled | quote }}
# <<< OPENFRAME(mysql-multitenancy)
## END FLEET SECTION
## BEGIN MYSQL SECTION
- name: FLEET_MYSQL_HOST
Expand Down
9 changes: 9 additions & 0 deletions charts/fleet/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,15 @@ fleet:
existingSecret: "" # Name of a K8s Secret. If set, secretKeyValue is ignored.
secretKeyKey: "FLEET_SETUP_ADMIN_PASSWORD" # Key name within the secret to read the value from.
secretKeyValue: "fleet" # Plain text password (for dev/test only). Ignored if secret is set.
# >>> OPENFRAME(mysql-multitenancy): OpenFrame feature block.
openframe:
multiTenancy:
enabled: false
tenantUuid: "" # pinned mode: tenant UUID, stored in the chart-managed ConfigMap
existingConfigMap: "" # optional: read the tenant UUID from your own ConfigMap instead
tenantUuidKey: "FLEET_OPENFRAME_TENANT_UUID" # key holding the UUID in either ConfigMap
teamId: "" # escape hatch: direct FLEET_OPENFRAME_TEAM_ID pin (prefer tenantUuid)
# <<< OPENFRAME(mysql-multitenancy)
mdm:
windows:
wstepIdentityCertKey: ""
Expand Down
21 changes: 21 additions & 0 deletions cmd/fleet/prepare.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import (
"encoding/json"
"fmt"
"os"
"time"

"github.com/WatchBeam/clock"
"github.com/fleetdm/fleet/v4/server/config"
Expand Down Expand Up @@ -32,6 +33,13 @@ To setup Fleet infrastructure, use one of the available commands.
// Whether to show table stats before and after the migration
showTableStats := false

// >>> OPENFRAME(mysql-multitenancy): how long a `prepare db` run waits for a concurrent
// migration run against the same shared MySQL to finish before giving up. Generous because
// index builds on a large shared hosts table can take minutes; a K8s Job that fails here is
// retried by its backoff policy anyway.
const openframeMigrationLockWait = 15 * time.Minute
// <<< OPENFRAME(mysql-multitenancy)

dbCmd := &cobra.Command{
Use: "db",
Short: "Given correct database configurations, prepare the databases for use",
Expand All @@ -49,6 +57,19 @@ To setup Fleet infrastructure, use one of the available commands.
initFatal(err, "creating db connection")
}

// >>> OPENFRAME(mysql-multitenancy): on a shared MySQL, serialize `prepare db`
// runs across clusters/jobs/replicas with a named MySQL lock — Fleet's goose has
// no advisory lock, so concurrent runs race on DDL. Held on a dedicated session
// (auto-released if the job dies). Flag-off runs are untouched.
if fleet.IsOpenframeMultitenancy() {
release, err := ds.AcquireOpenframeMigrationLock(cmd.Context(), openframeMigrationLockWait)
if err != nil {
initFatal(err, "acquiring openframe migration lock")
}
defer release()
}
// <<< OPENFRAME(mysql-multitenancy)

status, err := ds.MigrationStatus(cmd.Context())
if err != nil {
initFatal(err, "retrieving migration status")
Expand Down
37 changes: 37 additions & 0 deletions cmd/fleet/serve.go
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,15 @@ func runServeCmd(cmd *cobra.Command, configManager configpkg.Manager, debug, dev
applyDevFlags(&config)
}

// >>> OPENFRAME(mysql-multitenancy): validate the multitenancy configuration — with
// FLEET_OPENFRAME_MULTI_TENANCY_ENABLED on, a pinned process (tenant UUID/team id) and an
// unpinned shared-mode process are both valid, but a set-yet-unparsable team pin refuses to
// boot so a typo cannot silently change the isolation mode.
if err := fleet.ValidateOpenframeMultitenancy(); err != nil {
initFatal(err, "validating OpenFrame multitenancy configuration")
}
// <<< OPENFRAME(mysql-multitenancy)

license, err := initLicense(&config, devLicense, devExpiredLicense)
if err != nil {
initFatal(
Expand Down Expand Up @@ -263,6 +272,26 @@ func runServeCmd(cmd *cobra.Command, configManager configpkg.Manager, debug, dev
os.Exit(1)
}

// >>> OPENFRAME(mysql-multitenancy): pinned mode — resolve the Flamingo tenant UUID to its
// Fleet team id (create-if-absent via the teams.openframe_tenant_uuid bridge) and pin the
// process. The pin is then read by fleet.OpenframeTeamID everywhere the datastore fences scope
// by team. In shared mode there is no process pin: every request is pinned individually by the
// tenant middleware / host auth / enroll secret. Runs only under
// FLEET_OPENFRAME_MULTI_TENANCY_ENABLED; requires the openframe migrations to be applied (prepare db).
if fleet.IsOpenframeMultitenancy() {
if tenantUUID, ok := fleet.OpenframeTenantUUID(); ok {
teamID, err := mds.EnsureOpenframeTeamID(cmd.Context(), tenantUUID)
if err != nil {
initFatal(err, "resolving OpenFrame tenant team from FLEET_OPENFRAME_TENANT_UUID")
}
fleet.SetOpenframeTeamID(teamID)
logger.InfoContext(cmd.Context(), "OpenFrame multitenancy: pinned mode", "tenant_uuid", tenantUUID, "team_id", teamID)
} else if fleet.IsOpenframeSharedMode() {
logger.InfoContext(cmd.Context(), "OpenFrame multitenancy: shared per-request mode (no process pin)")
}
}
// <<< OPENFRAME(mysql-multitenancy)

if initializingDS, ok := ds.(initializer); ok {
if err := initializingDS.Initialize(); err != nil {
initFatal(err, "loading built in data")
Expand Down Expand Up @@ -730,6 +759,14 @@ func runServeCmd(cmd *cobra.Command, configManager configpkg.Manager, debug, dev
}
apiHandler = service.WithMDMSSOCallbackRedirect(svc, logger, apiHandler)

// >>> OPENFRAME(mysql-multitenancy): shared mode — pin every control-plane API request
// to the tenant team named by the gateway-injected X-Tenant-Id header (fail closed for
// non-agent paths without it). Returns apiHandler unchanged unless the process runs in
// shared per-request mode. Must come after apiendpoints.Validate, which type-asserts the
// raw *mux.Router.
apiHandler = service.WithOpenframeTenant(mds, logger, apiHandler)
// <<< OPENFRAME(mysql-multitenancy)

if serveCSP {
// Only injecting this if CSP is turned on since the default security headers add some overhead to each request
apiHandler = endpointer.BrowserSecurityHeadersHandler(serveCSP, apiHandler)
Expand Down
47 changes: 46 additions & 1 deletion openframe/docs/helm-chart.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,50 @@ assignments and the query-results TTL cleanup
(see [architecture-host-assignments.md](architecture-host-assignments.md),
[query-results-ttl-cleanup.md](query-results-ttl-cleanup.md)).

## MySQL-multitenancy feature envs (`mysql-multitenancy`)

`values.yaml` exposes a `fleet.openframe.multiTenancy` block — the chart-side wiring of the
platform property `openframe.fleet.multi-tenancy.enabled` (see
[process-team-pin.md](process-team-pin.md) for the flag/mode semantics):

```yaml
fleet:
openframe:
multiTenancy:
enabled: false # → FLEET_OPENFRAME_MULTI_TENANCY_ENABLED (deployment + migration Job)
tenantUuid: "" # pinned mode: static FLEET_OPENFRAME_TENANT_UUID
existingConfigMap: "" # pinned mode: read the UUID from a ConfigMap instead (wins over tenantUuid)
tenantUuidKey: "" # key within existingConfigMap (default FLEET_OPENFRAME_TENANT_UUID)
teamId: "" # escape hatch: direct FLEET_OPENFRAME_TEAM_ID pin (prefer tenantUuid)
```

Rendered env vars ([deployment.yaml](../../charts/fleet/templates/deployment.yaml)):
`FLEET_OPENFRAME_MULTI_TENANCY_ENABLED` is always emitted (`"false"` by default — pre-feature
fork behavior). `FLEET_OPENFRAME_TENANT_UUID` is **always read via `configMapKeyRef`** — there is
no inline value or branching in the Deployment (same pattern as the DB/cache config) — from one of:
- `existingConfigMap` set → the operator's own ConfigMap;
- `existingConfigMap` unset → the chart-managed **`fleet-openframe-tenant`** ConfigMap that
[configmap.yaml](../../charts/fleet/templates/configmap.yaml) creates, holding `tenantUuid`.

In shared per-request mode (`enabled: true`, no `tenantUuid`) and flag-off, the value is empty and
Fleet treats `""` as unset — so no pin. The **flag is also emitted into
[job-migration.yaml](../../charts/fleet/templates/job-migration.yaml)** so `fleet prepare db`
takes the `GET_LOCK` serialization on a shared MySQL.

Downstream (openframe-saas-tenant) pinned-mode wiring can reuse the existing per-namespace
`tenant` ConfigMap, whose `TENANT_ID` key already holds the tenant UUID (it is the same key the
Redis prefix reads):

```yaml
fleetmdm:
fleet:
openframe:
multiTenancy:
enabled: true
existingConfigMap: "tenant"
tenantUuidKey: "TENANT_ID"
```

## Externalized configuration (ConfigMaps + Secrets)

Where upstream takes Redis/MySQL connection details as plain Helm values, the fork
Expand All @@ -49,6 +93,7 @@ just references them.
|---------|-------------------|--------------------------------------------------|-------------------------------------|
| Database | `database.*` | `database.existingConfigMap`, `database.existingSecret` | `fleet-database` ConfigMap (host/port/db/user) + Secret (password) |
| Cache (Redis) | `cache.*` | `cache.existingConfigMap` | `fleet-cache` ConfigMap (address, key prefix) |
| Tenant UUID (multi-tenancy) | `fleet.openframe.multiTenancy.*` | `fleet.openframe.multiTenancy.existingConfigMap` | `fleet-openframe-tenant` ConfigMap (`FLEET_OPENFRAME_TENANT_UUID` = `tenantUuid`, empty in shared mode) |
| Admin setup | `fleet.setup.*` | `fleet.setup.adminPassword.existingSecret` | `fleet-setup` Secret (`FLEET_SETUP_ADMIN_PASSWORD`) |

Keys within the referenced ConfigMap are themselves configurable
Expand Down Expand Up @@ -218,7 +263,7 @@ helm upgrade --install fleet oci://ghcr.io/flamingo-stack/fleetmdm/helm-charts/f
| `charts/fleet/values.yaml` | OpenFrame mode, externalized DB/cache/setup config, `cache.keyPrefixKey`, `waitForMysql`, `additionalCAs`, `vulnProcessing`, `deploymentAnnotations` |
| `charts/fleet/templates/configmap.yaml` | **New** — generated DB/cache ConfigMaps |
| `charts/fleet/templates/secret.yaml` | **New** — generated DB password / admin-setup Secrets |
| `charts/fleet/templates/deployment.yaml` | `FLEET_OPENFRAME_MODE`, `FLEET_REDIS_KEY_PREFIX`, ConfigMap/Secret refs, annotations, CA init container |
| `charts/fleet/templates/deployment.yaml` | `FLEET_OPENFRAME_MODE`, `FLEET_OPENFRAME_MULTI_TENANCY_ENABLED` / `FLEET_OPENFRAME_TENANT_UUID` / `FLEET_OPENFRAME_TEAM_ID`, `FLEET_REDIS_KEY_PREFIX`, ConfigMap/Secret refs, annotations, CA init container |
| `charts/fleet/templates/job-migration.yaml` | `waitForMysql` init container, hook removal, TTL removal |
| `charts/fleet/templates/vulnprocessing/cronjob.yaml` | Dedicated vuln-processing cron + `FLEET_REDIS_KEY_PREFIX`, feed-cache PVC mount, fsGroup, schedule stagger (moved from `templates/cron-vulnprocessing.yaml`) |
| `charts/fleet/templates/vulnprocessing/pvc.yaml` | **New** — PVC persisting the vulnerability feed cache across cron runs |
Expand Down
Loading
Loading