Skip to content

feat: add support for Temporal Server v1.29, v1.30 and v1.31 - #987

Open
Azerothian wants to merge 16 commits into
alexandrevilain:mainfrom
CrunchyMonkies:feat/temporal-1.31
Open

feat: add support for Temporal Server v1.29, v1.30 and v1.31#987
Azerothian wants to merge 16 commits into
alexandrevilain:mainfrom
CrunchyMonkies:feat/temporal-1.31

Conversation

@Azerothian

@Azerothian Azerothian commented Jul 7, 2026

Copy link
Copy Markdown

This PR adds operator support for Temporal Server v1.29, v1.30 and v1.31. It was developed as a chained series (each version on top of the previous) and is presented here as a single PR spanning three minors — happy to split it into per-version PRs (1.29 → 1.30 → 1.31) if you'd prefer.

All >= 1.30 behaviour is version-gated (version.V1_30_0 / V1_31_0); clusters < 1.30 keep the previous dockerize/curl paths unchanged, so the operator stays backward compatible across >= 1.14.0 < 1.32.0.

v1.29

Version range/default bumps only. v1.29 is a dynamic-config-only release (task-queue fairness & task-queue config API), already covered by the cluster dynamicConfig field.

v1.30

Handles the breaking image changes:

  • dockerize/auto-setup were removed from temporalio/server and config templating moved into the server binary. For >= 1.30 the operator emits the # enable-template header + sprig {{ env "X" }} placeholders, sets TEMPORAL_SERVER_CONFIG_FILE_PATH, and selects the service via TEMPORAL_SERVICES (legacy SERVICES kept for compatibility).
  • curl/jq were removed from temporalio/admin-tools, which broke ES visibility setup. The operator now drives ES visibility via the temporal-elasticsearch-tool shipped in the image (setup-schema/create-index/update-schema); its embedded template applies all built-in search attributes automatically. The MTLS sidecar-shutdown step uses wget instead of curl.
  • v1.30.0 (no upstream GitHub release) is rejected as a broken release.

v1.31

New sql.passwordCommand datastore field (external command / cloud IAM auth token), wired into both the server config and the schema-setup jobs, webhook-validated and mutually exclusive with passwordSecretRef.

Build / deps

  • go directive → 1.26.4 (Dockerfile builder image bumped to match).
  • go.temporal.io/server v1.31.1, go.temporal.io/api v1.62.8, go.temporal.io/sdk v1.41.1, and associated Kubernetes dependency bumps.

Update: follow-up fixes

The original single commit has been extended with 15 commits that fix defects found while getting this work to build, lint and pass e2e downstream. Almost all of the diagnostic work is @johnduhart's, from johnduhart#1 — the commits are cherry-picked with -x and keep his authorship and commit messages, which carry the reasoning and evidence.

Correctness fixes to the 1.29/1.30/1.31 work itself:

  • Elasticsearch schema scripts — the most serious one. An empty ES username rendered a bare --user, and urfave/cli consumed the next token (setup-schema) as its value; the tool printed help and exited 0, so the schema job was recorded successful while creating neither the index template nor the visibility index. Also fixes set -eu skipping the sidecar-shutdown footer (job pod hangs Running forever under istio/linkerd) and getStoreTool returning temporal-elasticsearch-tool for admin-tools <= 1.29, where that binary does not exist.
  • e2e cluster version — this PR moved newDatastoreVersion to 1.31.1 while extending defaultUpgradePath to start at 1.25.2, making the first upgrade step a six-minor downgrade that ValidateUpdate rejects. Restored to 1.24.3.
  • Broken-release handling — adds retracted v1.26.0/v1.26.1 to ForbiddenBrokenReleases, and replaces the IncPatch() upgrade suggestion (which could point at another broken release — 1.26.0 suggested 1.26.1) with NextNonBrokenPatch. Also memoizes the semver constraints in GreaterOrEqual/LessThan, which were recompiled on every call with the parse error discarded.
  • SQLPasswordCommandSpec.Timeout → pointer — a value metav1.Duration is always serialized, so every reconcile rewrote the field and churned the webhook.
  • Single source for the server config file path/etc/temporal/config/config_template.yaml was written in four unrelated places (env var, MountPath, SubPath, ConfigMap key) with nothing tying them together. On >= 1.30 there is no built-in fallback location, so any mismatch means the server exits at startup.
  • passwordCommand admission warning — the command must already exist in the admin-tools image; the schema jobs have no override point. The webhook now says so rather than letting setup fail with an opaque password-authentication error.

Build/CI repairs, all consequences of this PR's own go 1.24.5 → 1.26.4 bump:

  • golangci-lint v1.64.8 → v2.12.2 (v1.64.8 is the last v1 release and hard-fails typecheck under Go 1.26; .golangci.yaml migrated to the v2 schema, action v6 → v9), and lint now runs without its analysis cache — a warm cache produced both a false 0 issues on a tree that had one and a false "unused nolint directive" on directives that were genuinely needed.
  • controller-gen v0.16.3 → v0.21.0 (v0.16.3 cannot compile under Go 1.26). Regenerated CRDs are identical bar one real fix: Cassandra consistency/serialConsistency move from type: integer to type: string. They are *gocql.Consistency/*gocql.SerialConsistency, which marshal as names like LOCAL_QUORUM; the old controller-gen derived the schema from the underlying uint16, so the API server was rejecting the documented values.
  • controller-runtime v0.21.0 → v0.23.3 — this PR forced client-go to 0.35.1 but left controller-runtime targeting 0.33.0. The generic webhook builder lets the four methods in webhooks/temporalcluster_webhook.go take *v1beta1.TemporalCluster directly, deleting getClusterFromRequest and its five call sites. mgr.GetEventRecorderFor is kept with a documented //nolint:staticcheck: its replacement moves events to events.k8s.io/v1, a behaviour change that does not belong in a dependency bump.
  • Remaining Makefile tool pins refreshed, including kustomize v4.5.7 → v5.8.1 (patchesStrategicMergepatches, varsreplacements). Verified output-neutral: make artifacts is byte-identical across all four kustomize roots before and after, with zero deprecation warnings.
  • The tool install recipes now hang off .PHONY targets rather than $(LOCALBIN)/<tool>. As file targets the recipe is skipped whenever the binary already exists, so the embedded version check never runs and bumping a pin silently has no effect for anyone with a warm bin/ — which is exactly how a v4 kustomize survived the v5 bump locally and emitted unsubstituted CERTIFICATE_NAMESPACE/CERTIFICATE_NAME placeholders.
  • ENVTEST_K8S_VERSION shadowing fixed. An unconditional ENVTEST_K8S_VERSION = 1.28.0 at the top of the Makefile made the ?= derivation from k8s.io/api further down a permanent no-op. With k8s.io/api at v0.35.1, envtest had been running 1.28 assets against a 1.35 client — seven minors of API skew under every envtest test. It now resolves to 1.35.0.
  • GOTOOLCHAIN is pinned from go.mod when building golangci-lint. Its own go.mod requires a lower Go than this module targets, so a plain go install builds it with that lower toolchain and it then refuses to run: "the Go language version used to build golangci-lint is lower than the targeted Go version".

Not included

This PR deliberately does not carry #986 or #988, which are open PRs from other contributors. Two further fixes from @johnduhart's branch patch code introduced by those PRs — a narrowing bug in #988's integer normalisation (a 5 GiB blob limit wraps on 32-bit, and exponent-form input like 1e9 falls through to float and is re-emitted as 1e+09, the exact bug #988 exists to prevent), and the fact that #986's merge-existing-first makes every operator-managed metadata key permanent (removing spec.mTLS leaves istio injecting sidecars indefinitely). Happy to open those as follow-ups against whichever PR lands first — they're meaningless on their own.

Testing

  • make test (unit + envtest) green across all packages, now against 1.35 envtest assets rather than the stale 1.28 ones.
  • make lint — 0 issues from a cold cache.
  • make generate manifests and make helm produce no diff.
  • kustomize v5 migration verified byte-identical against captured v4 artifacts.
  • Validated end-to-end on a real RKE2 cluster (k8s v1.35): 1.29.7, 1.30.5 and 1.31.1 clusters each reach Ready=True with a working namespace-create + workflow round-trip; the 1.30 sprig/entrypoint contract, passwordCommand auth (incl. webhook rejection paths), and the temporal-elasticsearch-tool ES visibility setup (index + v10–v13 built-in search attributes) were each exercised against real images.

Note that tests/e2e/persistence_test.go still has an if name != "cassandra persistence" { continue } filter that skips the postgres12 default-store path; that filter predates this PR and is left as-is here.

Notes for reviewers

The go 1.26.4 toolchain requirement and dependency bumps are the widest-reaching parts; the ES visibility rewrite changes how ES-backed clusters bootstrap on >= 1.30. The build/CI commits are separable from the functional ones if you'd rather take them as their own PR — they're the last four commits and touch only Makefile, config/** and .github/workflows/**.

Adds operator support for Temporal Server v1.29, v1.30 and v1.31, extending the
supported version range to `>= 1.14.0 < 1.32.0` (default version 1.31.1, default
UI 2.49.1). All >= 1.30 behaviour is version-gated (version.V1_30_0 / V1_31_0);
clusters < 1.30 keep the previous dockerize/curl paths unchanged, so the operator
stays backward compatible across the whole supported range.

v1.29
- Version range/default bumps only. v1.29 is a dynamic-config-only release
  (task-queue fairness, task-queue config API), already covered by the cluster
  dynamicConfig field.

v1.30
- dockerize/auto-setup were removed from the temporalio/server image and config
  templating moved into the server binary (embedded sprig engine). For clusters
  >= 1.30 the operator now emits config templates with the `# enable-template`
  header and sprig `{{ env "X" }}` placeholders (instead of dockerize
  `{{ .Env.X }}`), sets TEMPORAL_SERVER_CONFIG_FILE_PATH, and selects the service
  via the new TEMPORAL_SERVICES env var (legacy SERVICES kept for compatibility).
- curl and jq were removed from the temporalio/admin-tools image, which broke the
  operator's Elasticsearch visibility setup scripts. For clusters >= 1.30 the
  operator now drives ES visibility through the temporal-elasticsearch-tool
  shipped in the image (setup-schema, create-index, update-schema), analogous to
  temporal-sql-tool. Its embedded index template applies all built-in search
  attributes automatically. The MTLS sidecar-shutdown step uses wget instead of
  curl on >= 1.30.
- v1.30.0 has no published GitHub release upstream (silently skipped) and is now
  rejected as a broken release; use v1.30.1+.

v1.31
- New sql.passwordCommand datastore field: resolves the datastore password by
  running an external command (e.g. to generate a short-lived cloud IAM auth
  token for AWS RDS / GCP Cloud SQL). Wired into both the rendered server config
  and the persistence schema-setup jobs (temporal-sql-tool via a shell command
  substitution). Mutually exclusive with passwordSecretRef and validated by the
  webhook (rejected on clusters < 1.31 and when combined with a password secret).

Build / dependencies
- go directive bumped to 1.26.4 with the Dockerfile builder image updated to
  match.
- go.temporal.io/server v1.31.1, go.temporal.io/api v1.62.8,
  go.temporal.io/sdk v1.41.1, plus the associated Kubernetes dependency bumps.

Testing
- make test (unit + envtest) is green, including new unit tests that load the
  generated 1.30 config through the real go.temporal.io/server config loader and
  cover the passwordCommand and ES-tool script rendering.
- Validated end-to-end on a real RKE2 cluster (k8s v1.35): 1.29.7, 1.30.5 and
  1.31.1 clusters each reach Ready=True with a working namespace-create +
  workflow round-trip; the 1.30 sprig/entrypoint contract, passwordCommand auth
  (including webhook rejection paths) and the temporal-elasticsearch-tool ES
  visibility setup (index + v10-v13 built-in search attributes) were exercised
  against real images.
@Azerothian
Azerothian marked this pull request as ready for review July 9, 2026 13:02
Comment thread .github/workflows/publish-release-ghcr.yaml Fixed
Comment thread .github/workflows/publish-release-ghcr.yaml Fixed
Comment thread .github/workflows/publish-release-ghcr.yaml Fixed
Comment thread .github/workflows/publish-release-ghcr.yaml Fixed
Comment thread .github/workflows/publish-release-ghcr.yaml Fixed
Comment thread .github/workflows/publish-release-ghcr.yaml Fixed
Comment thread .github/workflows/publish-release-ghcr.yaml Fixed
@Azerothian
Azerothian force-pushed the feat/temporal-1.31 branch from 532f731 to 4fb3cb7 Compare July 17, 2026 07:23
johnduhart added a commit to johnduhart/temporal-operator that referenced this pull request Aug 3, 2026
PR alexandrevilain#987 extended defaultUpgradePath with 1.29.7/1.30.5/1.31.1, which is
correct, but also moved newDatastoreVersion from 1.24.3 to 1.31.1.

Those two variables play opposite roles: newDatastoreVersion is the
version the cluster is CREATED at (persistence_test.go:110,157,262,307),
and defaultUpgradePath is the sequence it is then upgraded THROUGH. With
both at 1.31.1 the first upgrade step asks for 1.25.2, a six-minor
downgrade that ValidateUpdate rejects via UpgradeConstraint.

Restoring 1.24.3 makes the walk 1.24.3 -> 1.25.2 -> ... -> 1.31.1 again,
so each step is the single-minor increment the constraint allows.
johnduhart added a commit to johnduhart/temporal-operator that referenced this pull request Aug 3, 2026
PR alexandrevilain#987 added v1.30.0 to ForbiddenBrokenReleases but missed the other
two retracted releases in the supported range.

Upstream's own go.mod carries the authoritative list:

    retract (
        v1.30.0
        v1.26.1 // Contains retractions only.
        v1.26.0 // Published accidentally.
    )

Confirmed independently: none of the three has a published GitHub
release, while v1.26.2 and v1.30.1 do. Without this, the webhook would
accept a spec.version that has no corresponding container image and the
cluster would sit in ImagePullBackOff.

Also rewords the v1.30.0 comment to cite the retraction rather than the
absent release page, since the retract block is the primary source.
johnduhart added a commit to johnduhart/temporal-operator that referenced this pull request Aug 3, 2026
PR alexandrevilain#987 moves the module to go 1.26.4, which the pinned golangci-lint
v1.64.8 cannot lint: the prebuilt binary is built with go1.24.1 and exits
1 with 'package requires newer Go version go1.26 (application built with
go1.24) (typecheck)'. v1.64.8 is the last v1 release, so there is no v1
version to move to - v2 is the only way forward.

- .github/workflows/tests.yaml: GOLANG_CI_VERSION v1.64.8 -> v2.12.2, and
  golangci-lint-action v6 -> v9 (v6 cannot drive a v2 binary). v2.12.2 is
  built with go1.26.2, the same language version as our module.
- Makefile: same version bump, plus the /v2 module suffix that the v2
  install path requires.
- .golangci.yaml: converted by 'golangci-lint migrate'. v2 merges
  gosimple, stylecheck and typecheck into staticcheck, and moves gofmt
  and goimports into a formatters section.

The migration is deliberately signal-neutral - it should not smuggle in
unrelated refactors or suppressions. Two settings restore the v1 scope:

- staticcheck: exclude QF*. v1 applied 'all' to stylecheck, where it
  meant the ST* checks; under the merged linter 'all' also pulls in the
  QF* quickfix suggestions, which v1 never ran (18 findings, all
  pre-existing).
- goconst: ignore-tests. Table-driven tests repeat short literals by
  nature and v1 did not report them (~69 findings, all pre-existing).

Dropped the stale run.go: "1.22" pin so the language version derives
from go.mod rather than silently holding linters to older semantics.

Genuine findings are fixed rather than suppressed:

- pkg/version/version.go: //nolint:stylecheck -> //nolint:staticcheck.
  The old directive silently stopped matching after the merge, which
  un-suppressed ST1003 on all nine V1_x_x constants.
- govet: disable the inline analyzer. It reports 'cannot inline: type
  parameter inference is not yet supported' on generic calls such as
  slices.Contains - the analyzer describing its own limitation, and it
  only started firing at go 1.26.
- prealloc (4) and goconst (1): preallocate the schema-job slices and the
  e2e feature table, and hoist the repeated 0.0.0.0 bind address to a
  constant.

Verified 0 issues both via the prebuilt binary CI uses and via make lint,
which installs from source.
johnduhart added a commit to johnduhart/temporal-operator that referenced this pull request Aug 3, 2026
PR alexandrevilain#987 declared Timeout as metav1.Duration - a struct - with omitempty.
omitempty has no effect on structs, so the field always serializes, and
the mutating webhook would emit "timeout":"0s" on every cluster that
uses passwordCommand without setting a timeout. That is the same class of
useless webhook patch churn our 8393ba1 removed from 21 other fields.

Making it *metav1.Duration lets omitempty work, and the consumer in
pkg/temporal/persistence/config.go now only sets the server-side Timeout
when the user actually specified one, so the server applies its own
default rather than receiving a hard 0s.

Regenerating required bumping controller-gen v0.16.3 -> v0.21.0: the old
version cannot build under the go 1.26.4 toolchain alexandrevilain#987 introduces
(golang.org/x/tools v0.24.0 fails to compile). The regenerated output is
almost identical, with one substantive and welcome difference:

  cassandra consistency / serialConsistency: type: integer -> type: string

gocql.Consistency is uint16 underneath but implements MarshalText, so it
serializes as a string, and the field already carried string enums
(ANY, ONE, LOCAL_QUORUM, ...). v0.16.3 typed it from the underlying kind
and produced a schema where those enum values could never validate.
v0.21.0 honours the TextMarshaler and emits the correct type. This fixes
a latent bug for Cassandra users; we run postgres12 so we are unaffected.

Chart CRDs regenerated to match; make verify-chart-crds passes.
johnduhart added a commit to johnduhart/temporal-operator that referenced this pull request Aug 3, 2026
Decided on platform-temporal#26: alexandrevilain#987 forces client-go, api and
apimachinery to 0.35.1 because go.temporal.io/server v1.31.1 requires
them, but left controller-runtime at v0.21.0, which targets client-go
0.33.0. Running the layer that drives every reconcile two minors ahead of
its tested client-go is the kind of skew that surfaces as subtle
informer/cache/watch misbehaviour rather than a build error, so we take
the matched pair instead.

v0.23.x is the release paired with client-go 0.35 (v0.24.x pairs with
0.36). The bump also pulled apiextensions-apiserver and component-base
from 0.33.3 to 0.35.0, closing the rest of the skew alexandrevilain#987 left behind.

Two API changes needed handling:

- The webhook builder is now generic: NewWebhookManagedBy takes the object
  and .For() is gone. Migrated to the typed builder rather than the
  deprecated CustomDefaulter/CustomValidator aliases, which let the four
  webhook methods take *v1beta1.TemporalCluster directly and made
  getClusterFromRequest and its five call sites redundant. No coverage is
  lost - no test exercised the wrong-type path it guarded.

- mgr.GetEventRecorderFor is deprecated in favour of GetEventRecorder, but
  the replacement returns events.EventRecorder rather than
  record.EventRecorder. Those are different interfaces, and switching
  moves event emission from the core v1 API group to events.k8s.io/v1 -
  an observable behaviour change that does not belong in a dependency
  bump. Kept with a documented nolint; there is a single Event call site
  (controllers/temporalcluster_controller.go:268) so the migration is
  cheap whenever we choose to do it.

Verified with a cold golangci-lint cache: a warm cache reported both a
false 0-issues result and a false 'unused nolint directive'.
johnduhart and others added 15 commits August 6, 2026 14:46
PR alexandrevilain#987 extended defaultUpgradePath with 1.29.7/1.30.5/1.31.1, which is
correct, but also moved newDatastoreVersion from 1.24.3 to 1.31.1.

Those two variables play opposite roles: newDatastoreVersion is the
version the cluster is CREATED at (persistence_test.go:110,157,262,307),
and defaultUpgradePath is the sequence it is then upgraded THROUGH. With
both at 1.31.1 the first upgrade step asks for 1.25.2, a six-minor
downgrade that ValidateUpdate rejects via UpgradeConstraint.

Restoring 1.24.3 makes the walk 1.24.3 -> 1.25.2 -> ... -> 1.31.1 again,
so each step is the single-minor increment the constraint allows.

(cherry picked from commit 6f6ce64)
PR alexandrevilain#987 added v1.30.0 to ForbiddenBrokenReleases but missed the other
two retracted releases in the supported range.

Upstream's own go.mod carries the authoritative list:

    retract (
        v1.30.0
        v1.26.1 // Contains retractions only.
        v1.26.0 // Published accidentally.
    )

Confirmed independently: none of the three has a published GitHub
release, while v1.26.2 and v1.30.1 do. Without this, the webhook would
accept a spec.version that has no corresponding container image and the
cluster would sit in ImagePullBackOff.

Also rewords the v1.30.0 comment to cite the retraction rather than the
absent release page, since the retract block is the primary source.

(cherry picked from commit a838862)
PR alexandrevilain#987 moves the module to go 1.26.4, which the pinned golangci-lint
v1.64.8 cannot lint: the prebuilt binary is built with go1.24.1 and exits
1 with 'package requires newer Go version go1.26 (application built with
go1.24) (typecheck)'. v1.64.8 is the last v1 release, so there is no v1
version to move to - v2 is the only way forward.

- .github/workflows/tests.yaml: GOLANG_CI_VERSION v1.64.8 -> v2.12.2, and
  golangci-lint-action v6 -> v9 (v6 cannot drive a v2 binary). v2.12.2 is
  built with go1.26.2, the same language version as our module.
- Makefile: same version bump, plus the /v2 module suffix that the v2
  install path requires.
- .golangci.yaml: converted by 'golangci-lint migrate'. v2 merges
  gosimple, stylecheck and typecheck into staticcheck, and moves gofmt
  and goimports into a formatters section.

The migration is deliberately signal-neutral - it should not smuggle in
unrelated refactors or suppressions. Two settings restore the v1 scope:

- staticcheck: exclude QF*. v1 applied 'all' to stylecheck, where it
  meant the ST* checks; under the merged linter 'all' also pulls in the
  QF* quickfix suggestions, which v1 never ran (18 findings, all
  pre-existing).
- goconst: ignore-tests. Table-driven tests repeat short literals by
  nature and v1 did not report them (~69 findings, all pre-existing).

Dropped the stale run.go: "1.22" pin so the language version derives
from go.mod rather than silently holding linters to older semantics.

Genuine findings are fixed rather than suppressed:

- pkg/version/version.go: //nolint:stylecheck -> //nolint:staticcheck.
  The old directive silently stopped matching after the merge, which
  un-suppressed ST1003 on all nine V1_x_x constants.
- govet: disable the inline analyzer. It reports 'cannot inline: type
  parameter inference is not yet supported' on generic calls such as
  slices.Contains - the analyzer describing its own limitation, and it
  only started firing at go 1.26.
- prealloc (4) and goconst (1): preallocate the schema-job slices and the
  e2e feature table, and hoist the repeated 0.0.0.0 bind address to a
  constant.

Verified 0 issues both via the prebuilt binary CI uses and via make lint,
which installs from source.

(cherry picked from commit 7a6f885)
PR alexandrevilain#987 declared Timeout as metav1.Duration - a struct - with omitempty.
omitempty has no effect on structs, so the field always serializes, and
the mutating webhook would emit "timeout":"0s" on every cluster that
uses passwordCommand without setting a timeout. That is the same class of
useless webhook patch churn our 8393ba1 removed from 21 other fields.

Making it *metav1.Duration lets omitempty work, and the consumer in
pkg/temporal/persistence/config.go now only sets the server-side Timeout
when the user actually specified one, so the server applies its own
default rather than receiving a hard 0s.

Regenerating required bumping controller-gen v0.16.3 -> v0.21.0: the old
version cannot build under the go 1.26.4 toolchain alexandrevilain#987 introduces
(golang.org/x/tools v0.24.0 fails to compile). The regenerated output is
almost identical, with one substantive and welcome difference:

  cassandra consistency / serialConsistency: type: integer -> type: string

gocql.Consistency is uint16 underneath but implements MarshalText, so it
serializes as a string, and the field already carried string enums
(ANY, ONE, LOCAL_QUORUM, ...). v0.16.3 typed it from the underlying kind
and produced a schema where those enum values could never validate.
v0.21.0 honours the TextMarshaler and emits the correct type. This fixes
a latent bug for Cassandra users; we run postgres12 so we are unaffected.

Chart CRDs regenerated to match; make verify-chart-crds passes.

(cherry picked from commit 80ad66c)
Decided on platform-temporal#26: alexandrevilain#987 forces client-go, api and
apimachinery to 0.35.1 because go.temporal.io/server v1.31.1 requires
them, but left controller-runtime at v0.21.0, which targets client-go
0.33.0. Running the layer that drives every reconcile two minors ahead of
its tested client-go is the kind of skew that surfaces as subtle
informer/cache/watch misbehaviour rather than a build error, so we take
the matched pair instead.

v0.23.x is the release paired with client-go 0.35 (v0.24.x pairs with
0.36). The bump also pulled apiextensions-apiserver and component-base
from 0.33.3 to 0.35.0, closing the rest of the skew alexandrevilain#987 left behind.

Two API changes needed handling:

- The webhook builder is now generic: NewWebhookManagedBy takes the object
  and .For() is gone. Migrated to the typed builder rather than the
  deprecated CustomDefaulter/CustomValidator aliases, which let the four
  webhook methods take *v1beta1.TemporalCluster directly and made
  getClusterFromRequest and its five call sites redundant. No coverage is
  lost - no test exercised the wrong-type path it guarded.

- mgr.GetEventRecorderFor is deprecated in favour of GetEventRecorder, but
  the replacement returns events.EventRecorder rather than
  record.EventRecorder. Those are different interfaces, and switching
  moves event emission from the core v1 API group to events.k8s.io/v1 -
  an observable behaviour change that does not belong in a dependency
  bump. Kept with a documented nolint; there is a single Event call site
  (controllers/temporalcluster_controller.go:268) so the migration is
  cheap whenever we choose to do it.

Verified with a cold golangci-lint cache: a warm cache reported both a
false 0-issues result and a false 'unused nolint directive'.

(cherry picked from commit d3ee960)
…cripts

Three defects in the temporal-elasticsearch-tool path added for >= 1.30, the
worst of which reports success while doing nothing.

An empty Elasticsearch username rendered a bare "--user". argsMapToString
renders an empty value as a flag with no value, and the tool's parser
(urfave/cli v1 over the stdlib flag package) takes the *following* token as a
string flag's value. For the setup script that token is the "setup-schema"
subcommand itself, so the tool ran no command at all, printed its help and
exited 0 -- the schema job was recorded successful while neither the index
template nor the visibility index was ever created, and the failure only
surfaced later as query errors from the frontend. Verified directly against
go.temporal.io/server v1.31.1's BuildCLIOptions: with a bare --user the
"user" flag comes back as "setup-schema" and no command action fires.

An empty username is what an auth-less Elasticsearch needs and the CRD permits
it (username is required but has no minimum length), so this was reachable.
The same rendering hazard existed for the SQL and Cassandra --user flags; those
fail loudly rather than silently, but they are guarded here too.

"set -eu" in both new templates could skip the shared "scripts" footer. Every
other template deliberately omits set -e so that footer always runs. With ES
visibility plus a linkerd or istio provider, a failing tool invocation exited
immediately and the sidecar was never told to shut down, leaving the Job pod
Running indefinitely instead of failing and retrying -- and persistence
reconciliation blocked on that job forever. The steps are now chained with &&,
which keeps fail-fast behaviour while leaving $? for the footer to propagate.
Chaining is safe because create-index is idempotent: the tool treats
resource_already_exists_exception as success.

The wget shutdown branches appended "|| true" while the curl branches did not.
Since $x is captured before the shutdown call, that could only ever matter
under set -e; with set -e gone it just hid an unreachable proxy, so both
providers now behave identically. Checked the image rather than assuming:
temporalio/admin-tools:1.31.1 ships BusyBox v1.37.0 wget (which supports
--post-data) and no curl at all.

Finally, getStoreTool no longer returns "temporal-elasticsearch-tool"
unconditionally. That binary does not exist in admin-tools <= 1.29, and the
>= 1.30 gate was duplicated in two callers, so any third caller would have
emitted a command that cannot run. The gate now lives in one helper and the
empty sentinel is restored for older versions.

(cherry picked from commit 9cd7d38)
The webhook's rejection message for a broken release suggested IncPatch(), but
broken releases can be consecutive: v1.26.0 and v1.26.1 are both retracted
upstream, as are v1.21.0 and v1.21.1. A user applying 1.26.0 was told to move
to 1.26.1 and was then rejected again, with no hint that 1.26.2 is the real
target. NextNonBrokenPatch skips any candidate that is itself forbidden.

Also memoize the compiled semver constraints behind GreaterOrEqual and
LessThan. Both formatted a constraint string and recompiled it on every call,
and they are called repeatedly within a single reconcile -- once per datastore,
once per deployment builder, once per config section -- nearly always against
the same handful of package-level version constants. The previous code also
discarded the parse error, which would have made Check dereference a nil
*Constraints had the format string ever changed.

(cherry picked from commit 26de637)
…asswordCommand

The existing test asserted the buggy suggestion (1.21.0 -> 1.21.1, itself a
forbidden release); it now asserts 1.21.2.

Also warn when sql.passwordCommand is set. The field works for the server pods,
which resolve the password natively, but the persistence schema jobs run the
same command inside the admin-tools image and that image cannot be extended:
SchemaJobBuilder.Build hardcodes the pod's volumes to the scripts ConfigMap
plus datastore TLS, and exposes only JobInitContainers/JobResources/
JobTTLSecondsAfterFinished -- an init container has no shared writable volume
through which to hand a binary over.

So for the documented use case, an RDS or Cloud SQL IAM token helper, the
command is not found, the substitution yields an empty string, and the first
create-database job fails with a password-authentication error that is hard to
attribute. Users get told this up front instead of discovering it there.
Giving the schema jobs pod-level overrides is a feature in its own right and is
tracked separately.

(cherry picked from commit 727dadc)
The path /etc/temporal/config/config_template.yaml was written out three times
-- as the TEMPORAL_SERVER_CONFIG_FILE_PATH value, as the config volumeMount's
MountPath, and as its SubPath -- plus a fourth time as the ConfigMap key in the
config builder. Changing the mount path or the key in one place would leave the
others pointing at a file that does not exist, and on Temporal >= 1.30 (which
has no fixed built-in location and relies on the env var) the server would exit
at startup with "could not read config file". Nothing links the four at compile
time and no test covers the pairing.

(cherry picked from commit 50db541)
The 1.30 and 1.31 entries each stated a default Temporal/UI version and a
supported range, so the Unreleased section named two different sets of defaults
in consecutive bullets. Only the 1.31 values match temporalcluster_defaults.go;
the 1.30 entry now describes the mechanism change without restating a
superseded default.

Document the CRD schema change that came in with the controller-gen v0.16.3 ->
v0.21.0 bump: cassandra.consistency and cassandra.serialConsistency move from
type: integer to type: string. This is a fix -- the old schema declared
type: integer alongside string enum values, so no value could ever validate,
and gocql.Consistency has always marshalled as text -- but it is a schema
change to pre-existing user-facing fields, and it was bundled inside a tooling
bump in a PR about 1.29/1.30/1.31 support. It belongs in the changelog.

Also record the known limitation of sql.passwordCommand in the schema jobs, and
add a Fixes section for the pod-metadata, broken-release-suggestion and
dynamic-config-integer defects.

Finally, replace the assertion in the goconst comment with the evidence for it.
The claim that v1 did not report these findings is testable: the v1 config
carried no test exclusions at all (only zz_generated), the lint job was green
on upstream main 1398896 with v1.64.8, and the files this setting affects
already existed at that commit. So this restores v1's signal rather than
suppressing it.

(cherry picked from commit 3f02e2f)
The lint job restores a golangci-lint cache whose contents change the verdict.
Two runs of this workflow over a byte-identical main.go disagreed: the run that
populated the cache reported no issues, and the next run, which restored it,
reported both `//nolint:staticcheck // SA1019` directives in main.go as unused.

The cached verdict is the wrong one. Deleting those directives and running cold
shows SA1019 firing at main.go:116 and main.go:136 (`mgr.GetEventRecorderFor`
is deprecated in controller-runtime v0.23), so the directives are used and
necessary. The cache fails in the other direction too: a warm local cache
reported "0 issues" on a tree that genuinely had one, which is the more
dangerous failure since it is silent.

setup-go already runs with cache: false in this job. The cached run still took
130s, so the cache buys roughly nothing while making lint results depend on
which commit last populated it.

(cherry picked from commit 1a44b2f)
The go 1.26.4 bump already forced controller-gen and golangci-lint forward.
The rest of the pins had drifted by years, and one was silently wrong.

  operator-sdk                1.37.0     -> 1.42.3
  yq                          v4.30.6    -> v4.53.3
  kind-with-registry          0.17.0     -> 0.32.0
  helm-docs                   v1.12.0    -> v1.14.2
  go-licenser                 v0.4.0     -> v0.4.2
  gen-crd-api-reference-docs  3f29e685   -> fca9c57b

yq is not cosmetic here: `manifests` drives 18 `yq -i` edits over the
generated CRDs and `helm` drives another over the chart copy, so a
23-minor jump could have rewritten emitted YAML. Verified it does not --
`make generate manifests helm` leaves config/crd/bases and the chart CRDs
byte-identical. The only regenerated change is one line in the API docs,
where the newer gen-crd-api-reference-docs stops leaking the import path
into a map type: `map[string][]./api/v1beta1.ConstrainedValue` becomes
`map[string][]ConstrainedValue`.

ENVTEST_K8S_VERSION was pinned to 1.28.0 by an unconditional assignment
at the top of the file. The derivation lower down reads it from k8s.io/api
but uses `?=`, so it could never take effect -- envtest has been running
1.28 assets against a client-go that is now 0.35.1, seven minors of skew
under every envtest test. Dropping the early assignment lets the
derivation work: ENVTEST_VERSION resolves to release-0.23 (matching
controller-runtime v0.23.3) and ENVTEST_K8S_VERSION to 1.35, which
setup-envtest publishes.

Left alone deliberately: .github/workflows/e2e.yaml pins KIND_VERSION
v0.29.0 alongside node image digests from that release, and kind does not
guarantee node image compatibility across releases -- `kind load` in
particular is called out. Bumping kind means re-pinning the whole matrix,
which is its own change. KIND_WITH_REGISTRY_VERSION only fetches a local
dev helper script and is unrelated to CI.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit 3632871)
…rceable

kustomize v4.5.7 predates the v5 line by three years and still understands the
deprecated `patchesStrategicMerge` and `vars` fields this tree was written
against. v5 drops `vars` outright, so the bump requires migrating both.

Field migrations:

  - `patchesStrategicMerge:` -> `patches:` in config/crd, config/default and
    config/local. config/local uses an inline patch, which becomes
    `patches: - patch: |-`.
  - `vars:` -> `replacements:` in config/default. The four vars fed
    certmanager/certificate.yaml (SERVICE_NAME/SERVICE_NAMESPACE) and
    webhookcainjection_patch.yaml (CERTIFICATE_NAMESPACE/CERTIFICATE_NAME);
    those consumers lose their `$(...)` syntax and carry plain placeholder
    tokens that the replacements overwrite by delimiter+index. Replacements run
    after the namePrefix and namespace transformers, so the sources carry final
    names -- same result the vars produced.

`kustomize edit fix` was used to scope the work but its output was not taken
wholesale: it deletes the entire commented patch block in
config/crd/kustomization.yaml, including both `#+kubebuilder:scaffold:` markers
that future scaffolding depends on, and appends a stray apiVersion/kind to
config/manifests/kustomization.yaml. config/crd was hand-edited to keep the
markers; config/manifests has no deprecated field and is left untouched. The
fixer also does not migrate `vars` at all -- it only reformats the block --
so the replacements above are hand-written in the kubebuilder v4 canonical form.

Verified output-neutral: all four kustomize roots (config/crd, config/default,
config/manifests, config/local) build byte-identical to the v4.5.7 baseline,
with zero deprecation warnings. This matters beyond the deploy path because
`make helm` copies temporal-operator.crds.yaml straight into the chart, so a
kustomize regression would ship to chart users.

Also fix the reason this bump was nearly a silent no-op. The version-checked
tools hung their recipe off `$(LOCALBIN)/<tool>`, a file target: make considers
it up to date whenever the binary exists, so the recipe -- and with it the
version check the recipe exists to perform -- never runs for anyone with a warm
bin/. A v4 kustomize therefore survived the pin bump here and emitted
unsubstituted `CERTIFICATE_NAMESPACE/CERTIFICATE_NAME` into the artifacts. The
recipes now hang off the .PHONY target so the check always runs. Every consumer
already depended on the phony name rather than the path variable, so nothing
else changes. Applied to kustomize, operator-sdk, golangci-lint and
controller-gen, and a check is added to yq, which had none at all despite
`make manifests` and `make helm` running it over the generated CRDs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit 56c339d)
`make lint` failed outright:

    Error: can't load config: the Go language version (go1.25) used to build
    golangci-lint is lower than the targeted Go version (1.26.4)

golangci-lint v2.12.2's own go.mod requires go >= 1.25.0. Go's toolchain
switching honours that requirement literally and builds it with go1.25.12 even
though this module targets 1.26.4, and golangci-lint then refuses to analyse a
module newer than the toolchain it was built with.

GOTOOLCHAIN=local is not the fix here -- the base go binary is 1.24.5 and only
reaches 1.26.4 by switching -- so the toolchain is pinned explicitly to the
version go.mod targets, read from `go list -m`. This tracks automatically on
the next Go bump instead of hardcoding a version that would go stale.

CI is unaffected either way, since golangci-lint-action installs a prebuilt
binary rather than going through `go install`; this only ever bit local runs.

Verified: from a removed bin/golangci-lint and a cleared analysis cache,
`make lint` builds v2.12.2 with go1.26.4 and reports 0 issues.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit 841cddf)
The exponent-form dynamic config entry and the pod-template metadata entry
document fixes that belong to alexandrevilain#988 and alexandrevilain#986 respectively, which are still
open PRs from other contributors. Their fixes -- and the follow-ups that
correct them -- are carried on the fork branch, not here, so this PR must
not claim them.

Also record the kustomize v5 migration and the envtest version fix, which
this PR does carry.
@sonarqubecloud

sonarqubecloud Bot commented Aug 6, 2026

Copy link
Copy Markdown

Azerothian pushed a commit to CrunchyMonkies/temporal-operator that referenced this pull request Aug 6, 2026
PR alexandrevilain#987 extended defaultUpgradePath with 1.29.7/1.30.5/1.31.1, which is
correct, but also moved newDatastoreVersion from 1.24.3 to 1.31.1.

Those two variables play opposite roles: newDatastoreVersion is the
version the cluster is CREATED at (persistence_test.go:110,157,262,307),
and defaultUpgradePath is the sequence it is then upgraded THROUGH. With
both at 1.31.1 the first upgrade step asks for 1.25.2, a six-minor
downgrade that ValidateUpdate rejects via UpgradeConstraint.

Restoring 1.24.3 makes the walk 1.24.3 -> 1.25.2 -> ... -> 1.31.1 again,
so each step is the single-minor increment the constraint allows.
Azerothian pushed a commit to CrunchyMonkies/temporal-operator that referenced this pull request Aug 6, 2026
PR alexandrevilain#987 added v1.30.0 to ForbiddenBrokenReleases but missed the other
two retracted releases in the supported range.

Upstream's own go.mod carries the authoritative list:

    retract (
        v1.30.0
        v1.26.1 // Contains retractions only.
        v1.26.0 // Published accidentally.
    )

Confirmed independently: none of the three has a published GitHub
release, while v1.26.2 and v1.30.1 do. Without this, the webhook would
accept a spec.version that has no corresponding container image and the
cluster would sit in ImagePullBackOff.

Also rewords the v1.30.0 comment to cite the retraction rather than the
absent release page, since the retract block is the primary source.
Azerothian pushed a commit to CrunchyMonkies/temporal-operator that referenced this pull request Aug 6, 2026
PR alexandrevilain#987 moves the module to go 1.26.4, which the pinned golangci-lint
v1.64.8 cannot lint: the prebuilt binary is built with go1.24.1 and exits
1 with 'package requires newer Go version go1.26 (application built with
go1.24) (typecheck)'. v1.64.8 is the last v1 release, so there is no v1
version to move to - v2 is the only way forward.

- .github/workflows/tests.yaml: GOLANG_CI_VERSION v1.64.8 -> v2.12.2, and
  golangci-lint-action v6 -> v9 (v6 cannot drive a v2 binary). v2.12.2 is
  built with go1.26.2, the same language version as our module.
- Makefile: same version bump, plus the /v2 module suffix that the v2
  install path requires.
- .golangci.yaml: converted by 'golangci-lint migrate'. v2 merges
  gosimple, stylecheck and typecheck into staticcheck, and moves gofmt
  and goimports into a formatters section.

The migration is deliberately signal-neutral - it should not smuggle in
unrelated refactors or suppressions. Two settings restore the v1 scope:

- staticcheck: exclude QF*. v1 applied 'all' to stylecheck, where it
  meant the ST* checks; under the merged linter 'all' also pulls in the
  QF* quickfix suggestions, which v1 never ran (18 findings, all
  pre-existing).
- goconst: ignore-tests. Table-driven tests repeat short literals by
  nature and v1 did not report them (~69 findings, all pre-existing).

Dropped the stale run.go: "1.22" pin so the language version derives
from go.mod rather than silently holding linters to older semantics.

Genuine findings are fixed rather than suppressed:

- pkg/version/version.go: //nolint:stylecheck -> //nolint:staticcheck.
  The old directive silently stopped matching after the merge, which
  un-suppressed ST1003 on all nine V1_x_x constants.
- govet: disable the inline analyzer. It reports 'cannot inline: type
  parameter inference is not yet supported' on generic calls such as
  slices.Contains - the analyzer describing its own limitation, and it
  only started firing at go 1.26.
- prealloc (4) and goconst (1): preallocate the schema-job slices and the
  e2e feature table, and hoist the repeated 0.0.0.0 bind address to a
  constant.

Verified 0 issues both via the prebuilt binary CI uses and via make lint,
which installs from source.
Azerothian pushed a commit to CrunchyMonkies/temporal-operator that referenced this pull request Aug 6, 2026
PR alexandrevilain#987 declared Timeout as metav1.Duration - a struct - with omitempty.
omitempty has no effect on structs, so the field always serializes, and
the mutating webhook would emit "timeout":"0s" on every cluster that
uses passwordCommand without setting a timeout. That is the same class of
useless webhook patch churn our 8393ba1 removed from 21 other fields.

Making it *metav1.Duration lets omitempty work, and the consumer in
pkg/temporal/persistence/config.go now only sets the server-side Timeout
when the user actually specified one, so the server applies its own
default rather than receiving a hard 0s.

Regenerating required bumping controller-gen v0.16.3 -> v0.21.0: the old
version cannot build under the go 1.26.4 toolchain alexandrevilain#987 introduces
(golang.org/x/tools v0.24.0 fails to compile). The regenerated output is
almost identical, with one substantive and welcome difference:

  cassandra consistency / serialConsistency: type: integer -> type: string

gocql.Consistency is uint16 underneath but implements MarshalText, so it
serializes as a string, and the field already carried string enums
(ANY, ONE, LOCAL_QUORUM, ...). v0.16.3 typed it from the underlying kind
and produced a schema where those enum values could never validate.
v0.21.0 honours the TextMarshaler and emits the correct type. This fixes
a latent bug for Cassandra users; we run postgres12 so we are unaffected.

Chart CRDs regenerated to match; make verify-chart-crds passes.
Azerothian pushed a commit to CrunchyMonkies/temporal-operator that referenced this pull request Aug 6, 2026
Decided on platform-temporal#26: alexandrevilain#987 forces client-go, api and
apimachinery to 0.35.1 because go.temporal.io/server v1.31.1 requires
them, but left controller-runtime at v0.21.0, which targets client-go
0.33.0. Running the layer that drives every reconcile two minors ahead of
its tested client-go is the kind of skew that surfaces as subtle
informer/cache/watch misbehaviour rather than a build error, so we take
the matched pair instead.

v0.23.x is the release paired with client-go 0.35 (v0.24.x pairs with
0.36). The bump also pulled apiextensions-apiserver and component-base
from 0.33.3 to 0.35.0, closing the rest of the skew alexandrevilain#987 left behind.

Two API changes needed handling:

- The webhook builder is now generic: NewWebhookManagedBy takes the object
  and .For() is gone. Migrated to the typed builder rather than the
  deprecated CustomDefaulter/CustomValidator aliases, which let the four
  webhook methods take *v1beta1.TemporalCluster directly and made
  getClusterFromRequest and its five call sites redundant. No coverage is
  lost - no test exercised the wrong-type path it guarded.

- mgr.GetEventRecorderFor is deprecated in favour of GetEventRecorder, but
  the replacement returns events.EventRecorder rather than
  record.EventRecorder. Those are different interfaces, and switching
  moves event emission from the core v1 API group to events.k8s.io/v1 -
  an observable behaviour change that does not belong in a dependency
  bump. Kept with a documented nolint; there is a single Event call site
  (controllers/temporalcluster_controller.go:268) so the migration is
  cheap whenever we choose to do it.

Verified with a cold golangci-lint cache: a warm cache reported both a
false 0-issues result and a false 'unused nolint directive'.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants