Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/release-notes-install.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
${IMAGE_REPO}:${VERSION}
```

Multi-arch (`linux/amd64`, `linux/arm64`), distroless nonroot base.
Multi-arch (`linux/amd64`, `linux/arm64`), Debian bookworm-slim nonroot base (includes `git` and `openssh-client` for `spec.git.engine: cli`).

OCI attestations (SBOM + SLSA provenance) are attached in GHCR and on the repository
[Attestations](https://github.com/${GITHUB_REPOSITORY}/attestations) page. Verify the signature:
Expand Down
10 changes: 7 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,13 @@ anchor on an older commit.

### Bug Fixes

- **e2e:** Validate collection via inventory HTTP [cd77870](https://github.com/konih/kollect/commit/cd778701acdd95cb4d2bf74e677499b856bbc702)
- **e2e:** Drop legacy sinkRefs from multitenant scope fixture

- **e2e,test:** Stabilize smoke bootstrap and debounce IT [99d72bb](https://github.com/konih/kollect/commit/99d72bb35ba1cffe4a8c7dbd6777bc5ae37665e2)
- **e2e:** Drop removed inventory sinkRefs field [48af9c4](https://github.com/konih/kollect/commit/48af9c4924c150bea36694dac1f691f3b7a88d82)

- **e2e:** Validate collection via inventory HTTP [ca816a0](https://github.com/konih/kollect/commit/ca816a072b89326d0bc6c6a6dae5f4d2f0ec8159)

- **e2e,test:** Stabilize smoke bootstrap and debounce IT [3677bb5](https://github.com/konih/kollect/commit/3677bb5dc2b1685def8a7aec05f066604e18985c)

- **samples:** Drop legacy sinkRefs from team-a scope [1648ab3](https://github.com/konih/kollect/commit/1648ab3ce0b67e58d8129da909487db91bf3c23b)

Expand All @@ -40,7 +44,7 @@ anchor on an older commit.

### Features

- **git:** Port transport retry and SSH host keys [b8bab57](https://github.com/konih/kollect/commit/b8bab57eba8ed6f554edad951ee761dc8c39a5a4)
- **git:** Port transport retry and SSH host keys [9f5b17f](https://github.com/konih/kollect/commit/9f5b17f40d0eda7b8e482e2c03cb69aac66f370a)

- **controller:** Wire family sink reconcilers and export [8162345](https://github.com/konih/kollect/commit/8162345b399d75ae315d05e06bb25a43c7eb4e0f)

Expand Down
15 changes: 11 additions & 4 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,18 @@ COPY . .
# by leaving it empty we can ensure that the container and binary shipped on it will have the same platform.
RUN CGO_ENABLED=0 GOOS=${TARGETOS:-linux} GOARCH=${TARGETARCH} go build -ldflags="-w -s" -a -o manager cmd/main.go

# Use distroless as minimal base image to package the manager binary
# Refer to https://github.com/GoogleContainerTools/distroless for more details
FROM gcr.io/distroless/static:nonroot@sha256:963fa6c544fe5ce420f1f54fb88b6fb01479f054c8056d0f74cc2c6000df5240
# Runtime image: Debian slim with git + openssh-client for spec.git.engine=cli and git ls-remote probes.
# go-git export (default engine) does not require the git binary; the CLI path and connection probes do.
FROM debian:bookworm-slim@sha256:0104b334637a5f19aa9c983a91b54c89887c0984081f2068983107a6f6c21eeb

RUN apt-get update && \
apt-get install -y --no-install-recommends ca-certificates git openssh-client && \
rm -rf /var/lib/apt/lists/* && \
groupadd --gid 65532 nonroot && \
useradd --uid 65532 --gid 65532 --home-dir /home/nonroot --shell /usr/sbin/nologin --no-create-home nonroot

WORKDIR /
COPY --from=builder /workspace/manager .
COPY --from=builder /workspace/manager /manager
USER 65532:65532

ENTRYPOINT ["/manager"]
2 changes: 1 addition & 1 deletion SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ Risks to consider when deploying:
- Sink endpoints must use verified TLS; credentials must not appear in CR specs or logs.
- Restrict egress with `NetworkPolicy` in production.

See [engineering guidelines](docs/development/guidelines.md) for hardening baselines (distroless image, non-root, secret
See [engineering guidelines](docs/development/guidelines.md) for hardening baselines (non-root runtime image, secret
handling, supply-chain checks in CI).

## Supply chain (releases)
Expand Down
7 changes: 7 additions & 0 deletions charts/kollect/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,13 @@ Installs the [Kollect](https://github.com/konih/kollect) operator and CRDs.
helm install kollect ./charts/kollect -n kollect-system --create-namespace
```

### Container image

The default image is **Debian bookworm-slim** (UID/GID 65532) with `git` and `openssh-client` so
`spec.git.engine: cli` and `git ls-remote` connection probes work out of the box. Default
`spec.git.engine: go-git` does not require those binaries. Pod `securityContext` defaults are
unchanged (`readOnlyRootFilesystem: true`, capabilities dropped, `/tmp` `emptyDir` for git workdirs).

## Values

| Key | Type | Default | Description |
Expand Down
7 changes: 7 additions & 0 deletions charts/kollect/README.md.gotmpl
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,13 @@ Installs the [Kollect](https://github.com/konih/kollect) operator and CRDs.
helm install kollect ./charts/kollect -n kollect-system --create-namespace
```

### Container image

The default image is **Debian bookworm-slim** (UID/GID 65532) with `git` and `openssh-client` so
`spec.git.engine: cli` and `git ls-remote` connection probes work out of the box. Default
`spec.git.engine: go-git` does not require those binaries. Pod `securityContext` defaults are
unchanged (`readOnlyRootFilesystem: true`, capabilities dropped, `/tmp` `emptyDir` for git workdirs).

{{ template "chart.valuesSection" . }}

Export debouncing is configured per **`KollectInventory.spec.exportMinInterval`** (CRD default
Expand Down
30 changes: 25 additions & 5 deletions charts/kollect/templates/validating-webhook-configuration.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,22 @@
{{- if .Values.webhooks.enabled }}
{{- $hooks := list
(dict "name" "vkollectclusterdatabasesink.kb.io" "kind" "kollectclusterdatabasesink" "resource" "kollectclusterdatabasesinks")
(dict "name" "vkollectclustereventsink.kb.io" "kind" "kollectclustereventsink" "resource" "kollectclustereventsinks")
(dict "name" "vkollectclusterinventory.kb.io" "kind" "kollectclusterinventory" "resource" "kollectclusterinventories")
(dict "name" "vkollectclusterprofile.kb.io" "kind" "kollectclusterprofile" "resource" "kollectclusterprofiles")
(dict "name" "vkollectclusterscope.kb.io" "kind" "kollectclusterscope" "resource" "kollectclusterscopes")
(dict "name" "vkollectclustersnapshotsink.kb.io" "kind" "kollectclustersnapshotsink" "resource" "kollectclustersnapshotsinks")
(dict "name" "vkollectclustertarget.kb.io" "kind" "kollectclustertarget" "resource" "kollectclustertargets")
(dict "name" "vkollectconnectiontest.kb.io" "kind" "kollectconnectiontest" "resource" "kollectconnectiontests")
(dict "name" "vkollectdatabasesink.kb.io" "kind" "kollectdatabasesink" "resource" "kollectdatabasesinks")
(dict "name" "vkollecteventsink.kb.io" "kind" "kollecteventsink" "resource" "kollecteventsinks")
(dict "name" "vkollectinventory.kb.io" "kind" "kollectinventory" "resource" "kollectinventories")
(dict "name" "vkollectprofile.kb.io" "kind" "kollectprofile" "resource" "kollectprofiles")
(dict "name" "vkollectremotecluster.kb.io" "kind" "kollectremotecluster" "resource" "kollectremoteclusters")
(dict "name" "vkollectscope.kb.io" "kind" "kollectscope" "resource" "kollectscopes")
(dict "name" "vkollectsnapshotsink.kb.io" "kind" "kollectsnapshotsink" "resource" "kollectsnapshotsinks")
(dict "name" "vkollecttarget.kb.io" "kind" "kollecttarget" "resource" "kollecttargets")
-}}
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
Expand All @@ -10,18 +28,20 @@ metadata:
cert-manager.io/inject-ca-from: {{ .Release.Namespace }}/{{ include "kollect.fullname" . }}-serving-cert
{{- end }}
webhooks:
{{- range $hooks }}
- admissionReviewVersions: [v1]
clientConfig:
service:
name: {{ include "kollect.webhookServiceName" . }}
namespace: {{ .Release.Namespace }}
path: /validate-kollect-dev-v1alpha1-kollectprofile
name: {{ include "kollect.webhookServiceName" $ }}
namespace: {{ $.Release.Namespace }}
path: /validate-kollect-dev-v1alpha1-{{ .kind }}
failurePolicy: Fail
name: vkollectprofile.kb.io
name: {{ .name }}
rules:
- apiGroups: [kollect.dev]
apiVersions: [v1alpha1]
operations: [CREATE, UPDATE]
resources: [kollectprofiles]
resources: [{{ .resource }}]
sideEffects: None
{{- end }}
{{- end }}
40 changes: 40 additions & 0 deletions charts/kollect/tests/validating_webhook_configuration_test.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
suite: validating webhook configuration
templates:
- validating-webhook-configuration.yaml
tests:
- it: renders no VWC when webhooks.enabled is false
set:
webhooks:
enabled: false
asserts:
- hasDocuments:
count: 0
- it: registers all sixteen validating webhooks when enabled
set:
webhooks:
enabled: true
certManager:
create: true
asserts:
- hasDocuments:
count: 1
- isKind:
of: ValidatingWebhookConfiguration
- equal:
path: metadata.annotations["cert-manager.io/inject-ca-from"]
value: NAMESPACE/RELEASE-NAME-kollect-serving-cert
- lengthEqual:
path: webhooks
count: 16
- equal:
path: webhooks[0].name
value: vkollectclusterdatabasesink.kb.io
- equal:
path: webhooks[0].clientConfig.service.path
value: /validate-kollect-dev-v1alpha1-kollectclusterdatabasesink
- equal:
path: webhooks[15].name
value: vkollecttarget.kb.io
- equal:
path: webhooks[15].rules[0].resources[0]
value: kollecttargets
2 changes: 1 addition & 1 deletion docs/ASSURANCE-CASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Security requirements are tracked as NFR-SEC in [REQUIREMENTS.md](REQUIREMENTS.m
| NFR-SEC-2 | Default verify TLS; `insecureSkipVerify` opt-in and surfaced | ADR-0104, sink validators |
| NFR-SEC-3 | Tenancy via `KollectScope` + SAR; least-privilege RBAC | ADR-0203, ADR-0704, `task audit:rbac` |
| NFR-SEC-4 | Sensitive-key redaction before export | ADR-0303, ADR-0405 |
| NFR-SEC-5 | Distroless nonroot image; minimal attack surface | Dockerfile, ADR-0705 |
| NFR-SEC-5 | Nonroot image (UID 65532); minimal runtime deps (`git`/`openssh-client` for git CLI engine) | Dockerfile, ADR-0705 |

## Trust boundaries

Expand Down
13 changes: 13 additions & 0 deletions docs/OPERATOR-MANUAL.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,19 @@ Walkthrough: [Postgres state store](examples/postgres-state-store.md).
TLS trust for sinks uses `caBundle` or `caSecretRef` on the sink spec — same resolution as export
and connection probes ([ADR-0403](adr/0403-connection-test.md)).

### Git snapshot sinks (`spec.git.engine`)

| `spec.git.engine` | Runtime needs | Notes |
| --- | --- | --- |
| `go-git` (default) | None beyond the manager binary | Pure Go transport; works on minimal images |
| `cli` | `git` and `openssh-client` in `PATH` | Native clone/commit/push; SSH uses `GIT_SSH_COMMAND` with `openssh-client` |

The published operator image (`ghcr.io/konih/kollect`) ships **Debian bookworm-slim** with
`git`, `openssh-client`, and `ca-certificates` on UID/GID **65532**. Default `go-git` export is
unchanged; the image change enables `engine: cli` and full `git ls-remote` connection probes.
Custom images built from an older distroless base must install `git` (and `openssh-client` for SSH)
when using `engine: cli`.

### Webhook serving certificate

Validating webhooks require a TLS serving cert mounted on every manager replica
Expand Down
23 changes: 15 additions & 8 deletions docs/PLATFORM-DECISIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ backward compatibility. Breaking changes are batched deliberately before a futur
6. Argo **`Application`** sample + **contract test** (contract test **first**; then samples)
7. Hub `mode: hub|spoke` + merge lib (`inprocess`); no hub CRD
8. **`KollectClusterTarget`** — API + webhook only until namespaced MVP proven; controller later
9. **Secondary watches** — Profile → Targets, Sink → Inventories (beta requirement)
9. **Secondary watches** — Profile → Targets, family Sink → Inventories (beta requirement)
10. **Generic CRD sample** — `cert-manager.io/Certificate` + contract test
11. **GitLab sink** — Phase 2 (custom CA via `tls.caSecretRef`; first enterprise presentation path)
12. **Hub ingest** — SAR **`create`** on `kollectremoteclusters`
Expand All @@ -75,7 +75,7 @@ Sink/transport reframe — [ADR-0401](adr/0401-sink-taxonomy-state-vs-stream.md)

| Topic | Decision |
| --- | --- |
| Secondary watches | **Ship** — `KollectProfile` change enqueues referring Targets; `KollectSink` change enqueues Inventories with `sinkRefs` |
| Secondary watches | **Ship** — `KollectProfile` change enqueues referring Targets; family sink change enqueues Inventories with matching `snapshotSinkRefs`, `databaseSinkRefs`, or `eventSinkRefs` |
| Generic CRD sample | **`cert-manager.io/Certificate`** — contract test first, then profile + target + walkthrough |
| `KollectClusterTarget` controller | **Defer** — API + webhook + sample only until namespaced e2e solid |
| `profileRef` (cluster target) | Resolves **`KollectProfile` in platform namespace** (Helm `platformNamespace`); `KollectClusterProfile` later |
Expand Down Expand Up @@ -122,7 +122,7 @@ the corresponding code merges.
| Inventory read SAR | **`get`** on `kollectinventories` in caller namespace; **`list`** for index | 1 |
| Hub ingest SAR | **`create`** on **`kollectremoteclusters`** in hub namespace | 2 |
| GitLab sink | **`type: gitlab`** backend + custom CA TLS; Phase 2 after Git path proven | 2 |
| Secondary watches | Profile → Targets; Sink → Inventories | 1 (beta) |
| Secondary watches | Profile → Targets; family Sink → Inventories | 1 (beta) |
| TokenReview/SAR cache | **30s TTL** in-memory per `(token hash, verb, resource)`; flag + `disabled` for dev | 1 |
| `maxExportBytes` | Global manager default (~**1.5 MiB**) + optional **`KollectInventory.spec.maxExportBytes`**; webhook rejects override > global cap | 1 |

Expand Down Expand Up @@ -188,7 +188,9 @@ the corresponding code merges.
flowchart TD
subgraph teamNs [Team namespace]
Prof[KollectProfile]
Sink[KollectSink]
Snap[KollectSnapshotSink]
Db[KollectDatabaseSink]
Ev[KollectEventSink]
Scope[KollectScope]
Tgt[KollectTarget]
Inv[KollectInventory]
Expand All @@ -199,15 +201,20 @@ flowchart TD
Scope -.-> Tgt
Scope -.-> Inv
Tgt --> Inv
Inv --> Sink
ConnTest -.-> Sink
ConnTest -.-> Prof
Inv --> Snap
Inv --> Db
Inv --> Ev
ConnTest -.-> Snap
ConnTest -.-> Db
ConnTest -.-> Ev
```

| Kind | Scope | Notes |
| --- | --- | --- |
| `KollectProfile` | Namespace | Same-ns `profileRef` on Target |
| `KollectSink` | **Namespace** | Same-ns `sinkRefs` on Inventory |
| `KollectSnapshotSink` | **Namespace** | Same-ns `snapshotSinkRefs` on Inventory |
| `KollectDatabaseSink` | **Namespace** | Same-ns `databaseSinkRefs` on Inventory |
| `KollectEventSink` | **Namespace** | Same-ns `eventSinkRefs` on Inventory |
| `KollectTarget` | Namespace | Default for `tenantMode`; same-ns `profileRef` |
| `KollectClusterTarget` | **Cluster** | Platform operator; `namespaceSelector` + `KollectClusterProfile` ref |
| `KollectInventory` | Namespace | Aggregates namespaced targets in namespace |
Expand Down
34 changes: 22 additions & 12 deletions docs/QUICKSTART.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,9 +103,10 @@ live API access.
| File | Kind | Role |
| --- | --- | --- |
| `kollect_v1alpha1_kollectprofile.yaml` | `KollectProfile` | Extract Deployment image + labels |
| `kollect_v1alpha1_kollectsink.yaml` | `KollectSink` | Git sink (example repo URL) |
| `kollect_v1alpha1_kollectdatabasesink.yaml` | `KollectDatabaseSink` | Postgres sink (portal SoR) |
| `kollect_v1alpha1_kollectsnapshotsink.yaml` | `KollectSnapshotSink` | Git snapshot sink (audit) |
| `kollect_v1alpha1_kollecttarget.yaml` | `KollectTarget` | Collect Deployments using the profile |
| `kollect_v1alpha1_kollectinventory.yaml` | `KollectInventory` | Aggregate and export to the sink |
| `kollect_v1alpha1_kollectinventory.yaml` | `KollectInventory` | Aggregate and export to family sinks |

Narrative walkthrough with expected behavior notes:
[examples/deployment-inventory.md](examples/deployment-inventory.md).
Expand Down Expand Up @@ -135,28 +136,37 @@ Set `watchMode: OptIn` to collect only explicitly `enabled` namespaces/resources

Sample opt-in target: `config/samples/kollect_v1alpha1_kollecttarget_opt-in.yaml`.

### Connection test (sink)
### Connection test (family sinks)

Samples set `spec.connectionTest: true` on `KollectSink`. The operator probes Git/Postgres/Kafka
and sets **`ConnectionVerified`** ([ADR-0403](adr/0403-connection-test.md)).
Samples set `spec.connectionTest: true` on **`KollectDatabaseSink`** and **`KollectSnapshotSink`**.
The operator probes Postgres/Git (and other wired backends) and sets **`ConnectionVerified`**
([ADR-0403](adr/0403-connection-test.md), [ADR-0414](adr/0414-sink-family-crds.md)).

```sh
kubectl wait --for=condition=ConnectionVerified kollectsink/git-inventory \
kubectl wait --for=condition=ConnectionVerified kollectdatabasesink/postgres-inventory-demo \
-n default --timeout=60s
kubectl describe kollectdatabasesink postgres-inventory-demo -n default
```

Git snapshot sink:

```sh
kubectl wait --for=condition=ConnectionVerified kollectsnapshotsink/git-inventory-demo \
-n default --timeout=60s
kubectl describe kollectsink git-inventory -n default
```

Re-test without editing spec:

```sh
kubectl annotate kollectsink git-inventory -n default kollect.dev/test-connection=true --overwrite
kubectl annotate kollectsnapshotsink git-inventory-demo -n default \
kollect.dev/test-connection=true --overwrite
```

### Optional: Git credentials

The sample Git sink references a placeholder repository. For real exports, create a Secret with
credentials and point `spec.secretRef` at it. Connection tests can pass without a writable remote;
export requires valid credentials and endpoint reachability.
The sample Git snapshot sink references a placeholder repository. For real exports, create a Secret
with credentials and point `spec.secretRef` at it. Connection tests can pass without a writable
remote; export requires valid credentials and endpoint reachability.

## Verify

Expand All @@ -175,7 +185,7 @@ Postgres row counts, etc. — not full payloads; see [ADR-0103](adr/0103-etcd-li
### CR status

```sh
kubectl get kollectprofiles,kollectsinks,kollectinventories
kubectl get kprof,ksnap,kdb,kinv -A
kubectl get kollecttargets -A
kubectl describe kollectinventory -n default team-inventory
```
Expand Down
2 changes: 1 addition & 1 deletion docs/adr/0101-kubebuilder-v4.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ patterns — thin reconcilers, workqueues, conditions, and leader election.
- **Layout:** standard Kubebuilder v4 project structure (`api/v1alpha1`, `internal/{controller,collect,sink}`,
`cmd/main.go`, `config/`, `charts/kollect/`).
- **API version:** `kollect.dev/v1alpha1` (alpha until export/doc flows stabilize).
- **Image:** distroless nonroot; evaluate `ko` for reproducible builds (see PLAN Phase 0 tooling).
- **Image:** Debian bookworm-slim nonroot (UID 65532) with `git`/`openssh-client` for git CLI engine; evaluate `ko` for reproducible builds (see PLAN Phase 0 tooling).

## Consequences

Expand Down
Loading
Loading