feat: add persisted logs

[leafty]

Adds support for persisted logs from Amalthea sessions and image builds.

• Logs are collected from the Loki read API and are stored in the database when attributed to an existing user, session launcher or image build
• Persisted logs for sessions (interactive and jobs) are readable from their owner only
• Persisted logs for image builds are readable by project editors (+repository readers for private image builds)
• Persisted logs are removed once their TTL has expired (set from helm chart values)

Given that persisted logs rely on specific Loki configuration settings, the feature is toggled off by default in the renku helm chart.

/deploy renku=leafty/feat-peristed-logs amalthea-sessions=leafty/feat-run-id-jobs renku-ui=leafty/feat-peristed-logs extra-values=dataService.persistedLogs.enabled=true,dataService.imageBuilders.enabled=true,dataService.imageBuilders.strategyName=renku-buildpacks-v3,dataService.imageBuilders.outputImagePrefix=harbor.dev.renku.ch/renku-build/,dataService.imageBuilders.nodeSelector.renku.io/node-purpose=user,dataService.imageBuilders.tolerations[0].effect=NoSchedule,dataService.imageBuilders.tolerations[0].key=renku.io/dedicated,dataService.imageBuilders.tolerations[0].operator=Equal,dataService.imageBuilders.tolerations[0].value=user

[RenkuBot]

You can access the deployment of this PR at https://renku-ci-ds-1383.dev.renku.ch

[coveralls]

Coverage Report for CI Build 31171640887

Coverage at 85.857% (no base build to compare)

Details

• Coverage remained the same as the base build.
• Patch coverage: 132 uncovered changes across 6 files (567 of 699 lines covered, 81.12%).
• No coverage regressions found.

Uncovered Changes

File	Changed	Covered	%
components/renku_data_services/persisted_logs/collector.py	168	81	48.21%
components/renku_data_services/persisted_logs/db.py	194	167	86.08%
bases/renku_data_services/data_tasks/task_defs.py	14	2	14.29%
components/renku_data_services/persisted_logs/config.py	17	14	82.35%
components/renku_data_services/persisted_logs/models.py	53	51	96.23%
components/renku_data_services/persisted_logs/apispec_base.py	17	16	94.12%
Total (18 files)	699	567	81.12%

Coverage Regressions

No coverage regressions found.

---

Coverage Stats

Relevant Lines:	32291
Covered Lines:	27724
Line Coverage:	85.86%
Coverage Strength:	1.49 hits per line

---

💛 - Coveralls

[olevski]

Thanks Flora. Just a few minor things.

[RenkuBot]

Tearing down the temporary RenkuLab deployment for this PR.