Skip to content

hotfix: wire StoreSizeLimit into tier1 linear pipeline#842

Merged
sduchesneau merged 3 commits into
developfrom
hotfix/store-size
Jul 14, 2026
Merged

hotfix: wire StoreSizeLimit into tier1 linear pipeline#842
sduchesneau merged 3 commits into
developfrom
hotfix/store-size

Conversation

@sduchesneau

@sduchesneau sduchesneau commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Problem

Operators set StoreSizeLimit (e.g. 3.5GiB) but tier1's own linear (dev-mode) pipeline still tripped at the 1GiB default:

store "uniswap_v4:store_pools" became too big at 1579123284, maximum size: 1073741824

service/tier1.go hardcoded 0 when building its store ConfigMap, so it always fell back to DefaultStoreSizeLimit. Tier2 subrequests received the limit correctly via ProcessRangeRequest; only tier1's linear path ignored it.

Fix

  • Add StoreSizeLimit to RuntimeConfig (next to StoresScratchSpace/StoresBackend) + WithStoreSizeLimit option, wired from the tier1 app config.
  • tier1 store ConfigMap now passes runtimeConfig.StoreSizeLimit instead of 0.

Test

TestStoreSizeLimit sets a 1-byte limit on the tier1 linear pipeline and asserts ErrStoreAboveMaxSize. Verified it fails if the wiring regresses (limit dropped back to 0).

Additional e2e tests fixes

  • mmap test was failing because the assertion for 'db files exist' was in a race condition with the deletion step. Now polls during the test to validate that files exist.

@sduchesneau

Copy link
Copy Markdown
Contributor Author

🔍 Vulnerabilities of ghcr.io/streamingfast/substreams:28382f1

📦 Image Reference ghcr.io/streamingfast/substreams:28382f1
digestsha256:e56f4cf17da19e27b6c1670eae370c4167f0ac71817d73dda204180af91d31aa
vulnerabilitiescritical: 0 high: 0 medium: 0 low: 0
platformlinux/amd64
size109 MB
packages354
📦 Base Image ubuntu:24.04
also known as
  • noble
  • noble-20260610
digestsha256:52df9b1ee71626e0088f7d400d5c6b5f7bb916f8f0c82b474289a4ece6cf3faf
vulnerabilitiescritical: 0 high: 0 medium: 14 low: 5

@sduchesneau
sduchesneau requested a review from maoueh July 14, 2026 17:10
@sduchesneau
sduchesneau merged commit 17acf91 into develop Jul 14, 2026
6 checks passed
@sduchesneau
sduchesneau deleted the hotfix/store-size branch July 14, 2026 17:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants