Skip to content

Fix nil deref, swallowed cache read errors and log truncation cap#821

Merged
maoueh merged 5 commits into
developfrom
fix/engine-cache-wasm-bugs
Jul 16, 2026
Merged

Fix nil deref, swallowed cache read errors and log truncation cap#821
maoueh merged 5 commits into
developfrom
fix/engine-cache-wasm-bugs

Conversation

@sduchesneau

@sduchesneau sduchesneau commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Fixes three confirmed bugs in the engine/cache/wasm paths.

Fixes

  1. pipeline/exec/sharedcache.goSharedCache.Execute called inst.Close(ctx) before checking the error from ExecuteNewCall. The wazero runtime returns (nil, err) on instantiation or heap-write failure, so the nil interface dereference panicked in the tier1 shared-cache path and was recovered upstream as a generic "unknown error", masking the real cause. Now only closes a non-nil instance so the real error propagates.

  2. pipeline/cache/engine.goEngine.NewBuffer checked if !ok { continue } before if err != nil. fileReader.Get returns (nil, false, err) on read errors, so corrupt/failed execout cache reads were silently treated as "block absent", producing silent data gaps instead of a failure. The error check now comes first.

  3. wasm/call.goReachedLogsMaxByteCount compared the cumulative LogsByteCount against maxLogByteCount (512 KiB per-message cap) instead of maxTotalLogsByteCount, truncating module logs 10x earlier than the truncation message itself claims. Fixed the comparison to use the total cap — and, since that alone would let logs grow up to the constant's previous value of 5 MiB (10x more output than what was actually ever enforced), lowered maxTotalLogsByteCount itself to 2 MiB so total log volume stays close to today's real-world behavior instead of jumping up.

Tests

  • pipeline/exec/sharedcache_test.go — fake wasm.Module whose ExecuteNewCall returns (nil, err): Execute must return the error without panicking.
  • pipeline/cache/engine_test.go — fake execout.FileReader: a read error must fail NewBuffer; absent blocks are still skipped; found payloads are still loaded into the buffer.
  • wasm/call_logs_test.go — cumulative logs above 512 KiB but below 2 MiB must not be truncated; logs reaching 2 MiB must be truncated with the marker message.

All new tests fail on the unfixed code and pass after the fixes. Full go test ./pipeline/... ./wasm/... passes.

🤖 Generated with Claude Code

@sduchesneau

sduchesneau commented Jul 6, 2026

Copy link
Copy Markdown
Contributor Author

🔍 Vulnerabilities of ghcr.io/streamingfast/substreams:df44e72

📦 Image Reference ghcr.io/streamingfast/substreams:df44e72
digestsha256:a6f9ba37b8d411b6b73fa7b49ad054b6466f1a8b7467f6e2cf724b921d959164
vulnerabilitiescritical: 0 high: 0 medium: 0 low: 0
platformlinux/amd64
size110 MB
packages361
📦 Base Image ubuntu:24.04
also known as
  • noble
  • noble-20260610
digestsha256:52df9b1ee71626e0088f7d400d5c6b5f7bb916f8f0c82b474289a4ece6cf3faf
vulnerabilitiescritical: 0 high: 0 medium: 14 low: 5

ExecuteNewCall returns a nil instance on instantiation failure;
closing it unconditionally panicked and masked the real error.
Get errors were checked after the found flag, so corrupt cache
reads were silently treated as absent blocks.
ReachedLogsMaxByteCount compared against the 512 KiB per-message
cap, truncating logs 10x earlier than documented.
@sduchesneau
sduchesneau force-pushed the fix/engine-cache-wasm-bugs branch from 4d2ed92 to a2ff4f3 Compare July 16, 2026 18:19
Fixing the total-cap comparison bug (previous commit) now lets logs
grow up to the intended 5 MiB, 10x bigger than what was actually
enforced before. Cap at 2 MiB instead to keep output size close to
current behavior.
@sduchesneau
sduchesneau marked this pull request as ready for review July 16, 2026 20:37
@sduchesneau
sduchesneau requested a review from maoueh July 16, 2026 20:37
@maoueh
maoueh merged commit 8e3aabc into develop Jul 16, 2026
6 checks passed
@maoueh
maoueh deleted the fix/engine-cache-wasm-bugs branch July 16, 2026 20:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants