Skip to content

fix(containerprofilemanager): cap identified call stacks per container#849

Open
matthyx wants to merge 2 commits into
mainfrom
fix/cap-identified-callstacks
Open

fix(containerprofilemanager): cap identified call stacks per container#849
matthyx wants to merge 2 commits into
mainfrom
fix/cap-identified-callstacks

Conversation

@matthyx

@matthyx matthyx commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Each distinct call site produces a distinct (deduped-by-hash) IdentifiedCallStack stored in the per-container callStacks map. A workload that launches many distinct executables (or a process-enumeration/fork-churn attack) grows this map unbounded within a reporting window, driving the node-agent OOM. Observed via heap profiling on a live cluster: Go heap grew to ~900MB under churn (createCallStackFromTrace/ReportIdentifiedCallStack the top allocation), node-agent OOMKilled. Cap at 512 distinct stacks per container; dedup-by-hash preserved below the cap.

🤖 Generated with Claude Code

Each distinct call site produces a distinct (deduped-by-hash) IdentifiedCallStack stored in the per-container callStacks map. A workload that launches many distinct executables (or a process-enumeration/fork-churn attack) grows this map unbounded within a reporting window, driving the node-agent OOM (observed on a live cluster: Go heap grew to ~900MB under churn, with the call-stack build/store path the top allocation). Cap at 512 distinct stacks per container - the profile only needs a representative set.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Docs-exempt: internal memory-safety cap, no API/behavior change
@matthyx matthyx added the ai-assisted Created through Armosec AI tooling (armosec-shared-rules plugin) label Jul 2, 2026
@coderabbitai

coderabbitai Bot commented Jul 2, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@matthyx, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 31 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: d96d02fb-e2b8-41aa-9404-4052755cee15

📥 Commits

Reviewing files that changed from the base of the PR and between 807d238 and 29a033e.

📒 Files selected for processing (1)
  • pkg/objectcache/callstackcache/callstackcache.go
📝 Walkthrough

[!WARNING]

Walkthrough skipped

File diffs could not be summarized.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/cap-identified-callstacks

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

The per-container CallStackSearchTree (built from the profile's identified call stacks) stores one bidirectional tree + trie + path set per distinct CallID. A container that launches many distinct executables produces many distinct call sites, growing the tree unbounded - heap profiling showed buildBidirectionalTree/convertNodesToFrames as the top allocation (~57MB) once the identified-call-stack map was capped. Cap at 512 distinct call sites per container's tree, matching the identified-call-stack cap.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Docs-exempt: internal memory-safety cap, no API/behavior change
@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown

Performance Benchmark Results

Node-Agent Resource Usage
Metric BEFORE AFTER Delta
Avg CPU (cores) 0.219 0.000 -100.0%
Peak CPU (cores) 0.228 0.000 -100.0%
Avg Memory (MiB) 344.267 0.000 -100.0%
Peak Memory (MiB) 346.203 0.000 -100.0%
Dedup Effectiveness

No data available.

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown

Performance Benchmark Results

Node-Agent Resource Usage
Metric BEFORE AFTER Delta
Avg CPU (cores) 0.137 0.127 -7.7%
Peak CPU (cores) 0.142 0.133 -6.4%
Avg Memory (MiB) 322.281 261.650 -18.8%
Peak Memory (MiB) 323.512 264.926 -18.1%
Dedup Effectiveness

No data available.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-assisted Created through Armosec AI tooling (armosec-shared-rules plugin)

Projects

Status: WIP

Development

Successfully merging this pull request may close these issues.

1 participant