Avoid int overflow in block_map heuristics for large matmuls#375
Open
EylonKrause wants to merge 1 commit into
Open
Avoid int overflow in block_map heuristics for large matmuls#375EylonKrause wants to merge 1 commit into
EylonKrause wants to merge 1 commit into
Conversation
GetTraversalOrder's working_set_size and GetCacheLocalityScore's total_read_bytes are int products of matrix dimensions ((scalar*rows + scalar*cols) * depth). For a large matmul these exceed INT_MAX and overflow (signed-overflow UB): e.g. a square f32 N-cube reaches 2^31 at N=16384. On wrap-to-negative, GetTraversalOrder treats a multi-GB working set as cache-resident and picks a linear instead of fractal traversal, and GetCacheLocalityScore feeds a garbage value to ceil_log2. Validate() does not bound dimensions, so these come straight from ruy::Mul. Promote the operands to int64 before multiplying, matching the sibling GetTentativeThreadCount in trmul.cc which already does this for the rows*cols*depth product.
Author
|
Disclosure: this contribution was authored with an AI coding assistant (Claude) and reviewed before submission. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Two heuristics in
block_map.cccompute a product of matrix dimensions in 32-bitint, which overflows (signed-overflow UB) for large matmuls and corrupts the traversal-order / block-size decision.Details
GetTraversalOrder(working_set_size):All operands are
int. For a square f32 matmul (scalar_size = 4,rows = cols = depth = N) this is8·N², which reachesINT_MAX + 1atN = 16384(8·16384² = 2³¹); int8 overflows atN ≥ 32769, and it also overflows with moderate rows/cols and large depth. On wrap-to-negative,working_set_size > cpu_cache_params.local_cache_sizeis false, so a multi-GB working set is treated as cache-resident and a linear traversal is chosen instead of the fractal order that exists to accelerate exactly these large, cache-non-resident matmuls.GetCacheLocalityScore(total_read_bytes) has the same 32-bit product (block_rows/block_colsare capped, butdepthis not), which overflows for largedepthand feeds a garbageceil_log2, corrupting the block-size score.Validate()checks only zero-points, not dimensions, so theseintdims are unbounded fromruy::Mul.Fix
Promote the operands to
std::int64_tbefore multiplying, in both functions. This mirrors the siblingGetTentativeThreadCountintrmul.cc, which already computes the analogousrows·cols·depthproduct inint64with the comment:The subsequent comparisons (against the
intcache-size params) andceil_log2(templated) then operate safely.Testing
No unit test: triggering the overflow requires multi-GB operands, impractical for a unit test (the sibling
int64promotion likewise has no dedicated overflow test). The overflow is provable by arithmetic (8·16384² = 2³¹), and the fix is a defensive widening matching the establishedtrmul.ccprecedent.