Skip to content

Avoid int overflow in block_map heuristics for large matmuls#375

Open
EylonKrause wants to merge 1 commit into
google:masterfrom
EylonKrause:fix/block-map-int64-overflow
Open

Avoid int overflow in block_map heuristics for large matmuls#375
EylonKrause wants to merge 1 commit into
google:masterfrom
EylonKrause:fix/block-map-int64-overflow

Conversation

@EylonKrause

Copy link
Copy Markdown

Problem

Two heuristics in block_map.cc compute a product of matrix dimensions in 32-bit int, which overflows (signed-overflow UB) for large matmuls and corrupts the traversal-order / block-size decision.

Details

GetTraversalOrder (working_set_size):

const int working_set_size =
    (lhs_scalar_size * rows_after_rectangularness_division +
     rhs_scalar_size * cols_after_rectangularness_division) * depth;

All operands are int. For a square f32 matmul (scalar_size = 4, rows = cols = depth = N) this is 8·N², which reaches INT_MAX + 1 at N = 16384 (8·16384² = 2³¹); int8 overflows at N ≥ 32769, and it also overflows with moderate rows/cols and large depth. On wrap-to-negative, working_set_size > cpu_cache_params.local_cache_size is false, so a multi-GB working set is treated as cache-resident and a linear traversal is chosen instead of the fractal order that exists to accelerate exactly these large, cache-non-resident matmuls.

GetCacheLocalityScore (total_read_bytes) has the same 32-bit product (block_rows/block_cols are capped, but depth is not), which overflows for large depth and feeds a garbage ceil_log2, corrupting the block-size score.

Validate() checks only zero-points, not dimensions, so these int dims are unbounded from ruy::Mul.

Fix

Promote the operands to std::int64_t before multiplying, in both functions. This mirrors the sibling GetTentativeThreadCount in trmul.cc, which already computes the analogous rows·cols·depth product in int64 with the comment:

Be defensive here by explicitly promoting operands to int64 to avoid the pitfall of int64 result = x * y; overflowing as x and y are still narrow.

The subsequent comparisons (against the int cache-size params) and ceil_log2 (templated) then operate safely.

Testing

No unit test: triggering the overflow requires multi-GB operands, impractical for a unit test (the sibling int64 promotion likewise has no dedicated overflow test). The overflow is provable by arithmetic (8·16384² = 2³¹), and the fix is a defensive widening matching the established trmul.cc precedent.

GetTraversalOrder's working_set_size and GetCacheLocalityScore's
total_read_bytes are int products of matrix dimensions ((scalar*rows +
scalar*cols) * depth). For a large matmul these exceed INT_MAX and overflow
(signed-overflow UB): e.g. a square f32 N-cube reaches 2^31 at N=16384. On
wrap-to-negative, GetTraversalOrder treats a multi-GB working set as
cache-resident and picks a linear instead of fractal traversal, and
GetCacheLocalityScore feeds a garbage value to ceil_log2. Validate() does not
bound dimensions, so these come straight from ruy::Mul. Promote the operands
to int64 before multiplying, matching the sibling GetTentativeThreadCount in
trmul.cc which already does this for the rows*cols*depth product.
@EylonKrause

Copy link
Copy Markdown
Author

Disclosure: this contribution was authored with an AI coding assistant (Claude) and reviewed before submission.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant