[TOGSim] Do not rescan the blocked head of the ready queue on every issue#302
Merged
Conversation
…ssue Core's issue scan walked the ready queue from the front every cycle and issued the first instruction it could. The ones it walked past are blocked -- and only ever on one of two things: free spad bytes (try_occupy_sram) or a free SA weight slot (pick_free_weight_sa). Every other path in the scan issues. So the scan re-walked the same blocked prefix on its way to every single issue. Measured on an 8x8 conv (the CI wrapper3 config, where the small array keeps preloads waiting on weight slots so the prefix grows to ~6000): case steps walked issues steps per issue CM16 762,271,822 232,072 3284 MM16 1,070,436,422 302,112 3543 Give each Tile a cursor at the last instruction the scan found blocked and let the next scan resume after it. Blocked stays blocked until a resource GROWS, so the cursor is keyed on the resource levels (_sram_used, _weight_free) at the time, not on a free event: a freed weight slot that the woken preload immediately takes leaves the rest of the prefix blocked, and the cursor stays good. It is dropped only when its own instruction is erased. New ready instructions are appended, so they always land after the cursor. case steps walked steps per issue CM16 257,156 1.11 MM16 402,460 1.33 The scan issues the same instruction on the same cycle -- it just stops re-deriving what it already knew. Total execution cycles are identical on every case checked: CM16 803,796, MM16 803,345, MM36 9,835,778, CI36 39,449,132. The 8x8 conv2d of tests/ops/conv/test_conv2d.py (CI36) goes from 4h36m to 7m19s. That job currently fails on develop (SIGKILL, out of memory) and still failed after the on-demand TileGraph (3h job limit, simulation unfinished).
YWHyuk
force-pushed
the
togsim/issue-scan-cursor
branch
from
July 14, 2026 02:52
6539b6c to
237d3a3
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #301. That PR stopped the 8x8 conv2d from running out of memory, but
test-pytorchsim-wrapper3 / Run test_conv2d.pystill fails: the simulation itselfdoes not finish inside the 3h job limit. This is the missing half.
The problem
Core::cycle's issue scan walks the ready queue from the front every cycle andissues the first instruction it can. The ones it walks past are blocked -- and only
ever on one of two things: free spad bytes (
try_occupy_sram) or a free SA weightslot (
pick_free_weight_sa). Every other path in the scan issues. So the scanre-walked the same blocked prefix on its way to every single issue.
On the 8x8 config the array is small, so preloads sit waiting on weight slots and
the blocked prefix grows to ~6000. Counted:
The fix
Give each
Tilea cursor at the last instruction the scan found blocked, and letthe next scan resume after it.
The subtle part is when the cursor is still good. Keying it on "was a resource
freed?" is not enough: a freed weight slot that the woken preload immediately takes
leaves the rest of the prefix blocked, yet the cursor would have been thrown away
and the prefix re-walked. So the cursor is keyed on the resource levels
(
_sram_used,_weight_free) at the time it was set -- blocked stays blocked foras long as neither grew. It is dropped only when its own instruction is erased. New
ready instructions are appended, so they always land after the cursor.
Result
The 8x8
test_conv2d.pycase (batch 2, 128->512, 14x14, k7) goes from 4h36m to7m19s.
The scan issues the same instruction on the same cycle -- it just stops re-deriving
what it already knew. Total execution cycles are identical on every case checked:
MM36 9,835,778 / CI36 39,449,132
core_spad_size_kbcut to 1024/512 sotry_occupy_sramactually stalls -- the cycle counts shift, confirming the throttle is live):
CM16 803,296 / MM16 803,345 / MM36 9,838,863, before and after
🤖 Generated with Claude Code
https://claude.ai/code/session_01QEFKqEkhwSurePkBTAmeTU