Skip to content

[TOGSim] Do not rescan the blocked head of the ready queue on every issue#302

Merged
YWHyuk merged 1 commit into
developfrom
togsim/issue-scan-cursor
Jul 14, 2026
Merged

[TOGSim] Do not rescan the blocked head of the ready queue on every issue#302
YWHyuk merged 1 commit into
developfrom
togsim/issue-scan-cursor

Conversation

@YWHyuk

@YWHyuk YWHyuk commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator

Follow-up to #301. That PR stopped the 8x8 conv2d from running out of memory, but
test-pytorchsim-wrapper3 / Run test_conv2d.py still fails: the simulation itself
does not finish inside the 3h job limit. This is the missing half.

The problem

Core::cycle's issue scan walks the ready queue from the front every cycle and
issues the first instruction it can. The ones it walks past are blocked -- and only
ever on one of two things: free spad bytes (try_occupy_sram) or a free SA weight
slot (pick_free_weight_sa). Every other path in the scan issues. So the scan
re-walked the same blocked prefix on its way to every single issue.

On the 8x8 config the array is small, so preloads sit waiting on weight slots and
the blocked prefix grows to ~6000. Counted:

case steps walked issues steps per issue
CM16 762,271,822 232,072 3284
MM16 1,070,436,422 302,112 3543

The fix

Give each Tile a cursor at the last instruction the scan found blocked, and let
the next scan resume after it.

The subtle part is when the cursor is still good. Keying it on "was a resource
freed?" is not enough: a freed weight slot that the woken preload immediately takes
leaves the rest of the prefix blocked, yet the cursor would have been thrown away
and the prefix re-walked. So the cursor is keyed on the resource levels
(_sram_used, _weight_free) at the time it was set -- blocked stays blocked for
as long as neither grew. It is dropped only when its own instruction is erased. New
ready instructions are appended, so they always land after the cursor.

case steps walked steps per issue
CM16 257,156 1.11
MM16 402,460 1.33

Result

The 8x8 test_conv2d.py case (batch 2, 128->512, 14x14, k7) goes from 4h36m to
7m19s
.

The scan issues the same instruction on the same cycle -- it just stops re-deriving
what it already knew. Total execution cycles are identical on every case checked:

  • weight-slot arm (8x8, spad throttle off): CM16 803,796 / MM16 803,345 /
    MM36 9,835,778 / CI36 39,449,132
  • spad arm (128x128 with core_spad_size_kb cut to 1024/512 so try_occupy_sram
    actually stalls -- the cycle counts shift, confirming the throttle is live):
    CM16 803,296 / MM16 803,345 / MM36 9,838,863, before and after

🤖 Generated with Claude Code

https://claude.ai/code/session_01QEFKqEkhwSurePkBTAmeTU

…ssue

Core's issue scan walked the ready queue from the front every cycle and issued
the first instruction it could. The ones it walked past are blocked -- and only
ever on one of two things: free spad bytes (try_occupy_sram) or a free SA weight
slot (pick_free_weight_sa). Every other path in the scan issues. So the scan
re-walked the same blocked prefix on its way to every single issue.

Measured on an 8x8 conv (the CI wrapper3 config, where the small array keeps
preloads waiting on weight slots so the prefix grows to ~6000):

  case   steps walked      issues     steps per issue
  CM16   762,271,822       232,072    3284
  MM16   1,070,436,422     302,112    3543

Give each Tile a cursor at the last instruction the scan found blocked and let
the next scan resume after it. Blocked stays blocked until a resource GROWS, so
the cursor is keyed on the resource levels (_sram_used, _weight_free) at the
time, not on a free event: a freed weight slot that the woken preload
immediately takes leaves the rest of the prefix blocked, and the cursor stays
good. It is dropped only when its own instruction is erased. New ready
instructions are appended, so they always land after the cursor.

  case   steps walked      steps per issue
  CM16   257,156           1.11
  MM16   402,460           1.33

The scan issues the same instruction on the same cycle -- it just stops
re-deriving what it already knew. Total execution cycles are identical on every
case checked: CM16 803,796, MM16 803,345, MM36 9,835,778, CI36 39,449,132.

The 8x8 conv2d of tests/ops/conv/test_conv2d.py (CI36) goes from 4h36m to 7m19s.
That job currently fails on develop (SIGKILL, out of memory) and still failed
after the on-demand TileGraph (3h job limit, simulation unfinished).
@YWHyuk
YWHyuk force-pushed the togsim/issue-scan-cursor branch from 6539b6c to 237d3a3 Compare July 14, 2026 02:52
@YWHyuk
YWHyuk merged commit dd45382 into develop Jul 14, 2026
@YWHyuk
YWHyuk deleted the togsim/issue-scan-cursor branch July 14, 2026 05:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant