Hi, I've been looking at the opencl/sgemm2 test and found that the compiler by default produces rather bad code. With TILE_SIZE=16 and 32 threads/8 warps, so that each thread in a core maps to an element in a tile, the compiler produces something like below for the central loop in tests/opencl/sgemm2/kernel.cl:
[...]
800001c0: 1407a583 lw a1, 0x140(a5)
800001c4: 04b12623 sw a1, 0x4c(sp)
800001c8: 1807a583 lw a1, 0x180(a5)
800001cc: 04b12423 sw a1, 0x48(sp)
800001d0: 01052583 lw a1, 0x10(a0)
800001d4: 04b12223 sw a1, 0x44(sp)
[...]
That is, the tile is first loaded from global memory and stored onto the stack, but since the stack is in global memory, this is just unnecessary traffic. The compiler produces better results if the loop is instead written as
for (int j = 0; j < TILE_SIZE; j++) {
sum += localA[localRow][j] * localB[j][localCol];
__asm volatile ("" ::: "memory");
}
which disables reordering loads/stores between loop iterations. Now each element in the loop is loaded from memory and directly operated on:
[...]
800001a0: 18072983 lw s3, 0x180(a4)
800001a4: 01fe0e33 add t3, t3, t6
800001a8: 01c52f83 lw t6, 0x1c(a0)
800001ac: 028e8eb3 mul t4, t4, s0
800001b0: 1c072403 lw s0, 0x1c0(a4)
800001b4: 01de0e33 add t3, t3, t4
800001b8: 02052e83 lw t4, 0x20(a0)
800001bc: 03e98f33 mul t5, s3, t5
[...]
The runtime performance of this is rather significant, I get a 4.2x reduction in cycles with simx and a 3.3x reduction with rtlsim:
memory reordering:
simx:
PERF: instrs=11736, cycles=806503, IPC=0.015
rtlsim:
PERF: instrs=11736, cycles=293533, IPC=0.040
__asm volatile ("" ::: "memory") to disable reordering:
simx:
PERF: instrs=8952, cycles=191904, IPC=0.047
rtlsim:
PERF: instrs=8952, cycles=88736, IPC=0.101
(I was also surprised by the large difference between simx and rtlsim, but that's not strictly relevant for this issue)
Unsure if this approach of disabling memory reordering between loop iterations is useful when done programmatically/more generally, but this seems like a lot of performance to be left unused for a relatively simple kernel.
I guess the main culprit here is just that stack accesses are really slow, while the compiler assumes they're relatively fast. Mapping the stack to local memory presumably could be an option, but a couple other issues asking about it seem to have gotten fairly curt replies (#232, #251), so I'm guessing you're not considering it as a valid approach?
Hi, I've been looking at the
opencl/sgemm2test and found that the compiler by default produces rather bad code. WithTILE_SIZE=16and 32 threads/8 warps, so that each thread in a core maps to an element in a tile, the compiler produces something like below for the central loop intests/opencl/sgemm2/kernel.cl:That is, the tile is first loaded from global memory and stored onto the stack, but since the stack is in global memory, this is just unnecessary traffic. The compiler produces better results if the loop is instead written as
which disables reordering loads/stores between loop iterations. Now each element in the loop is loaded from memory and directly operated on:
The runtime performance of this is rather significant, I get a 4.2x reduction in cycles with
simxand a 3.3x reduction withrtlsim:(I was also surprised by the large difference between
simxandrtlsim, but that's not strictly relevant for this issue)Unsure if this approach of disabling memory reordering between loop iterations is useful when done programmatically/more generally, but this seems like a lot of performance to be left unused for a relatively simple kernel.
I guess the main culprit here is just that stack accesses are really slow, while the compiler assumes they're relatively fast. Mapping the stack to local memory presumably could be an option, but a couple other issues asking about it seem to have gotten fairly curt replies (#232, #251), so I'm guessing you're not considering it as a valid approach?