Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions sycl/source/detail/scheduler/commands.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -2292,10 +2292,12 @@ std::string_view ExecCGCommand::getTypeString() const {
// for users who need more control.
static void adjustNDRangePerKernel(NDRDescT &NDR, ur_kernel_handle_t Kernel,
const device_impl &DeviceImpl) {
if (NDR.GlobalSize[0] != 0)
return; // GlobalSize is set - no need to adjust
// check the prerequisites:
assert(NDR.LocalSize[0] == 0);
if (NDR.NumWorkGroups[0] == 0)
return; // Not parallel_for_work_group -- nothing to fill in.
// In pfwg mode NumWorkGroups is the only field the user sets; GlobalSize

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But non-zero LocalSize was coming from pytorch. This statement does not seem to be true.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per @KseniyaTikhomirova explanation this holds true due to modified if condition in the beginning of the funciton.

// and LocalSize must both be zero (see NDRDescT contract in
// ndrange_desc.hpp).
assert(NDR.GlobalSize[0] == 0 && NDR.LocalSize[0] == 0);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From https://registry.khronos.org/SYCL/specs/sycl-2020/html/sycl-2020.html#_work_group_data_parallel_kernels: "When the global size is zero, the kernel function is not executed, the local size is ignored, and any dependencies are satisfied."

But the modified code doesn't ignore the local size: it checks it. And having a check wrapped in an assertion gives a problem that behavior will be different depending on how SYCL runtime was compiled. We can't assume that release builds will be done with disabled assertions. On the contrary, most if not all Linux distributions build projects with enabled assertions.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe for the case nd_range(0, N) we won't reach this assert. check on line 2295 guarantees that for the all cases except parallel_for_work_group we exit early.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a test and it includes this case, of both being zero.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previously we had assertion firing with NDR.LocalSize[0] != 0 which was coming from pytorch. Per my understanding the new assert will also fire. I will check this later today, but I have concerns if this change really fixes the reported issue.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tried. Yes, I am getting assertion fired with the modified assertion (line number are different as I applied the change to sycl-rel-7_1 branch):

third_party/torch-xpu-ops/test/xpu/extended/test_ops_xpu.py python3: /home/dvrogozh/git/intel-llvm/sycl/source/detail/scheduler/commands.cpp:2323: void sycl::_V1::detail::adjustNDRangePerKernel(NDRDescT&, ur_kernel_handle_t, const device_impl&): Assertion `NDR.GlobalSize[0] == 0 && NDR.LocalSize[0] == 0' failed.
Fatal Python error: Aborted

If assertion can not be narrowed or we don't fully understand why/when execution may reach this place, maybe it would be better to just remove assert entirely till implementation will be clarified?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@KseniyaTikhomirova pointed out that I have overlooked the condition change in the very beginning of this functions right before the assert. After properly cherry-picking the PR it works for the pytorch. Sorry for confusion.

// TODO might be good to cache this info together with the kernel info to
// avoid get_kernel_work_group_info on every kernel run
range<3> WGSize = get_kernel_device_specific_info<
Expand Down
65 changes: 65 additions & 0 deletions sycl/test-e2e/Basic/parallel_for_zero_range.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
// RUN: %{build} -o %t.out
// RUN: %{run} %t.out

// The native_cpu UR adapter explicitly rejects zero global work size in
// urEnqueueKernelLaunch (DIE_NO_IMPLEMENTATION ->
// UR_RESULT_ERROR_UNSUPPORTED_FEATURE).
// XFAIL: target-native_cpu
// XFAIL-TRACKER: CMPLRLLVM-77780

// SYCL 2020 (Work-group data parallel kernels): "When the global size is
// zero, the kernel function is not executed, the local size is ignored, and
// any dependencies are satisfied."
//
// See intel/llvm#22893
//
// A USM-shared sentinel byte is set to 0 before each submit. If a kernel
// body actually ran, it would flip the byte to 0xFF. After Q.wait(), we
// assert the byte is still 0.

#include <sycl/detail/core.hpp>
#include <sycl/usm.hpp>

#include <cassert>

using namespace sycl;

int main() {
queue Q;

unsigned char *Sentinel = malloc_shared<unsigned char>(1, Q);
assert(Sentinel && "USM shared alloc failed");

// Case 1: parallel_for(range<1>{0}) -- plain empty range.
*Sentinel = 0x00;
Q.submit([&](handler &cgh) {
cgh.parallel_for<class zero_range_pf>(range<1>{0},
[=](id<1>) { *Sentinel = 0xFF; });
}).wait();
assert(*Sentinel == 0x00 && "parallel_for(range{0}) unexpectedly launched");

// Case 2: parallel_for(nd_range<1>{{0}, {32}}) -- the PyTorch shape:
// zero global size, non-zero local size. Pre-fix, this tripped the
// assertion in adjustNDRangePerKernel.
*Sentinel = 0x00;
Q.submit([&](handler &cgh) {
cgh.parallel_for<class zero_range_ndr>(
nd_range<1>{range<1>{0}, range<1>{32}},
[=](nd_item<1>) { *Sentinel = 0xFF; });
}).wait();
assert(*Sentinel == 0x00 &&
"parallel_for(nd_range{0, 32}) unexpectedly launched");

// Case 3: parallel_for(nd_range<1>{{0}, {0}})
*Sentinel = 0x00;
Q.submit([&](handler &cgh) {
cgh.parallel_for<class zero_zero_range_ndr>(
nd_range<1>{range<1>{0}, range<1>{0}},
[=](nd_item<1>) { *Sentinel = 0xFF; });
}).wait();
assert(*Sentinel == 0x00 &&
"parallel_for(nd_range{0, 0}) unexpectedly launched");

free(Sentinel, Q);
return 0;
}
Loading