gdb: Implement stop-on-solib-events for GPU code objects by amd-bfilipov · Pull Request #235 · ROCm/ROCgdb

amd-bfilipov · 2026-07-27T15:42:37Z

stop-on-solib-events support for GPU code objects

GDB's set stop-on-solib-events 1 setting allows users to stop execution when shared libraries are loaded or unloaded, enabling inspection and breakpoint placement before library code executes. This feature works for CPU shared libraries but was not working for GPU code objects loaded by the AMD ROCm runtime.

This PR implements stop-on-solib-events support for GPU code objects, making GPU code object load/unload events behave consistently with CPU shared library events.

Related: AIROCGDB-589

Root Cause

The amd_dbgapi_target_breakpoint::check_status() function unconditionally set bs->stop = 0 and bs->print_it = print_it_noop, regardless of the stop_on_solib_events setting. This is in contrast to internal_breakpoint::check_status() for CPU shared libraries, which respects the setting.

Changes

Commit 1: Implement stop-on-solib-events for GPU code objects

Implementation:

Added code_object_list_updated flag to amd_dbgapi_inferior_info to track when AMD_DBGAPI_EVENT_KIND_CODE_OBJECT_LIST_UPDATED events occur
Modified check_status() to check this flag after processing events and update bs->stop, bs->print, and bs->print_it based on stop_on_solib_events
Set the flag in process_one_event() when CODE_OBJECT_LIST_UPDATED events are seen

Custom message printing:

Implemented print_it() method in amd_dbgapi_target_breakpoint to print a custom message distinguishing GPU code object events from CPU shared library events
Message format follows the same pattern as CPU events but uses "GPU code object event" instead of "shared library event"
Lists loaded/unloaded code objects with file:// or memory:// URIs
Added MI interface support with object-kind field set to "gpu-code-object" to allow IDE frontends to distinguish GPU from CPU solib events

Testing:

Added gdb.rocm/gpu-solib-event.exp test to verify GPU code object events trigger stops
Test uses hipModuleLoad() for file:// events and hipModuleLoadData() for memory:// events
Tests both load and unload events for both URI types

Commit 2: Add GPU/CPU control to stop-on-solib-events setting

Implementation:

Modified stop-on-solib-events to accept integer values that control stopping on CPU shared libraries vs GPU code objects independently:

0 = Do not stop on shared library events
1 = Stop on CPU shared library events only (backward compatible - previous behavior)
2 = Stop on GPU code object events only
3 = Stop on both CPU and GPU library events

This follows GDB's precedent for integer-encoded mode settings (like annotation_level) and preserves backward compatibility - users who previously set stop-on-solib-events to 1 continue to get CPU-only behavior.

User experience improvements:

The show command now displays a human-readable explanation of the current value:

(gdb) show stop-on-solib-events
Stopping for shared library events is 1 (CPU libraries only).

Updated help text documents all four values clearly

Test changes:

Updated gpu-solib-event.exp test to use value 2 (GPU events only)
This eliminates the need to skip CPU shared library events during test execution
CPU solib event testing is already covered by existing GDB testsuite tests

Example Output

With the fix applied:

(gdb) set stop-on-solib-events 2
(gdb) run
Starting program: /path/to/simple
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Stopped due to GPU code object event:
  Inferior loaded file:///path/to/simple.co#offset=4096&size=63200
(gdb) c
Continuing.
Stopped due to GPU code object event:
  Inferior loaded memory://2785254#offset=0x457e30&size=43616
(gdb) c
Continuing.
result is 3
Stopped due to GPU code object event:
  Inferior unloaded file:///path/to/simple.co#offset=4096&size=63200
(gdb) c
Continuing.
Stopped due to GPU code object event:
  Inferior unloaded memory://2785254#offset=0x457e30&size=43616
(gdb) c
Continuing.
[Inferior 1 (process 2785254) exited normally]

Note: With value 2, CPU shared library events are skipped entirely, showing only GPU code object events.

Backward Compatibility

Setting stop-on-solib-events to 1 (or any non-zero value in the old behavior) continues to work for CPU libraries
GPU code object events now require value 2 or 3 to stop
Default value remains 0 (disabled)

Benefits

Consistent debugging experience - GPU code objects behave like CPU shared libraries
Independent control - Users can choose to stop on CPU events only, GPU events only, or both
IDE integration - MI interface object-kind field allows programmatic distinction
Improved test isolation - Tests can focus on GPU events without CPU noise

lumachad · 2026-07-29T09:54:36Z

Does this need a review? If so, we need to flip it to review.

amd-bfilipov · 2026-07-29T13:17:19Z

Does this need a review? If so, we need to flip it to review.

Wanted to check some things first, it's ready for review now.

czidev-amd

IMHO, you need to check bs->print_it assign. Any other comment is non blocking.

lumachad · 2026-07-30T15:58:17Z

@amd-bfilipov When force-pushing, please make a comment clarifying what it is that you changed. Or go through the review comments and clarify for each of those what you've done.

amd-bfilipov · 2026-08-04T13:02:32Z

@amd-bfilipov When force-pushing, please make a comment clarifying what it is that you changed. Or go through the review comments and clarify for each of those what you've done.

@lumachad Understood. Replied to comments.

aktemur · 2026-08-09T13:21:28Z

+	set seen_code_object_loaded 0
+
+	# Continue until we see GPU code object events or the program ends.
+	for {set i 0} {$i < 20} {incr i} {


I don't think we need this loop. See the comment above.

lumachad

This seems reasonable. But I wonder if whoever is chasing GPU bugs and wants to see only GPU solib/code object events would be best server by a switch that only stops for GPU code object events rather than all the non-interesting CPU solib events.

lumachad · 2026-08-12T11:27:57Z

+    current_uiout->text (_("Stopped due to GPU code object event (no "
+			   "code objects added or removed)\n"));


Do you know if/when this happens? Why would we have a code object event where code objects haven't been added or removed?

I don't know if it happens for GPU events but while testing I've seen "no code objects added or removed" events on CPU.

aktemur · 2026-08-12T11:22:36Z

+#define CHECK(cmd)                                                           \
+  {                                                                          \
+    hipError_t error = cmd;                                                  \
+    if (error != hipSuccess)                                                 \
+      {                                                                      \
+	fprintf (stderr, "error: '%s'(%d) at %s:%d\n",                       \
+		 hipGetErrorString (error), error, __FILE__, __LINE__);      \
+	exit (EXIT_FAILURE);                                                 \
+      }                                                                      \
+  }
+
+/* Prevent inlining to ensure kernel stays visible for debugging.  */
+#define NOP(x) asm ("s_nop " #x)


Let's please include rocm-test-utils.h. We then don't need to define CHECK and NOP.

Replaced custom CHECK and NOP macros with #include "rocm-test-utils.h" which provides the same definitions

aktemur · 2026-08-12T11:24:27Z

+    return
+}
+
+proc do_test {} {


IMHO we don't need this proc, but this style is used in some other tests, so I wouldn't require that you remove it.

kept it as it is used in other tests

aktemur · 2026-08-12T11:25:32Z

+
+	for {set i 0} {$i < $max_continues} {incr i} {
+	    gdb_test_multiple "continue" "continue looking for GPU event" {
+		-re "Stopped due to GPU code object event.*Inferior loaded (file://\[^\r\n\]+).*$::gdb_prompt $" {


Please use -wrap and remove the prompt anchor at the end. Also applies to the cases below.

Added -wrap to all gdb_test_multiple patterns and removed explicit $::gdb_prompt $ anchors

aktemur · 2026-08-12T11:28:46Z

+		    # CPU solib event, continue looping.
+		}
+		-re "Inferior.*exited.*$::gdb_prompt $" {
+		    fail "program exited before GPU event"


We don't need this fail. There is no "pass" correspondence. The asserts below will emit FAILs anyway.

aktemur · 2026-08-12T11:29:51Z

+
+	    if {$seen_gpu_solib_event} {
+		break
+	    }


This seems unneeded. We already have a break at the single place where we set the boolean to true.

aktemur · 2026-08-12T11:32:32Z

+	# Verify we saw the GPU code object event.
+	gdb_assert {$seen_gpu_solib_event} "saw GPU code object event"
+
+	# Verify the loaded code object has the file:// prefix.
+	gdb_assert {$seen_file_prefix} "code object has file:// prefix"


These two flags are either both false or both true. There is only one case that sets both. We should either separate the scenarios (is there a scenario that emits a GPU event but no file prefix?) or only one flag would suffice.

Merged seen_gpu_solib_event and seen_file_prefix into single flag

aktemur · 2026-08-12T11:37:40Z

This seems reasonable. But I wonder if whoever is chasing GPU bugs and wants to see only GPU solib/code object events would be best server by a switch that only stops for GPU code object events rather than all the non-interesting CPU solib events.

I'd think that the user gets frustrated by the CPU solib events in the first try, and then they quickly learn that they should rather turn the setting on at a later point after CPU libs are loaded.

I wouldn't mind a separate "stop-on-gpu-solib-events" or a similar setting. It sounds useful.

lumachad

I have a few more automated comments, will post shortly.

lumachad · 2026-08-12T11:48:54Z

+     process_event_queue.  Used to implement stop-on-solib-events for GPU
+     code objects.  */
+  bool code_object_list_updated = false;
+


Nit: the comment does not mention that this flag is reset at the top of check_status before each use. A brief note would help future readers understand the intended lifetime.

lumachad · 2026-08-12T11:48:54Z

+amd_dbgapi_target_breakpoint::print_it (const bpstat *bs) const
+{
+  /* Check if this is a GPU code object event by looking at the solib lists.
+     We only reach here if check_status () set bs->print_it to print_it_normal,


The comment contradicts itself: the first sentence says "Check if this is a GPU code object event" but the rest establishes we already know it is. Suggested rewording:

/* We only reach here when check_status set bs->print_it to print_it_normal, which happens only for GPU code object events when stop_on_solib_events is enabled. */

lumachad · 2026-08-12T11:48:54Z

+  if (current_uiout->is_mi_like_p ())
+    current_uiout->field_string ("reason",
+				 async_reason_lookup (EXEC_ASYNC_SOLIB_EVENT));
+


Both GPU code-object events and CPU shared-library events emit reason="solib-event" over MI. An MI front-end has no way to distinguish them without parsing free-form text. Consider a new async reason or an extra field such as "object-kind", "gpu-code-object" to allow programmatic differentiation.

Added object-kind='gpu-code-object' field to MI output at amd-dbgapi-target.c:953, allowing MI frontends to distinguish GPU from CPU solib events programmatically

lumachad · 2026-08-12T11:48:54Z

+      current_uiout->text (_("  Inferior unloaded "));
+      ui_out_emit_list list_emitter (current_uiout, "removed");
+      for (int ix = 0;
+	   ix < current_program_space->deleted_solibs.size ();


The deleted_solibs block uses an index-based for loop while the added_solibs block below uses a range-based loop. Please pick one style and apply it to both.

Converted deleted_solibs to range-based loop

lumachad · 2026-08-12T11:48:54Z

+#include <hip/hip_runtime.h>
+#include <cstdlib>
+#include <stdio.h>
+


Nit: mixed C and C++ headers — <cstdlib> (C++) alongside <stdio.h> (C). Please use <cstdio> instead of <stdio.h> for consistency, or drop <cstdlib> if EXIT_FAILURE is already provided by the HIP header.

Removed all manual headers since rocm-test-utils.h includes both stdio.h and stdlib.h

lumachad · 2026-08-12T11:48:54Z

+	set seen_gpu_solib_event 0
+	set seen_file_prefix 0
+	set max_continues 20
+


The magic number 20 has no explanation. Please add a comment saying why 20 iterations is a sufficient upper bound to cover all CPU solib events before the first GPU event.

Added comment explaining that the limit of 20 iterations should be sufficient to cover all CPU shared library events that load before the first GPU code object

lumachad · 2026-08-12T11:48:54Z

+
+	for {set i 0} {$i < $max_continues} {incr i} {
+	    gdb_test_multiple "continue" "continue looking for GPU event" {
+		-re "Stopped due to GPU code object event.*Inferior loaded (file://\[^\r\n\]+).*$::gdb_prompt $" {


gdb_test_multiple is missing a catch-all prompt arm. If GDB stops for an unexpected reason the call returns without setting any flag and the loop silently continues. Consider adding:

-re "$::gdb_prompt $" { fail "unexpected stop before GPU code object event" break }

Added catch-all pattern with -wrap and fixed CPU solib pattern to include .* at end to match optional text like '(no libraries added or removed)'

lumachad · 2026-08-12T11:48:54Z

+	for {set i 0} {$i < $max_continues} {incr i} {
+	    gdb_test_multiple "continue" "continue looking for GPU event" {
+		-re "Stopped due to GPU code object event.*Inferior loaded (file://\[^\r\n\]+).*$::gdb_prompt $" {
+		    set seen_gpu_solib_event 1


The test only verifies file:// code object events. The PR description shows both file:// and memory:// events occur. Consider also checking that memory:// events are printed correctly.

Completely rewrote test using hipModule* API to test both file:// (hipModuleLoad) and memory:// (hipModuleLoadData) events, as well as unload events for both

aktemur · 2026-08-12T12:32:33Z

This is a user-visible feature. I think it's worth adding a CHANGELOG entry so that it is mentioned in the release notes.

lancesix · 2026-08-12T13:44:29Z

@@ -0,0 +1,48 @@
+/* This testcase is part of GDB, the GNU debugger.
+
+   Copyright 2022-2026 Free Software Foundation, Inc.


why do we start from 2022?

lancesix · 2026-08-12T13:52:24Z

+	gdb_test_no_output "set stop-on-solib-events 1"
+
+	# Continue through solib events until we hit the GPU code object event.
+	# There may be CPU shared library events before the GPU event.


I am not sure the approach is the best one. For example, if someone runs in an environment where the environment variable HIP_ENABLE_DEFERRED_LOADING=0 all GPU code objects will be loaded before you reach main.

A better way would be to use the hipModule* series of functions to explicitly load a GPU code object at runtime. The testcase could run until the load, step over (or continue over) the load and check that we receive the GPU code object loading. This way, we control everything, which should ensure we have just one event coming.

You can check testcases such as gdb.rocm/code-object-load-while-breakpoint-hip or gdb.rocm/snapshot-objfile-on-load for examples.

A better way would be to use the hipModule* series of functions to explicitly load a GPU code object at runtime. The testcase could run until the load, step over (or continue over) the load and check that we receive the GPU code object loading. This way, we control everything, which should ensure we have just one event coming.

This sounds very good to me.

Completely rewrote test using hipModule* API to test both file:// (hipModuleLoad) and memory:// (hipModuleLoadData) events, as well as unload events for both

GDB's "set stop-on-solib-events 1" setting was not working for GPU code objects loaded by the AMD ROCm runtime. While CPU shared library events correctly triggered stops when this setting was enabled, GPU code object load events were silently ignored. The root cause was in amd_dbgapi_target_breakpoint::check_status(), which unconditionally set bs->stop = 0 and bs->print_it = print_it_noop, regardless of the stop_on_solib_events setting. This is in contrast to internal_breakpoint::check_status() for CPU shared libraries, which respects the setting by checking the stop_on_solib_events global variable. The fix adds: 1. A code_object_list_updated flag to amd_dbgapi_inferior_info to track when AMD_DBGAPI_EVENT_KIND_CODE_OBJECT_LIST_UPDATED events are seen during process_event_queue(). 2. Logic in check_status() to check this flag after processing events, and update bs->stop, bs->print, and bs->print_it based on stop_on_solib_events. 3. A print_it() override to display "Stopped due to GPU code object event" to distinguish GPU events from CPU shared library events. This makes GPU code object load events behave consistently with CPU shared library events, allowing users to stop execution when GPU code objects are loaded for inspection and breakpoint placement. A test is included in gdb.rocm/solib-event.exp.

Modified stop-on-solib-events to accept integer values that control stopping on CPU shared libraries vs GPU code objects independently: 0 = Do not stop on shared library events 1 = Stop on CPU shared library events only 2 = Stop on GPU code object events only 3 = Stop on both CPU and GPU library events This follows GDB's precedent for integer-encoded mode settings (like annotation_level) and preserves backward compatibility - users who previously set stop-on-solib-events to 1 continue to get CPU-only behavior. The show command now displays a human-readable explanation of the current value alongside the numeric setting. Renamed solib-event test to gpu-solib-event to better reflect that it specifically tests GPU code object events. Updated the test to use value 2 (GPU events only), which eliminates the need to skip CPU shared library events during test execution. CPU solib event testing is already covered by existing GDB testsuite tests. Ticket: AIROCGDB-589

amd-bfilipov · 2026-08-17T10:44:07Z

Added independent CPU/GPU control (Commit 2)

Modified stop-on-solib-events to accept integer values for independent control:

0 = disabled (no stops)
1 = CPU libraries only (backward compatible - preserves previous behavior)
2 = GPU code objects only
3 = both CPU and GPU

This follows GDB's precedent for integer-encoded settings (like annotation_level). The show command now displays
human-readable explanations:

(gdb) show stop-on-solib-events
Stopping for shared library events is 2 (GPU code objects only).

Updated the test to use value 2, eliminating the need to filter out CPU shared library events during execution.
CPU solib testing is already covered by existing GDB testsuite tests.

Enhanced stop-on-solib-events to accept both numeric values (0-3) and string keywords ("none", "cpu", "gpu", "all") for improved usability. Implementation uses GDB's extra_literals mechanism, which provides: - Automatic tab completion for string keywords - Display translation (shows "cpu" instead of "1") - Backward compatibility with numeric input Users can now set the value using either format: set stop-on-solib-events 2 # Numeric (existing) set stop-on-solib-events gpu # String keyword (new) Both formats are equivalent and produce the same internal value. The show command displays the literal string when available, making the current setting more readable. This follows GDB's precedent used by settings like "unlimited" which accept both numeric and string input. Ticket: AIROCGDB-589

amd-bfilipov force-pushed the users/bfilipov/fix-solib branch 3 times, most recently from f665745 to 335c12d Compare July 28, 2026 14:55

amd-bfilipov marked this pull request as ready for review July 29, 2026 13:16

amd-bfilipov requested a review from a team as a code owner July 29, 2026 13:16

github-actions Bot assigned lancesix Jul 29, 2026

czidev-amd requested changes Jul 29, 2026

View reviewed changes

Comment thread gdb/testsuite/gdb.rocm/solib-event.exp Outdated

Comment thread gdb/amd-dbgapi-target.c

Comment thread gdb/testsuite/gdb.rocm/solib-event.exp Outdated

Comment thread gdb/testsuite/gdb.rocm/solib-event.exp Outdated

This comment was marked as outdated.

Sign in to view

lumachad requested changes Jul 30, 2026

View reviewed changes

Comment thread gdb/amd-dbgapi-target.c

Comment thread gdb/amd-dbgapi-target.c

Comment thread gdb/amd-dbgapi-target.c

Comment thread gdb/testsuite/gdb.rocm/solib-event.exp Outdated

Comment thread gdb/testsuite/gdb.rocm/solib-event.exp Outdated

amd-bfilipov force-pushed the users/bfilipov/fix-solib branch 2 times, most recently from 9dd1ab3 to 79095ea Compare July 30, 2026 15:50

amd-bfilipov requested a review from lumachad July 30, 2026 15:55

aktemur self-assigned this Aug 4, 2026

aktemur reviewed Aug 9, 2026

View reviewed changes

aktemur removed their assignment Aug 9, 2026

amd-bfilipov force-pushed the users/bfilipov/fix-solib branch 2 times, most recently from 45ca7a7 to ec49403 Compare August 10, 2026 16:18

lumachad reviewed Aug 12, 2026

View reviewed changes

aktemur reviewed Aug 12, 2026

View reviewed changes

lumachad reviewed Aug 12, 2026

View reviewed changes

lancesix reviewed Aug 12, 2026

View reviewed changes

amd-bfilipov force-pushed the users/bfilipov/fix-solib branch from ec49403 to 45d0164 Compare August 17, 2026 09:28

		current_uiout->text (_("Stopped due to GPU code object event (no "
		"code objects added or removed)\n"));

		@@ -0,0 +1,48 @@
		/* This testcase is part of GDB, the GNU debugger.

		Copyright 2022-2026 Free Software Foundation, Inc.

Uh oh!

Conversation

amd-bfilipov commented Jul 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

stop-on-solib-events support for GPU code objects

Root Cause

Changes

Commit 1: Implement stop-on-solib-events for GPU code objects

Commit 2: Add GPU/CPU control to stop-on-solib-events setting

Example Output

Backward Compatibility

Benefits

Uh oh!

lumachad commented Jul 29, 2026

Uh oh!

amd-bfilipov commented Jul 29, 2026

Uh oh!

czidev-amd left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lumachad commented Jul 30, 2026

Uh oh!

amd-bfilipov commented Aug 4, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

lumachad left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aktemur commented Aug 12, 2026

Uh oh!

lumachad left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

amd-bfilipov commented Jul 27, 2026 •

edited

Loading

lumachad left a comment •

edited

Loading