Restore RAG details in Local Dashboard - #538
Conversation
There was a problem hiding this comment.
Pull request overview
Restores the detailed RAG (and related indirect injection) trace rendering in the local dashboard, and adjusts Textual snapshot tests to avoid path-with-spaces import failures when running snap_compare.
Changes:
- Route
"rag"attack types through the existing indirect prompt injection detail renderer in the dashboard (trace view + history report detail). - Update history detail pre-parsing so RAG uses the raw serialized trace list (to be rendered via the indirect injection view).
- Update TUI snapshot tests to instantiate apps directly instead of relying on
pytest-textual-snapshot’s file-path import flow that can break on repo paths containing spaces.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| tests/unit/cli/tui/test_view_snapshots.py | Reworks snapshot test loading to bypass import_app/shlex.split path issues by importing the module and creating an app instance directly. |
| hackagent/server/dashboard/_trace_analysis_mixin.py | Restores RAG detail rendering in the attack-specific trace loader by routing "rag" to _render_indirect_injection_view. |
| hackagent/server/dashboard/_run_history_results_mixin.py | Ensures RAG history detail data uses serialized traces so the indirect injection view can render details. |
| hackagent/server/dashboard/_reports_mixin.py | Restores RAG goal-detail rendering in the history/report detail panel via _render_indirect_injection_view. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (1)
tests/unit/cli/tui/test_view_snapshots.py:45
_load_app_instancealways creates a second TextualAppinstance because the snapshot app modules themselves already construct a module-levelapp = ...App()duringexec_module(). Returning a new...App()here doubles initialization work and can introduce extra side effects/global state in Textual. Consider returning the module’sappwhen available (and only falling back to instantiatingclass_name).
path = _APPS / app_file
spec = importlib.util.spec_from_file_location(path.stem, path)
assert spec is not None and spec.loader is not None
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (1)
tests/unit/cli/tui/test_view_snapshots.py:46
_load_app_instanceimports the snapshot module (which already constructs a module-levelapp = ...App()), then instantiates a secondAppviagetattr(module, class_name)(). This creates an unused extraAppinstance and may introduce side effects/flakiness; it also diverges from howpytest-textual-snapshothistorically consumes these files (module-levelapp). Prefer returningmodule.appwhen present, and only fall back to constructing the class if needed.
spec = importlib.util.spec_from_file_location(path.stem, path)
assert spec is not None and spec.loader is not None
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return getattr(module, class_name)()
|
Copilot fix merge conflicts |
# Conflicts: # hackagent/server/dashboard/_run_history_results_mixin.py
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.
Suppressed comments (3)
tests/unit/cli/tui/test_view_snapshots.py:46
_load_app_instanceimports the snapshot module which already creates a module-levelapp = ...instance, but then it instantiates the App class again. This creates an extra unused Textual App per test case and can introduce unintended side effects during import. Reuse the module-levelappwhen it matches the requested class, and only instantiate as a fallback.
spec = importlib.util.spec_from_file_location(path.stem, path)
assert spec is not None and spec.loader is not None
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return getattr(module, class_name)()
tests/unit/server/dashboard/test_rag_history_dispatch.py:202
_build_history_goal_detail_datahas a dedicatedcrescendodispatch branch, but the "all branches" test suite doesn't cover it. This leaves the extracted dispatch helper partially untested and makes the class docstring inaccurate.
def test_pair(self):
page, result = self._build("pair")
self.assertEqual(result, {"r1": "PAIR_RESULT"})
def test_tap(self):
tests/unit/server/dashboard/test_rag_history_dispatch.py:132
_AllParsersHistoryResultsPageis intended to stub every_parse_*_tracescollaborator so the dispatch helper can be fully exercised, but it currently lacks_parse_crescendo_traces. Adding the stub prevents accidental AttributeError if a crescendo test is added (and makes the double match the production interface).
This issue also appears on line 198 of the same file.
def _parse_pair_traces(self, traces):
self.calls.append(("pair", traces))
return "PAIR_RESULT"
def _parse_tap_traces(self, traces):
With some prior PR, the detailed view of the RAG attack on the local dashboard was lost. This PR restores it.