Skip to content

Restore RAG details in Local Dashboard - #538

Merged
Nicola Franco (franconicola) merged 5 commits into
mainfrom
fix-rag-dashboard
Aug 5, 2026
Merged

Restore RAG details in Local Dashboard#538
Nicola Franco (franconicola) merged 5 commits into
mainfrom
fix-rag-dashboard

Conversation

@marcorusso97

Copy link
Copy Markdown
Contributor

With some prior PR, the detailed view of the RAG attack on the local dashboard was lost. This PR restores it.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Restores the detailed RAG (and related indirect injection) trace rendering in the local dashboard, and adjusts Textual snapshot tests to avoid path-with-spaces import failures when running snap_compare.

Changes:

  • Route "rag" attack types through the existing indirect prompt injection detail renderer in the dashboard (trace view + history report detail).
  • Update history detail pre-parsing so RAG uses the raw serialized trace list (to be rendered via the indirect injection view).
  • Update TUI snapshot tests to instantiate apps directly instead of relying on pytest-textual-snapshot’s file-path import flow that can break on repo paths containing spaces.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
tests/unit/cli/tui/test_view_snapshots.py Reworks snapshot test loading to bypass import_app/shlex.split path issues by importing the module and creating an app instance directly.
hackagent/server/dashboard/_trace_analysis_mixin.py Restores RAG detail rendering in the attack-specific trace loader by routing "rag" to _render_indirect_injection_view.
hackagent/server/dashboard/_run_history_results_mixin.py Ensures RAG history detail data uses serialized traces so the indirect injection view can render details.
hackagent/server/dashboard/_reports_mixin.py Restores RAG goal-detail rendering in the history/report detail panel via _render_indirect_injection_view.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tests/unit/cli/tui/test_view_snapshots.py
@codecov

codecov Bot commented Jul 28, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 74.57627% with 15 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...ent/server/dashboard/_run_history_results_mixin.py 72.72% 15 Missing ⚠️

📢 Thoughts on this report? Let us know!

Copilot AI review requested due to automatic review settings July 28, 2026 15:30

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

tests/unit/cli/tui/test_view_snapshots.py:45

  • _load_app_instance always creates a second Textual App instance because the snapshot app modules themselves already construct a module-level app = ...App() during exec_module(). Returning a new ...App() here doubles initialization work and can introduce extra side effects/global state in Textual. Consider returning the module’s app when available (and only falling back to instantiating class_name).
    path = _APPS / app_file
    spec = importlib.util.spec_from_file_location(path.stem, path)
    assert spec is not None and spec.loader is not None
    module = importlib.util.module_from_spec(spec)
    spec.loader.exec_module(module)

Copilot AI review requested due to automatic review settings July 28, 2026 15:51

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

tests/unit/cli/tui/test_view_snapshots.py:46

  • _load_app_instance imports the snapshot module (which already constructs a module-level app = ...App()), then instantiates a second App via getattr(module, class_name)(). This creates an unused extra App instance and may introduce side effects/flakiness; it also diverges from how pytest-textual-snapshot historically consumes these files (module-level app). Prefer returning module.app when present, and only fall back to constructing the class if needed.
    spec = importlib.util.spec_from_file_location(path.stem, path)
    assert spec is not None and spec.loader is not None
    module = importlib.util.module_from_spec(spec)
    spec.loader.exec_module(module)
    return getattr(module, class_name)()

@marcorusso97

Copy link
Copy Markdown
Contributor Author

Copilot fix merge conflicts

# Conflicts:
#	hackagent/server/dashboard/_run_history_results_mixin.py
Copilot AI review requested due to automatic review settings August 3, 2026 00:00

Copilot AI commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Copilot fix merge conflicts

Merge conflicts resolved in cb21e8e. Merged origin/main into this branch and combined the crescendo branch (from main) with the rag/indirect_prompt_injection handling (from this PR) in _build_history_goal_detail_data. Dashboard, snapshot, and crescendo tests all pass.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Suppressed comments (3)

tests/unit/cli/tui/test_view_snapshots.py:46

  • _load_app_instance imports the snapshot module which already creates a module-level app = ... instance, but then it instantiates the App class again. This creates an extra unused Textual App per test case and can introduce unintended side effects during import. Reuse the module-level app when it matches the requested class, and only instantiate as a fallback.
    spec = importlib.util.spec_from_file_location(path.stem, path)
    assert spec is not None and spec.loader is not None
    module = importlib.util.module_from_spec(spec)
    spec.loader.exec_module(module)
    return getattr(module, class_name)()

tests/unit/server/dashboard/test_rag_history_dispatch.py:202

  • _build_history_goal_detail_data has a dedicated crescendo dispatch branch, but the "all branches" test suite doesn't cover it. This leaves the extracted dispatch helper partially untested and makes the class docstring inaccurate.
    def test_pair(self):
        page, result = self._build("pair")
        self.assertEqual(result, {"r1": "PAIR_RESULT"})

    def test_tap(self):

tests/unit/server/dashboard/test_rag_history_dispatch.py:132

  • _AllParsersHistoryResultsPage is intended to stub every _parse_*_traces collaborator so the dispatch helper can be fully exercised, but it currently lacks _parse_crescendo_traces. Adding the stub prevents accidental AttributeError if a crescendo test is added (and makes the double match the production interface).

This issue also appears on line 198 of the same file.

    def _parse_pair_traces(self, traces):
        self.calls.append(("pair", traces))
        return "PAIR_RESULT"

    def _parse_tap_traces(self, traces):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@franconicola
Nicola Franco (franconicola) merged commit da46b7c into main Aug 5, 2026
24 checks passed
@franconicola
Nicola Franco (franconicola) deleted the fix-rag-dashboard branch August 5, 2026 10:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants