Skip to content

Add Attack configuration into Local Dashboard #445

Description

Goal

A visual, drag-and-drop way to assemble an attack (Target, Dataset/Goals, Attack technique + params, optional Guardrails, optional chained fallback steps) from inside the Local Dashboard, run it, and see results in the same dashboard — without dropping to the CLI first.

Current state

The building blocks already exist and are well-factored on the CLI side — this should be a UI over them, not a new execution engine:

  • Attack catalog: cli/commands/attack/catalog.py lists all 14 supported attack_types (AdvPrefix, PAIR, TAP, AutoDAN-Turbo, BoN, CipherChat, h4rm3l, PAP, MML, FC/tFC-Attack, FlipAttack, Static Template, Baseline) with label + description — this is directly usable as the palette of draggable "attack" blocks.
  • Shared config shape: _build_attack_config in cli/commands/attack/config.py builds a plain dict — {"attack_type": ..., "goals": [...] | "dataset": {...}, ...} — that already separates the Dataset/Goals block from the technique block.
  • Target + guardrail blocks: cli/commands/attack/options.py defines the common fields every attack needs: target (agent-name/agent-type/endpoint), and optional before/after guardrails (identifier/agent_type/endpoint each, via _build_guardrail_config).
  • Dataset providers: hackagent/datasets/providers/huggingface.py, url_json.py, file.py, plus presets.py/intents.py — these are the concrete "Dataset" block variants.
  • Chaining already has fallback-ladder semanticshackagent eval chain (cli/commands/attack/chain.py) and HackAgent.hack_chain() (agent.py:327) run an ordered list of attack_configs against a shared goal pool: a goal that succeeds is dropped, a goal that's mitigated escalates to the next attack in the list. This is exactly the semantic a canvas's node connections should express — connecting attack-block A → attack-block B on the canvas is "B is A's fallback," not a generic graph. The board doesn't need to invent new orchestration; it needs to produce the attacks: [...] list hack_chain() already consumes.
  • Progress streaming already exists, just not wired to the dashboard: HackAgent.hack()/hack_chain() take a _tui_event_bus parameter, threaded through attacks/orchestrator.py and every technique's base.py/generation.py, currently consumed only by cli/tui/views/attacks/executor.py. Live progress in the new dashboard panel should plug into this same bus rather than poll.
  • What's missing today: the dashboard (server/dashboard/) is currently read-only — its mixins (_runs_mixin.py, _reports_mixin.py, etc.) only load and render runs/results that already exist in self.backend. There is no code path anywhere under server/dashboard/ that constructs a HackAgent or calls .hack()/.hack_chain(). This issue is the first time the dashboard becomes a place attacks are launched, not just reviewed.

Proposed design

  1. New DashboardAttackBuilderMixin (server/dashboard/_attack_builder_mixin.py), following the existing mixin-per-concern pattern wired into DashboardPage in _page.py, with its own nav entry alongside dashboard/runs/history/reports.
  2. Canvas nodes map 1:1 to existing config sections, not to something new:
    • Target block → agent-name/agent-type/endpoint
    • Dataset/Goals block → goals list or dataset section (provider + params)
    • Attack block(s) → one per ATTACK_CATALOG entry, palette generated from that dict so a new CLI-supported technique automatically appears on the canvas with no dashboard code change
    • Guardrail block(s) → optional before/after, same three fields as the CLI
    • Connecting two attack blocks in sequence → append to an attacks: [...] chain list (fallback-ladder order), matching --config-file's documented chain shape in chain.py
  3. Submit → serialize to the exact same dict shape _build_attack_config/chain already build, then call HackAgent(...).hack(attack_config) or .hack_chain(attacks=[...]) in-process from the NiceGUI server (this is what "local mode" already means for hackagent web — see cli/commands/web.py), on a background task so the event loop isn't blocked, streaming progress into the panel via _tui_event_bus.
  4. Draft persistence: canvas layouts (node positions + the underlying config, before it's run) need to be saved/reopened. Reuse the active StorageBackend (server/storage/local.py for SQLite, server/storage/remote.py for the hosted API — see below) rather than inventing a separate file format, so drafts round-trip the same way runs do.
  5. Results: once a run is submitted this way, it's written through the same StorageBackend the read-only mixins already query — no new results-rendering code needed, the existing Runs/History/Reports panels pick it up automatically.

Open question to resolve before implementation: what does "local vs remote mode" mean here

The issue text says results should save "accordingly" to local or remote mode and still be visible in the local dashboard — but today those two things are coupled differently than that implies:

  • server/storage/ already has both a local.py (SQLite) and remote.py (hosted API) StorageBackend, so a HackAgent instance genuinely can persist either way.
  • But hackagent web itself (cli/commands/web.py) currently treats "remote mode" (API key configured) as "don't run the local NiceGUI dashboard at all — open app.hackagent.dev instead." So there is currently no scenario where the local dashboard is running and writing through the remote backend.

Before building the canvas, decide: (a) should hackagent web gain a mode where it still serves the local UI but a configured API key routes attack runs through the remote StorageBackend (diverging from today's all-or-nothing redirect), or (b) is "remote mode" out of scope for this issue and the builder only ever targets the local SQLite backend, with the cloud dashboard (a separate codebase, hackagent-webapp) getting its own equivalent builder later. This changes where the "run" button's write path goes.

Acceptance criteria

  • New attack-builder panel in the dashboard, palette driven from ATTACK_CATALOG (no hardcoded technique list to keep in sync)
  • Target, Dataset/Goals, Guardrail blocks with the same fields/validation as the CLI options
  • Chaining two-or-more attack blocks produces a hack_chain()-compatible attacks list; single block produces a plain hack() call
  • Submitted runs execute in-process against the active StorageBackend without blocking the NiceGUI event loop, with live progress via _tui_event_bus
  • Draft canvases can be saved and reopened
  • Resulting runs appear in the existing Runs/History/Reports panels with no special-casing
  • Local-vs-remote question above is explicitly decided (not left implicit) and documented in the PR description

Metadata

Metadata

Labels

No labels
No labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions