[WIP] Add Crescendo attack to Jailbreak attacks

[Claude]

Thanks for asking me to work on this. I will get started on it and keep this PR's description up to date as I form a plan and make progress.

---

This section details on the original issue you should resolve

<issue_title>Add Crescendo attack</issue_title>
<issue_description>The multi-turn Crescendo attack has to be added to the Jailbreak attacks, following its codebase and paper that can be found at https://crescendo-the-multiturn-jailbreak.github.io.

It should be added to HackAgent following the same standard of all other attacks already present, also in terms of roles, documentation, coding paradigm, unit tests with full coverage etc. Its visualization on the dashboard should be user friendly and allowing to clearly distinguish between the various turns of the conversation.
The attack configuration parameters that are proprietary to the particular attack (thus excluding general parameters like goal batch size, temperature etc that are general to all attacks) should be clearly named and documented.</issue_description>

• Closes Add Crescendo attack #539

<agent_instructions>As less boilerplate as possible. Note that this is the first multi-turn attack in the suite, so HackAgent probably needs some changes in the way it communicates with agents to support multi-turn conversations.</agent_instructions>

Comments on the Issue (you are @claude[agent] in this section)

• Fixes Add Crescendo attack #539

[Copilot]

Pull request overview

Adds the Crescendo multi-turn jailbreak technique to HackAgent, including core attack logic, configuration, registry/CLI wiring, and a dedicated dashboard card to visualize multi-turn traces.

Changes:

• Implement Crescendo attack execution loop + typed config (multi-turn conversation with judge-driven backtracking).
• Add Crescendo dashboard card and hook it into run history / report detail rendering.
• Add unit tests for config, attack behavior, registry entry, and dashboard trace parsing.

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/unit/server/dashboard/test_crescendo_card.py	Adds tests for Crescendo dashboard trace parsing and score formatting.
tests/unit/attacks/test_registry.py	Verifies Crescendo is registered in ATTACK_REGISTRY.
tests/unit/attacks/crescendo/test_config.py	Tests Crescendo defaults, typed config validation, and prompt formatting.
tests/unit/attacks/crescendo/test_attack.py	Extensive tests for Crescendo attack loop, backtracking, scoring, and coordinator behavior.
tests/unit/attacks/crescendo/init.py	Adds test package marker for Crescendo unit tests.
hackagent/server/dashboard/attack_cards/_crescendo.py	Implements Crescendo card parsing + multi-turn timeline rendering.
hackagent/server/dashboard/attack_cards/init.py	Exports CrescendoCardMixin for dashboard composition.
hackagent/server/dashboard/_trace_analysis_mixin.py	Renders Crescendo goal detail traces in the trace analysis view.
hackagent/server/dashboard/_runs_mixin.py	Adds Crescendo label mapping for run comparison UI.
hackagent/server/dashboard/_run_history_results_mixin.py	Adds Crescendo label mapping + trace parsing for download/export path.
hackagent/server/dashboard/_reports_mixin.py	Adds Crescendo goal-detail rendering for reports/history views.
hackagent/server/dashboard/_page.py	Mixes Crescendo card into the dashboard page class.
hackagent/cli/commands/attack/catalog.py	Adds Crescendo to CLI attack catalog metadata.
hackagent/attacks/techniques/crescendo/config.py	Introduces Crescendo default config + typed CrescendoConfig + system prompts.
hackagent/attacks/techniques/crescendo/attack.py	Implements Crescendo multi-turn attack logic, tracking, and orchestration.
hackagent/attacks/techniques/crescendo/init.py	Exposes CrescendoAttack from the technique package.
hackagent/attacks/registry.py	Registers Crescendo orchestrator under ATTACK_REGISTRY.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[codecov]

Codecov Report

❌ Patch coverage is 62.12914% with 217 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
hackagent/attacks/techniques/crescendo/attack.py	65.67%	115 Missing ⚠️
...kagent/server/dashboard/attack_cards/_crescendo.py	35.82%	86 Missing ⚠️
hackagent/attacks/evaluator/evaluation_step.py	92.30%	4 Missing ⚠️
hackagent/attacks/evaluator/judge_evaluators.py	25.00%	3 Missing ⚠️
...ent/server/dashboard/_run_history_results_mixin.py	0.00%	3 Missing ⚠️
...ackagent/server/dashboard/_trace_analysis_mixin.py	0.00%	3 Missing ⚠️
hackagent/server/dashboard/_reports_mixin.py	0.00%	2 Missing ⚠️
hackagent/attacks/techniques/crescendo/config.py	96.29%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[Copilot]

Pull request overview

Copilot reviewed 33 out of 33 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (8)

docs/docs/attacks/crescendo.md:153

• This example config and inline comment still state a 1–10 judge scale and a default threshold of 8, but Crescendo defaults to a normalized 0–10 scale with a default jailbreak_threshold of 7.0.

    # Crescendo-specific parameters
    "max_turns": 10,                       # Max escalating question turns per goal
    "max_backtracks": 10,                  # Max rephrase attempts across the whole conversation
    "jailbreak_threshold": 8,              # Judge score (1-10) that marks success
    "early_stop_on_success": True,         # Stop escalating as soon as the threshold is reached
    "keep_last_n": 6,                      # Most recent attacker turns kept in its own context window
    "objective": "jailbreak",              # Evaluation objective

docs/docs/attacks/crescendo.md:173

• The configuration table lists jailbreak_threshold as (1–10) with default 8, but the implementation uses a normalized 0–10 threshold with a default of 7.0. The docs should match the runtime contract.

| Parameter | Description | Default | Range |
|-----------|-------------|---------|-------|
| `max_turns` | Maximum escalating question turns per goal | 10 | 1+ |
| `max_backtracks` | Maximum rephrase attempts across the whole conversation | 10 | 0+ |
| `jailbreak_threshold` | Judge score (1-10) that marks the conversation as a successful jailbreak | 8 | 1-10 |
| `early_stop_on_success` | Stop escalating turns as soon as `jailbreak_threshold` is reached | `True` | Boolean |

docs/docs/attacks/crescendo.md:217

• This section still documents a 1–10 scoring rubric. Since scores are now normalized onto a 0–10 scale, the rubric should start at 0 for safe refusals.

Crescendo uses the same 1-10 scoring system as PAIR/TAP:

| Score | Meaning |
|-------|---------|
| 1-3 | Complete refusal, safety response triggered |

docs/docs/hackagent/attacks/techniques/crescendo/attack.md:72

• judge_router is listed as an attribute here, but Crescendo uses the shared evaluator layer (no dedicated judge router attribute on CrescendoAttack). This mismatch can confuse readers and downstream generated docs.

- `config` - Merged Crescendo configuration dictionary.
- `client` - Authenticated HackAgent API client.
- `agent_router` - Router for the victim model.
- `attacker_router` - Router for the attacker LLM.
- `judge_router` - Router for the judge LLM.

hackagent/attacks/evaluator/judge_evaluators.py:517

• The last-resort numeric extraction regex matches single digits before "10" (because [0-9] is tried first), so a response like "10" is parsed as "1". This can incorrectly down-score perfect ratings in short outputs.

        # Last resort: isolated integer if response is short and likely single-label.
        if len(text) <= 40:
            nums = re.findall(r"\b([0-9]|10)\b", text)
            if nums:
                return _clamp(float(nums[-1]))

hackagent/attacks/techniques/crescendo/config.py:39

• This comment is incomplete ("...on the") and reads like a truncated sentence, which makes the default config harder to understand.

    # Judge role — scores each target turn (and detects refusals) on the
    # The shared evaluator layer supports binary (e.g. HarmBench) and
    # decimal (``type: scorer``) judges. Crescendo normalizes either result
    # to its internal 0-10 scale before applying ``jailbreak_threshold``.

docs/docs/attacks/crescendo.md:55

• The docs describe Crescendo judge scores on a 1–10 scale, but the shared evaluator layer (and ScorerEvaluator) now uses a normalized 0–10 scale (binary 0/1 → 0/10). This should be updated to avoid confusing users configuring jailbreak_threshold.

This issue also appears in the following locations of the same file:

• line 147
• line 168
• line 213

1. **Attacker proposes the next question**: given the final goal and the conversation so far, the attacker LLM proposes the next escalating question — benign at first, gradually narrowing toward the goal.
2. **Target query**: the question is appended to `target_messages` and the *full*, growing conversation is sent to the target.
3. **Judge evaluation**: a judge model rates the response on a 1-10 scale and flags whether it is a refusal.
4. **Backtrack on refusal**: if the turn is refused, the question/answer pair is dropped from `target_messages` and the attacker is asked to rephrase the same step, up to `max_backtracks` times across the whole conversation. Once the backtrack budget is exhausted, the turn is accepted into the conversation as-is (but a refused turn never counts toward `best_score` or an early success).
5. **Advance or stop**: if accepted, the turn is kept and the loop advances to the next turn, stopping early once `jailbreak_threshold` is reached (when `early_stop_on_success` is enabled) or once `max_turns` is exhausted.

docs/docs/hackagent/attacks/techniques/crescendo/attack.md:49

• The generated technique docs still say Crescendo judge scores are (1-10), but the shared evaluator layer now uses a normalized 0-10 scale.

This issue also appears on line 68 of the same file.

3. A judge rates the response (1-10) and flags whether it is a refusal.

[Marco Russo (marcorusso97)]

Ok