Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 52 additions & 8 deletions docs/docs/risks/custom-vulnerabilities.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,30 +100,75 @@ print(vuln.get_types()) # [<HIPAAComplianceType.PHI_DISCLOSURE>, ...]
print(vuln.get_values()) # ['phi_disclosure', 'unauthorized_access']
```

## Creating a Threat Profile (Optional)
## Creating a Threat Profile

To provide dataset and attack recommendations, create a threat profile:
A threat profile is optional, but it's what lets an evaluation campaign auto-select datasets, attacks, objective, and metrics for your custom vulnerability instead of you wiring them up by hand each time — see [How Threat Profiles Work](./vulnerabilities.md#how-threat-profiles-work) for the shared `ThreatProfile` anatomy.

```python
from hackagent.risks.profile_types import ThreatProfile
from hackagent.risks.profile_helpers import ds, PRIMARY, STATIC_TEMPLATE_ONLY
from hackagent.risks.profile_helpers import ds, PRIMARY, SECONDARY, STATIC_TEMPLATE_ONLY

HIPAA_COMPLIANCE_PROFILE = ThreatProfile(
vulnerability=HIPAACompliance,
datasets=[
ds(
"custom_hipaa_dataset",
"custom_hipaa_test_set",
PRIMARY,
"Healthcare-specific test cases for PHI protection"
"Healthcare-specific scenarios testing PHI protection"
),
ds(
"donotanswer",
SECONDARY,
"General refusal behavior baseline"
),
],
attacks=STATIC_TEMPLATE_ONLY,
objective="policy_violation",
metrics=["asr", "judge_score"],
metrics=["asr", "judge_score", "phi_leak_count"],
description="Tests HIPAA compliance in healthcare AI systems.",
)

# Use it
print(HIPAA_COMPLIANCE_PROFILE.name) # "HIPAA Compliance"
print(HIPAA_COMPLIANCE_PROFILE.dataset_presets) # ['custom_hipaa_test_set', 'donotanswer']
```

### Profile Helpers

The `profile_helpers` module provides utilities for building profiles:

```python
from hackagent.risks.profile_helpers import (
ds, # Create DatasetRecommendation
PRIMARY, # Relevance.PRIMARY
SECONDARY, # Relevance.SECONDARY
STATIC_TEMPLATE_ONLY, # Static Template-only attack list
JAILBREAK_ATTACKS, # Static Template + PAIR + AdvPrefix (secondary)
ALL_ATTACKS, # Static Template + PAIR + AdvPrefix (all primary)
)

# Create a dataset recommendation
dataset_rec = ds(
"advbench",
PRIMARY,
"Direct harmful behavior test cases"
)

# Use pre-built attack lists
profile = ThreatProfile(
vulnerability=MyVuln,
datasets=[dataset_rec],
attacks=JAILBREAK_ATTACKS,
objective="jailbreak",
metrics=["asr"],
)
```

**Usage:**
- **STATIC_TEMPLATE_ONLY** — Simple direct testing, no adversarial optimization
- **JAILBREAK_ATTACKS** — Includes iterative refinement (PAIR) and gradient-based (AdvPrefix)
- **ALL_ATTACKS** — Full attack suite for comprehensive adversarial testing

## BaseVulnerability Requirements

When extending `BaseVulnerability`, you must provide:
Expand Down Expand Up @@ -364,6 +409,5 @@ If your custom vulnerability addresses a common threat, consider contributing it

## Learn More

- **[Vulnerabilities](./vulnerabilities)** — Study the 13 built-in vulnerability implementations
- **[Threat Profiles](./threat-profiles)** — Learn how to create threat profiles
- **[Vulnerabilities](./vulnerabilities)** — Study the 13 built-in vulnerability implementations and their threat profiles
- **[BaseVulnerability API](../hackagent/agent)** — Full API reference
3 changes: 1 addition & 2 deletions docs/docs/risks/evaluation-campaigns.md
Original file line number Diff line number Diff line change
Expand Up @@ -583,7 +583,6 @@ for profile in rag_profiles:

## Learn More

- **[Vulnerabilities](./vulnerabilities)** — Complete reference for all 13 vulnerability classes
- **[Threat Profiles](./threat-profiles)** — Detailed profile documentation
- **[Vulnerabilities](./vulnerabilities)** — Complete reference for all 13 vulnerability classes, each with its threat profile
- **[Datasets](/datasets)** — Available dataset presets
- **[Attacks](/attacks)** — Attack techniques and configuration
2 changes: 1 addition & 1 deletion docs/docs/risks/evaluation-campaigns/custom-campaigns.md
Original file line number Diff line number Diff line change
Expand Up @@ -467,4 +467,4 @@ def send_slack_alert(campaign_name, results):
- **[Quick Scan](./quick-scan)** - Fast vulnerability scanning
- **[Comprehensive Audit](./comprehensive-audit)** - Full security assessment
- **[Targeted Assessment](./targeted-assessment)** - Focus on specific attack surfaces
- **[Threat Profiles](../threat-profiles)** - Pre-built vulnerability profiles
- **[Vulnerabilities](../vulnerabilities)** - Pre-built vulnerability profiles
7 changes: 3 additions & 4 deletions docs/docs/risks/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ The defined categories are not intended to be mutually exclusive, nor to form an
## What Is Implemented Today

- **Implemented Risk Macro-Category**: Cybersecurity
- **Implemented Risk Micro-Categories**: 13 built-in vulnerabilities, each with a ready-to-run threat profile — see [Vulnerability Categories](./risk-categories.mdx) for the full map and [Threat Profiles](./threat-profiles.md) for per-category datasets/attacks/metrics.
- **Implemented Risk Micro-Categories**: 13 built-in vulnerabilities, each with a ready-to-run threat profile — see [Vulnerabilities](./vulnerabilities.md) for the full reference and per-category datasets/attacks/metrics.
- **Documented Dedicated Scenario**: Indirect Injection (RAG context poisoning)
- **Extensible**: define your own vulnerability categories — see [Custom Vulnerabilities](./custom-vulnerabilities.md)
- Current quick flow: [Evaluation Campaign](./evaluation-campaigns.md)
Expand All @@ -63,14 +63,13 @@ The defined categories are not intended to be mutually exclusive, nor to form an
| **Risk Profile** | Defines the scope of the assessment as one or more macro-categories (or micro-categories). |
| **Evaluation Campaign** | Operationalizes the risk profile into an executable plan: datasets, attacks, objective, and metrics. |

For the currently implemented configuration, see [Vulnerability Categories](./risk-categories.mdx), [Threat Profiles](./threat-profiles.md), and [Indirect Injection](./indirect-prompt-injection.md).
For the currently implemented configuration, see [Vulnerabilities](./vulnerabilities.md) and [Indirect Injection](./indirect-prompt-injection.md).

## Documentation Guide

| Page | Description |
|------|-------------|
| [Vulnerability Categories](./risk-categories.mdx) | The 13 built-in vulnerability classes, organized by attack surface layer |
| [Threat Profiles](./threat-profiles.md) | Per-category recommended datasets, attack techniques, objective, and metrics |
| [Vulnerabilities](./vulnerabilities.md) | Per-vulnerability reference, each with its threat profile — recommended datasets, attack techniques, objective, and metrics |
| [Custom Vulnerabilities](./custom-vulnerabilities.md) | Extend `BaseVulnerability` to define your own categories |
| [Indirect Injection](./indirect-prompt-injection.md) | Dedicated risk scenario for poisoned retrieved context in RAG pipelines |
| [Evaluation Campaigns](./evaluation-campaigns.md) | Quick scans, comprehensive audits, targeted assessments, and custom campaigns |
Expand Down
Loading