Skip to content

Allow Infidelity to accept all-negative attributions#376

Open
delaidam wants to merge 1 commit into
understandable-machine-intelligence-lab:mainfrom
delaidam:fix/375-infidelity-negative-attributions-assert
Open

Allow Infidelity to accept all-negative attributions#376
delaidam wants to merge 1 commit into
understandable-machine-intelligence-lab:mainfrom
delaidam:fix/375-infidelity-negative-attributions-assert

Conversation

@delaidam

Copy link
Copy Markdown

Infidelity computes a_batch * (x - x_perturbed) summed over features (see evaluate_batch), a signed dot product with no mathematical requirement for non-negative attributions (see the referenced paper's formula, https://arxiv.org/pdf/1901.09392). The shared assert_attributions() check used by all metrics via Metric.general_preprocess()/explain_batch() rejected all-negative attributions unconditionally, which is overly constraining for Infidelity specifically (e.g. SHAP explanations for a binary classifier, where the sign indicates which class is promoted).

Adds Metric.allow_negative_attributions (default False, preserving current behavior for every other metric) and a check_all_negative parameter on assert_attributions(), set by Infidelity to skip just that one check.

Verified directly (pytest could not run end-to-end in this environment -- tests/conftest.py's autouse cifar10.load_data() fixture requires network access that isn't available here):

  • assert_attributions() rejects all-negative by default, allows it with check_all_negative=False
  • Infidelity.allow_negative_attributions is True, other metrics (checked: FaithfulnessCorrelation) remain False
  • Infidelity(...)(...) runs end-to-end on a tiny synthetic model with an all-negative precomputed a_batch and returns valid scores, where it previously raised AssertionError before evaluation started

Closes #375

Description

  • Insert a clear and concise description of what the PR is about.
  • Optionally, make a reference to an existing issue.
  • Describe the expected improvements of Quantus that results from this PR. Why was this work completed?

Implemented changes

  • Insert a description of the changes implemented in the pull request.
    • Change 1
    • Change 2
    • Change 3

Minimum acceptance criteria

  • Specify what is necessary for the PR to be merged with the main branch.
  • @mentions of the person that is apt to review these changes e.g., @annahedstroem

Infidelity computes a_batch * (x - x_perturbed) summed over features (see
evaluate_batch), a signed dot product with no mathematical requirement
for non-negative attributions (see the referenced paper's formula,
https://arxiv.org/pdf/1901.09392). The shared assert_attributions() check
used by all metrics via Metric.general_preprocess()/explain_batch()
rejected all-negative attributions unconditionally, which is overly
constraining for Infidelity specifically (e.g. SHAP explanations for a
binary classifier, where the sign indicates which class is promoted).

Adds Metric.allow_negative_attributions (default False, preserving current
behavior for every other metric) and a check_all_negative parameter on
assert_attributions(), set by Infidelity to skip just that one check.

Verified directly (pytest could not run end-to-end in this environment --
tests/conftest.py's autouse cifar10.load_data() fixture requires network
access that isn't available here):
- assert_attributions() rejects all-negative by default, allows it with
  check_all_negative=False
- Infidelity.allow_negative_attributions is True, other metrics (checked:
  FaithfulnessCorrelation) remain False
- Infidelity(...)(...) runs end-to-end on a tiny synthetic model with an
  all-negative precomputed a_batch and returns valid scores, where it
  previously raised AssertionError before evaluation started

Closes understandable-machine-intelligence-lab#375
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

infidelity inherits an overly constraining assert for non-negative explanations

1 participant