Skip to content

Use per-dimension mean (axis=0) in mahalanobis metric - #786

Open
uttam12331 wants to merge 1 commit into
huggingface:mainfrom
uttam12331:fix-mahalanobis-mean-axis
Open

Use per-dimension mean (axis=0) in mahalanobis metric#786
uttam12331 wants to merge 1 commit into
huggingface:mainfrom
uttam12331:fix-mahalanobis-mean-axis

Conversation

@uttam12331

Copy link
Copy Markdown

Summary

The Mahalanobis distance centers X on the mean vector (per-column centroid) of the reference distribution, but _compute uses np.mean(reference_distribution) with no axis, which collapses the 2-D reference array to a single scalar grand mean and broadcasts it across every feature:

X_minus_mu = X - np.mean(reference_distribution)      # scalar grand mean
cov = np.cov(reference_distribution.T)                # per-column covariance

The very next line, np.cov(reference_distribution.T), treats each column as a variable — confirming the per-dimension intent. As a result the metric is wrong for any distribution whose feature means differ.

Reproduction

import evaluate
m = evaluate.load("mahalanobis")
m.compute(reference_distribution=[[0, 10], [4, 10], [0, 14], [4, 14]], X=[[2, 12]])
# X = [2, 12] is exactly the centroid, so the distance should be 0.0
# actual:   {'mahalanobis': array([9.375])}
# expected: {'mahalanobis': array([0.])}

Fix

-        X_minus_mu = X - np.mean(reference_distribution)
+        X_minus_mu = X - np.mean(reference_distribution, axis=0)

Tests

Added a doctest showing a point on the distribution's centroid has distance 0. The existing doctest (reference_distribution=[[0, 1], [1, 0]]) is symmetric — its column means equal the grand mean — so it yields array([0.5]) under both the old and new code and remains unchanged.

The Mahalanobis distance centers X on the reference distribution's mean
vector (per-column centroid), but `np.mean(reference_distribution)` with no
axis collapses the 2D array to a single scalar grand mean, broadcast across
all features. `np.cov(reference_distribution.T)` already treats each column
as a variable, confirming the per-column intent. As a result a point sitting
on the distribution's own centroid reported a large nonzero distance.

Use `np.mean(reference_distribution, axis=0)` and add a doctest showing a
centroid point has distance 0.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant