Skip to content

Enable out-of-sample projection with precomputed affinities #181

Description

@aumona

Feature request

PHATE.transform(X) raises a ValueError when the model is initialized with knn_dist="precomputed_affinity".

This configuration should be supported natively, as the PHATE out-of-sample projection only requires an affinity matrix between query samples and training samples (or training landmarks). Therefore, precomputed_affinity is a valid input for PHATE.transform() and should not trigger an error.

To Reproduce

import forestgeom
import phate
from forestgeom import ForestProximity
from sklearn.datasets import load_iris
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split

iris = load_iris()
x = iris.data
y = iris.target

x_train, x_test, y_train, y_test = train_test_split(
    x, y, test_size=0.25, random_state=42, stratify=y
)

forest = RandomForestClassifier(random_state=42, n_estimators=100)
proximity_model = ForestProximity(forest=forest, weight_scheme="gap")
proximity_model.fit(x_train, y_train)

train_affinities = proximity_model.training_proximity(adjust_diagonal=True)  # sparse NxN training affinities

phate_op = phate.PHATE(
    knn_dist="precomputed_affinity",
    n_components=2,
    n_landmark=10,
    random_state=42,
    verbose=1,
)
emb_train = phate_op.fit_transform(train_affinities)

test_train_affinities = proximity_model.transform(x_test)  # sparse N_test x N affinity matrix 
emb_test = phate_op.transform(test_train_affinities)

Expected behavior

For any input affinity matrix X of shape (n_query, n_train) and a PHATE operator initialized with knn_dist="precomputed_affinity", PHATE.transform(X) should perform the standard PHATE out-of-sample extension.

Case 1: Full Diffusion Operator

If PHATE was fitted without landmarks, the output embedding should be

Z_query = P_query-train · Z_train

where:

  • P_query-train has shape (n_query, n_train),
  • P_query-train is obtained by row-normalizing the input affinity matrix X,
  • Z_train is the PHATE embedding of the training samples.

Case 2: Landmark Diffusion Operator

If PHATE was fitted using landmarks, the output embedding should be

Z_query = P_query-landmark · Z_landmark

where:

  • P_query-landmark has shape (n_query, n_landmark),
  • P_query-landmark is obtained by aggregating the affinities in X according to the landmark assignments computed during landmark operator construction, followed by row normalization,
  • Z_landmark is the PHATE embedding of the landmark points.

Actual behavior

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[3], line 32
     28 )
     29 emb_train = phate_op.fit_transform(train_affinities)
     30 
     31 test_train_affinities = proximity_model.transform(x_test)  # sparse N_test x N affinity matrix
---> 32 emb_test = phate_op.transform(test_train_affinities)

File ~/Projects/RF-PHATE/.venv/lib/python3.14/site-packages/phate/phate.py:980, in PHATE.transform(self, X, t_max, plot_optimal_t, ax)
    970 warnings.warn(
    971     "Pre-fit PHATE should not be used to transform a "
    972     "new data matrix. Please fit PHATE to the new"
    973     " data by running 'fit' with the new data.",
    974     RuntimeWarning,
    975 )
    976 if (
    977     isinstance(self.graph, graphtools.graphs.TraditionalGraph)
    978     and self.graph.precomputed is not None
    979 ):
--> 980     raise ValueError(
    981         "Cannot transform additional data using a "
    982         "precomputed distance matrix."
    983     )
    984 else:
    985     if self.embedding is None:

ValueError: Cannot transform additional data using a precomputed distance matrix.

System information:

Output of phate.__version__:

2.0.0

Output of forestgeom.__version__:

0.3.0

Output of pd.show_versions():

Details
INSTALLED VERSIONS
------------------
commit                : 72f2fea91530b5abb3cf2100cb22d84e504695c0
python                : 3.14.5
python-bits           : 64
OS                    : Darwin
OS-release            : 25.5.0
Version               : Darwin Kernel Version 25.5.0: Mon Apr 27 20:41:15 PDT 2026; root:xnu-12377.121.6~2/RELEASE_ARM64_T6041
machine               : arm64
processor             : arm
byteorder             : little
LC_ALL                : None
LANG                  : C.UTF-8
LOCALE                : C.UTF-8

pandas                : 3.0.3
numpy                 : 2.5.0
dateutil              : 2.9.0.post0
pip                   : None
Cython                : None
sphinx                : None
IPython               : 9.14.0
adbc-driver-postgresql: None
adbc-driver-sqlite    : None
bs4                   : None
bottleneck            : None
fastparquet           : None
fsspec                : None
html5lib              : None
hypothesis            : None
gcsfs                 : None
jinja2                : None
lxml.etree            : None
matplotlib            : 3.10.9
numba                 : None
numexpr               : None
odfpy                 : None
openpyxl              : None
psycopg2              : None
pymysql               : None
pyarrow               : None
pyiceberg             : None
pyreadstat            : None
pytest                : None
python-calamine       : None
pytz                  : None
pyxlsb                : None
s3fs                  : None
scipy                 : 1.18.0
sqlalchemy            : None
tables                : None
tabulate              : None
xarray                : None
xlrd                  : None
xlsxwriter            : None
zstandard             : None
qtpy                  : None
pyqt5                 : None

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions