feat: make PyTorch an optional extra - #370
Merged
hmgomes merged 3 commits intoAug 7, 2026
Merged
Conversation
hmgomes
added a commit
to hmgomes/CapyMOA
that referenced
this pull request
Aug 4, 2026
`__all__` drives `from package import *`. The lazy torch-backed names were
still listed there, so a wildcard import resolved every one of them and
raised OptionalDependencyError in a torch-free install -- even for a user
who only wanted the core names:
from capymoa.base import * -> OptionalDependencyError
from capymoa.classifier import * -> OptionalDependencyError
Affected capymoa.base, .classifier, .stream, .anomaly, .ssl and
.drift.detectors.
When PyTorch is missing, the lazy names are now removed from `__all__`, so
`import *` yields exactly what is usable. An explicit
`from capymoa.classifier import Finetune` still raises the actionable
error. With PyTorch installed `__all__` is untouched, so the public API and
the Sphinx docs are unchanged.
The torch probe moves into a cached `torch_available()` rather than being
re-resolved per attribute access.
Adds 8 regression tests (wildcard import per package, plus __all__
filtering with and without torch) and a CI step in the no-torch job.
Reported in review of adaptive-machine-learning#370.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
hmgomes
force-pushed
the
feat/torch-optional
branch
from
August 4, 2026 11:47
0bfce80 to
c54b436
Compare
`pip install capymoa` pulled torch and, on Linux, the whole CUDA stack -- ~2.9 GB across 16 nvidia/triton packages -- for every user, including someone running a Hoeffding tree on a CSV. torch is now behind the `capymoa[torch]` extra. Making it optional required the core import path to stop importing torch. Annotation-only uses were not the hard part; class bodies were, e.g. `x_dtype: torch.dtype = torch.float32`, which executes at import. So the torch-backed classes move behind a lazy boundary: - BatchClassifier / BatchRegressor split out of _classifier.py / _regressor.py into their own modules, leaving Classifier, MOAClassifier, SKClassifier and friends torch-free. - TensorDatasetWithTransform moved to datasets/_torch_utils.py; only capymoa.ocl uses it, and OCL requires torch anyway. - Torch-backed public names (Batch*, Finetune, Autoencoder, OSNN, TorchStream, ABCD) are re-exported lazily via PEP 562 module __getattr__, so `from capymoa.classifier import Finetune` still works verbatim when torch is installed. This is not an API change for existing users. - evaluation.py detects batch learners via sys.modules instead of importing them: a Batch instance cannot exist unless its module is already loaded. - capymoa.ocl and capymoa.ann require torch wholesale and now raise OptionalDependencyError naming the install command. Also declares typing-extensions, which four core modules import and which was only arriving transitively through torch. tests/test_no_torch.py blocks torch on sys.meta_path and asserts the core packages import, a classification loop runs, and torch-only names raise a helpful error. Verified additionally in a real venv with no torch installed: ARF on ElectricityTiny gives 87.9 accuracy, identical to the torch-installed run. `import capymoa` no longer loads torch's 769 submodules (1.64s -> 1.00s). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Follow-up to the previous commit, completing the optional-PyTorch work. - New "Install without PyTorch" job in pr.yml installs CapyMOA with no extras and asserts torch is absent, `import capymoa` does not load it, a prequential ARF run works, and capymoa.ocl / capymoa.ann raise an actionable OptionalDependencyError. This is the regression guard: without it the first stray top-level `import torch` silently restores the ~2.9 GB install. - README and the setup guide document `pip install capymoa[torch]`, list which features need it, and explain the CPU-only route on Linux where the default PyPI wheel is CUDA-enabled. - Fix a regression in the lazy-export helper: overriding a module's __dir__ to return only the curated name list hid eagerly-imported names from Sphinx autosummary, which then documented classes under their private module paths (capymoa.drift.detectors.abcd.ABCD instead of capymoa.drift.detectors.ABCD). Every cross-reference to those classes broke -- 249 nitpick failures, and since docs build with `-W -n` the Documentation job failed. __dir__ now unions the module's real namespace with the lazy names. - Ignore the bare `Optimizer` typehint in nitpick_ignore_regex, alongside the existing `Tensor` and `nn.Module` entries, since autodoc_typehints_format="short" strips the torch.optim.optimizer prefix. Verified: pytest 190 passed, doctest 125 passed, all 18 notebooks pass, ruff clean, and `invoke docs.build` exits 0 with the same stub attribution as main. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`__all__` drives `from package import *`. The lazy torch-backed names were
still listed there, so a wildcard import resolved every one of them and
raised OptionalDependencyError in a torch-free install -- even for a user
who only wanted the core names:
from capymoa.base import * -> OptionalDependencyError
from capymoa.classifier import * -> OptionalDependencyError
Affected capymoa.base, .classifier, .stream, .anomaly, .ssl and
.drift.detectors.
When PyTorch is missing, the lazy names are now removed from `__all__`, so
`import *` yields exactly what is usable. An explicit
`from capymoa.classifier import Finetune` still raises the actionable
error. With PyTorch installed `__all__` is untouched, so the public API and
the Sphinx docs are unchanged.
The torch probe moves into a cached `torch_available()` rather than being
re-resolved per attribute access.
Adds 8 regression tests (wildcard import per package, plus __all__
filtering with and without torch) and a CI step in the no-torch job.
Reported in review of adaptive-machine-learning#370.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
hmgomes
force-pushed
the
feat/torch-optional
branch
from
August 7, 2026 10:12
c54b436 to
17c0116
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #371.
What and why
pip install capymoapulledtorchandtorchvisionas hard dependencies. On Linux that resolves the default PyPI PyTorch wheel, which is CUDA-enabled, so every user downloaded ~2.9 GB across 16 nvidia/triton packages — including someone who just wants a Hoeffding tree on a CSV. Nothing on the core stream-learning path uses it.PyTorch is now the
capymoa[torch]extra.import capymoaimport capymoatimeThis is not just a
pyproject.tomlchangeThe core import path had to stop importing torch. Type annotations were the easy part; class bodies were the blocker, since they execute at import:
No
TYPE_CHECKINGguard helps there, so the torch-backed classes move behind a lazy boundary:BatchClassifier/BatchRegressorsplit out of_classifier.py/_regressor.pyinto their own modules, leavingClassifier,MOAClassifier,SKClassifier,Regressorand friends torch-free.TensorDatasetWithTransformmoved todatasets/_torch_utils.py. Onlycapymoa.ocluses it, and OCL requires torch anyway.Batch*,Finetune,Autoencoder,OSNN,TorchStream,ABCD— are re-exported lazily via :pep:562module__getattr__(capymoa/_optional.py).evaluation.pydetects batch learners throughsys.modulesinstead of importing them: aBatchinstance cannot exist unless its module is already loaded.capymoa.oclandcapymoa.annneed torch wholesale and raiseOptionalDependencyErrornaming the install command.Not a breaking API change
from capymoa.classifier import Finetunestill works verbatim when torch is installed — the lazy re-export is transparent. Without torch you get:instead of a bare
ModuleNotFoundError. Existing environments are unaffected:pip install -U capymoadoes not uninstall a torch that is already there.Two things found along the way
typing_extensionswas an undeclared dependency. Four core modules importoverridefrom it, but it was only arriving transitively via torch's owntyping-extensions>=4.10.0. Removing torch exposed it. Now declared explicitly — a latent packaging bug independent of this change.A lazy-export pitfall, and the reason for the
conf.pyhunk. Overriding a module's__dir__to return only a curated list hides eagerly-imported names from Sphinx autosummary, which then documents classes under private module paths (capymoa.drift.detectors.abcd.ABCDinstead ofcapymoa.drift.detectors.ABCD) and breaks every cross-reference to them — 249 nitpick failures, and since docs build with-W -n, a failed Documentation job.__dir__now unions the module's real namespace with the lazy names. The one remaining reference, the bareOptimizertypehint, is ignored alongside the existingTensorandnn.Moduleentries, becauseautodoc_typehints_format = "short"strips thetorch.optim.optimizerprefix.Wildcard imports
Raised in review:
__all__drivesfrom package import *, so listing the lazy names there made a wildcard import resolve every torch-backed name and fail in a torch-free install, even for someone who only wanted the core ones. It affectedcapymoa.base,.classifier,.stream,.anomaly,.ssland.drift.detectors.When PyTorch is missing the lazy names are now dropped from
__all__, soimport *yields exactly what is usable. An explicitfrom capymoa.classifier import Finetunestill raises the actionable error, and with PyTorch installed__all__is untouched — so the public API and the generated docs are unchanged. Covered by 8 new tests and a CI step.Validation
New "Install without PyTorch" CI job installs with no extras and asserts torch is absent,
import capymoadoes not load it, a prequential ARF run works, andcapymoa.ocl/capymoa.annraise an actionable error. This is what makes the fix stick — without it the first stray top-levelimport torchsilently restores the 2.9 GB install.tests/test_no_torch.pyblocks torch onsys.meta_pathso the same guarantees are checked inside the normal test matrix.Verified locally:
invoke docs.buildexits 0, with the same stub attribution asmainCaveat that survives this change
pip install capymoa[torch]on Linux still pulls the CUDA stack, because that is what the default PyPI wheel depends on, and package metadata cannot pin an index (PEP 508 has no such field). The CPU route is now documented in the setup guide:Note for reviewers
This touches
evaluation/evaluation.py, which #356 also restructures. Merging this first means #356 needs a rebase — deliberate, since this is user-facing friction.