Summary
This PR removes eager TensorFlow initialization from shared pipeline import paths.
Previously, importing non-TFRecord pipeline functionality could transitively import TensorFlow:
dpsynth.data_generation
→ creating_data_recorder_converter
→ tfrecord_descriptor
→ tensorflow
As a result, workflows that do not use TFRecord, including the SWIFT pipeline test, still required TensorFlow to initialize successfully during module import.
This PR defers TFRecord-specific imports until DataFormat.TFRECORD is selected and imports TensorFlow only inside TFRecord-specific I/O paths.
Changes
- Lazily import
tfrecord_descriptor when DataFormat.TFRECORD is selected.
- Move TensorFlow imports into TFRecord-specific branches in pipeline I/O.
- Preserve TensorFlow type annotations using the existing
from __future__ import annotations support and TYPE_CHECKING imports without requiring TensorFlow at module import time.
- Add regression coverage to verify that importing non-TFRecord pipeline modules does not load TensorFlow.
- Preserve existing public APIs and dependency extras.
Why
Python executes module-level imports when importing a module. The previous import structure therefore initialized TensorFlow during test collection:
pytest collection
→ import SWIFT test
→ import shared DPSynth modules
→ import TFRecord implementation
→ import TensorFlow
This happened before any SWIFT test executed.
The SWIFT workflow uses PipelineDP's LocalBackend and a dummy record converter and does not require TFRecord support. However, it could still be blocked if TensorFlow failed to initialize in the environment.
The change makes TensorFlow initialization conditional on actually entering a TFRecord code path:
shared pipeline import
→ select data format
├── non-TFRecord → TensorFlow not imported
└── TFRECORD → load TFRecord implementation → import TensorFlow
This preserves TFRecord support while preventing unrelated pipeline workflows from depending on successful TensorFlow initialization.
Verification
Confirmed that importing the shared modules no longer loads TensorFlow:
python -c "import sys; \
from dpsynth import data_generation; \
from dpsynth.pipeline_transformations import input_output; \
assert 'tensorflow' not in sys.modules"
The previously blocked pipeline tests now complete successfully:
Some unrelated JAX, Beam, httplib2, and Pyparsing warnings remain.
Summary
This PR removes eager TensorFlow initialization from shared pipeline import paths.
Previously, importing non-TFRecord pipeline functionality could transitively import TensorFlow:
As a result, workflows that do not use TFRecord, including the SWIFT pipeline test, still required TensorFlow to initialize successfully during module import.
This PR defers TFRecord-specific imports until
DataFormat.TFRECORDis selected and imports TensorFlow only inside TFRecord-specific I/O paths.Changes
tfrecord_descriptorwhenDataFormat.TFRECORDis selected.from __future__ import annotationssupport andTYPE_CHECKINGimports without requiring TensorFlow at module import time.Why
Python executes module-level imports when importing a module. The previous import structure therefore initialized TensorFlow during test collection:
This happened before any SWIFT test executed.
The SWIFT workflow uses PipelineDP's
LocalBackendand a dummy record converter and does not require TFRecord support. However, it could still be blocked if TensorFlow failed to initialize in the environment.The change makes TensorFlow initialization conditional on actually entering a TFRecord code path:
This preserves TFRecord support while preventing unrelated pipeline workflows from depending on successful TensorFlow initialization.
Verification
Confirmed that importing the shared modules no longer loads TensorFlow:
The previously blocked pipeline tests now complete successfully:
Some unrelated JAX, Beam,
httplib2, and Pyparsing warnings remain.