serious_python 3.0.0: in-process dart_bridge FFI transport, Flutter 3.44.2, manifest-driven versions#209
Merged
Conversation
…sion Restructured from the abandoned dart-bridge repo, scoped to the byte-transport primitives only. The plugin is independently consumable and shares its version + release cadence with the other serious_python* packages. - native/dart_bridge.c: drop DartBridge_RunPython and its threading machinery; keep the Dart-callable core (DartBridge_InitDartApiDL, DartBridge_EnqueueMessage) and the Python-callable shim (set_enqueue_handler_func, send_bytes) - send_bytes now detects an uninitialized Dart API DL and raises RuntimeError instead of segfaulting on a NULL Dart_PostCObject_DL function pointer - native/dart_api/: vendored Dart SDK headers needed by Dart_InitializeApiDL - python/: setup.py + pyproject.toml building the dart_bridge Python extension; cibuildwheel matrix gated to cp312/cp313/cp314 matching ci.yml - .gitignore: cover compiled extension artifacts (*.so, *.pyd, *.dylib, *.dll, *.egg-info/, *.egg/)
Generic hook for plugins shipping statically linked Python C extensions (intended primarily for iOS, where dlopen of late-loaded extensions is forbidden, but available on macOS too so callers don't need platform conditionals). Callers register (name, PyInit_*) pairs; the plugin calls PyImport_AppendInittab on each entry before every Py_Initialize, so the existing Py_Initialize/Py_Finalize cycle works correctly. - Names are strdup'd into stable storage (PyImport_AppendInittab does not copy and requires lifetime >= interpreter) - NSLock-guarded registration list, safe to call from any thread - No bridge-specific code: serious_python_darwin stays agnostic, any future static Python extension can register the same way
…ridge Standard FFI Flutter plugin layout, sibling to serious_python_*. Pure FFI on Linux/Windows/Android; iOS+macOS additionally declare pluginClass + sharedDarwinSource for the Swift code that will call SeriousPython.registerPythonExtension (added in a later step). - lib/serious_python_bridge.dart: PythonBridge singleton wrapping the native symbols (DartBridge_InitDartApiDL, DartBridge_EnqueueMessage) and Python's send_bytes reply path via a ReceivePort. Generic byte-transport API (Stream<Uint8List> messages, send(Uint8List), nativePort, dispose) — no protocol/msgpack/Flet specifics. DynamicLibrary discovery is platform- aware: process() on iOS (static link), .dylib/.dll/.so elsewhere. - pubspec.yaml: declares ffiPlugin on all five platforms, path deps on serious_python + serious_python_darwin (patch_pubspec.py rewrites at release). - README/CHANGELOG/LICENSE/analysis_options matching sibling plugin pattern. - python/pyproject.toml: drop readme reference outside project root that setuptools rejected.
Standard Flutter FFI plugin layout: per-platform CMakeLists wrappers under
linux/ and windows/ each call add_subdirectory("../native"), and set
serious_python_bridge_bundled_libraries to $<TARGET_FILE:flet_bridge> so
Flutter bundles the produced shared library alongside the app binary.
The cross-platform native/CMakeLists.txt uses find_package(Python3 ...
Development.Module) + Python3::Module — this gives header-only Python deps
and the correct per-platform link behavior (Linux: dynamic resolution;
Windows: pythonXY.lib import library; macOS: -undefined dynamic_lookup,
though macOS desktop uses the podspec, not this CMakeLists).
Verified locally on macOS host: configure + build produces libflet_bridge.dylib
(53K) with exported DartBridge_InitDartApiDL + DartBridge_EnqueueMessage and
unresolved Py* references (PyGILState_*, PyBytes_*, PyObject_CallFunctionObjArgs,
etc.) — resolved at runtime against libpython loaded by serious_python.
Known limitation: uses the host system's Python.h. Cross-build correctness
(Android NDK, manylinux CI) will need an override pointing at the matching
python-build-standalone tarball; deferred until step that needs it.
Unified Apple-platform path: both iOS and macOS desktop statically link dart_bridge.c into the app process and register PyInit_dart_bridge via serious_python_darwin's registerPythonExtension hook. This diverges from the original plan (which had macOS in the same "two artifacts via RTLD_GLOBAL" bucket as Linux/Windows/Android), and is cleaner: the macOS wheel matrix vanishes, Dart's view and Python's view of global_enqueue_handler_func are guaranteed to share storage, and the iOS and macOS code paths are now symmetrical. - darwin/serious_python_bridge.podspec: source_files pulls Classes/**/* + ../native/dart_bridge.c + the Dart SDK headers; OTHER_LDFLAGS includes -undefined dynamic_lookup so Py* references defer to the final app link against serious_python_darwin's vendored Python xcframework; depends on serious_python_darwin pod. - darwin/Classes/SeriousPythonBridgePlugin.h: forward-declares PyInit_dart_bridge for Swift module-map visibility. - darwin/Classes/SeriousPythonBridgePlugin.swift: imports Python + serious_python_darwin, calls registerPythonExtension at Flutter plugin init time on both iOS and macOS. - lib/serious_python_bridge.dart: _openNativeLibrary now uses DynamicLibrary.process() on macOS too (no separate dylib lookup). pod spec lint --quick produces the same 2 warnings as the sibling serious_python_darwin.podspec (license type + source primary key) — both publication-metadata, irrelevant for in-monorepo plugins.
Bridge plugin's Android gradle downloads the same python-android-dart tarball serious_python_android uses, but extracts only include/ + lib/libpython*.so into a build-time scratch dir (build/python-dist/<abi>/). libflet_bridge.so links against libpython3.so at build time so Android's strict dynamic linker accepts the .so on load; at runtime libpython is resolved against the copy serious_python_android ships in the same jniLibs/<abi>/ directory of the APK. We do NOT ship libpython from this plugin. - android/build.gradle: AGP 7.3.0 + de.undercouch.download (matching the sibling), per-ABI downloadPythonDist_<abi> + extractPythonDist_<abi> tasks wired into preBuild. ABI matrix matches the sibling's PEP-738 logic (32-bit only on 3.12). Caches under the shared FLET_CACHE_DIR. - android/CMakeLists.txt: sets FLET_BRIDGE_PYTHON_INCLUDE_DIRS and FLET_BRIDGE_PYTHON_LIBRARIES to point at the staged dir, then includes ../native via add_subdirectory. Adds the Android 15 16KB-page-size link flag matching the sibling. - android/src/main/AndroidManifest.xml: minimal, matches the sibling. - native/CMakeLists.txt: refactored to accept FLET_BRIDGE_PYTHON_* from a parent scope (Android uses this) and fall back to find_package(Python3 ... Development.Module) + Python3::Module on desktop platforms. Backward-compatible: macOS host configure+build still produces libflet_bridge.dylib with the same exports + unresolved Py* references.
Minimal Flutter app exercising serious_python_bridge end-to-end without
Flet. Sends bytes from Dart via PythonBridge.send(), Python echoes them
back via dart_bridge.send_bytes(). Scaffolded with `flutter create
--platforms=android,ios,linux,macos,windows --org=com.flet`; bulk of the
126 files is standard Flutter platform boilerplate.
Hand-authored pieces:
- lib/main.dart: PythonBridge.init() + ReceivePort listener, fire-and-
forget SeriousPython.run("app/app.zip"), 8-byte little-endian handshake
frame containing bridge.nativePort, button-driven `ping #N` sends.
- app/src/main.py: dart_bridge echo loop; first frame interpreted as
native_port, subsequent frames echoed back via dart_bridge.send_bytes()
prefixed with b"echo: ". threading.Event().wait() keeps the interpreter
alive indefinitely.
- app/src/requirements.txt: empty (dart_bridge is static-linked on iOS+
macOS via inittab, downloaded via --bridge on Linux/Windows/Android).
- pubspec.yaml: slimmed from default scaffold; deps are just
serious_python (path) + serious_python_bridge (path) +
integration_test dev dep. Aligned SDK constraint to monorepo standard.
- integration_test/.gitkeep: placeholder; step 5 of the plan populates
this with the round-trip echo test.
- README.md: build + run instructions.
- .gitignore: appended /app/app.zip and /app/app.zip.hash (build
artifacts from `dart run serious_python:main package`).
- Removed auto-generated test/widget_test.dart (referenced default
scaffold class names).
flutter analyze: 1 warning (asset_does_not_exist for app/app.zip — the
build artifact built by package_command at integration test time);
zero errors.
On tag pushes (refs/tags/v*) the workflow now: 1. release_build (matrix): cibuildwheel v2.21.3 against src/serious_python_bridge/python on ubuntu-24.04, ubuntu-24.04-arm, windows-latest, producing wheels for cp312/cp313/cp314. Wheels uploaded as bridge-wheels-<os> artifacts. Apple platforms (iOS + macOS desktop) excluded — they use static-link + inittab via the bridge plugin's darwin podspec; no wheel needed. 2. collect_release: downloads all wheel artifacts, creates (or updates) the GitHub Release for the tag, attaches every .whl as a release asset via softprops/action-gh-release@v2. 3. publish: now gated on collect_release (binaries must be on the GitHub Release before pub.dev pushes are irreversible). Bridge package added to the patch_pubspec.py loop and to the dart pub publish sequence; published last (after a sleep 600) so its path-deps on serious_python + serious_python_darwin have time to propagate on pub.dev. Android wheels deferred — cibuildwheel's Android support is experimental and serious_python_android already downloads platform- specific tarballs via a custom NDK-aware gradle path. Will be added in a follow-up CI change. Pre-flight: python -m build --wheel against the package locally produced dart_bridge-0.0.0-cp312-cp312-macosx_14_0_arm64.whl containing just dart_bridge.cpython-312-darwin.so + metadata, confirming the ../native relative path in setup.py survives PEP 517.
abi3 / Limited API
------------------
One binary per platform now serves all Python 3.12+ minor versions
(forward-compatible with 3.15, 3.16, etc. with no rebuild). Every Py*
symbol used in dart_bridge.c is in the Limited API since 3.2 (oldest:
PyGILState_Ensure/Release in 3.4).
- native/dart_bridge.c: #define Py_LIMITED_API 0x030c0000 before
Python.h. Comment lists symbol provenance.
- python/setup.py: Extension(..., py_limited_api=True,
define_macros=[("Py_LIMITED_API", "0x030c0000")]). py_limited_api on
Extension controls the module suffix (.abi3.so) but NOT the wheel
filename tag.
- python/setup.cfg (new): [bdist_wheel] py_limited_api = cp312 so the
produced wheel filename gets the cp312-abi3-<plat> tag (consumed by
any Python 3.12+).
- python/pyproject.toml: narrowed [tool.cibuildwheel] build from
"cp312-* cp313-* cp314-*" to just "cp312-*". One wheel build per
host suffices.
Verified locally on macOS: setup.py build_ext --inplace produces
dart_bridge.abi3.so (smoke test passes — set_enqueue_handler_func,
send_bytes(0) and send_bytes(non_zero) all behave correctly).
python -m build --wheel produces
dart_bridge-0.0.0-cp312-abi3-macosx_14_0_arm64.whl.
Android NDK cross-build
-----------------------
New release_build_android job in ci.yml runs on ubuntu-24.04, matrix
of just ABIs (arm64-v8a, armeabi-v7a, x86_64). Single Python version
(3.12) pinned for headers + libpython to link against; abi3 makes the
resulting .so usable across all Python 3.12+.
- Downloads python-android-dart-3.12-<abi>.tar.gz from
flet-dev/python-build (same source serious_python_android uses).
- Discovers Python.h + libpython3.12.so defensively via `find`.
- Cross-compiles with NDK clang to dart_bridge.abi3-android-<abi>.so
(ABI in filename so all three ABIs coexist as Release assets without
collision; package_command.dart will rename to canonical
dart_bridge.abi3.so when placing into the bundled site-packages).
- 16K page size link flag for Android 15.
- collect_release extended: downloads bridge-android-* artifacts
alongside bridge-wheels-*, attaches both to the GitHub Release.
Total Release-asset reduction: 16 → 6 binaries (3 wheels + 3 Android
.so files).
Mirrors release_build + release_build_android from ci.yml without the refs/tags/v gate and without GH Release / pub.dev publish steps. Triggered on push to dart-bridge branch or via workflow_dispatch, so we can verify the cibuildwheel matrix + Android NDK cross-build actually produce the expected abi3 binaries before we ever cut a real release tag. Adds "list built wheels" + "inspect output" steps so failures are diagnosable from CI logs. Delete this workflow file before merging to main.
The python-android-dart-<ABI>.tar.gz variant only ships libpython3.X.so and libpythonbundle.so — no Python.h. That was fine for serious_python_android (which uses pre-generated ffigen bindings) but broke our bridge cross-compile, which #include <Python.h>. Switching to python-android-mobile-forge-3.12.tar.gz: - Single 314MB tarball contains all 4 Android ABIs' python-3.12.13/ installs with both include/ and lib/ subtrees - Selective tar extraction (only install/android/<ABI>/python-3.12.13/ include/ + lib/) keeps disk usage per cell modest - find paths now match install/android/<ABI>/python-3.12.13/include/ python3.12/Python.h and lib/libpython3.12.so Verified locally: tarball contents match the expected layout (install/android/<abi>/python-3.12.13/include/python3.12/Python.h exists for arm64-v8a, armeabi-v7a, x86, x86_64). Same fix applied to both ci.yml's release_build_android and the throwaway test-bridge-build.yml.
…asses)
Round-trip echo test in integration_test/bridge_echo_test.dart sends a 1KB
random Uint8List from Dart, Python's app/src/main.py echoes it back with
a b"echo: " prefix, Dart asserts byte-identical equality. Passes on macOS
host in ~1s.
Several issues discovered + fixed getting the bridge to actually run:
native/dart_bridge.c
Add Py_IsInitialized() guard at the top of DartBridge_EnqueueMessage.
Dart's first handshake send arrives BEFORE Python's Py_Initialize
completes; without the guard PyGILState_Ensure aborts with
"PyMUTEX_LOCK(gil->mutex) failed" against the uninitialized gil
mutex. The guard makes early sends a silent no-op so Dart's retry
loop can resend until Python is up.
darwin/Classes/dart_bridge.c, darwin/Classes/dart_api/** (symlinks)
CocoaPods silently drops s.source_files entries that traverse outside
the pod root via '../', so committed per-file symlinks into native/
are the only reliable way to compile dart_bridge.c + dart_api_dl.c on
Apple platforms. Directory symlinks don't work either — Ruby's
Dir.glob doesn't descend through them. Per-file symlinks inside a
real dart_api/ directory + a real dart_api/internal/ directory work.
darwin/serious_python_bridge.podspec
- Add Python.framework's Headers/ subdirectory to HEADER_SEARCH_PATHS
so dart_bridge.c's `#include <Python.h>` (no framework prefix)
resolves once CocoaPods has extracted the xcframework slice into
PODS_XCFRAMEWORKS_BUILD_DIR.
- Drop public_header_files (no SeriousPythonBridgePlugin.h anymore).
darwin/Classes/SeriousPythonBridgePlugin.{swift,h}
Drop the C header that declared PyInit_dart_bridge — having <Python.h>
in the pod's umbrella modulemap required Python framework search
paths during modulemap scanning that CocoaPods doesn't propagate
reliably from transitive deps. Swift now uses dlsym(RTLD_DEFAULT,
"PyInit_dart_bridge") to find the symbol via the global process
table.
example/app/src/main.py
After capturing native_port from the 8-byte handshake, echo the same
8 bytes back. Dart uses that echo as the readiness signal.
example/lib/main.dart
Replace the racy `Future.delayed(2s)` with a retry-until-Python-
echoes-the-handshake loop. Subscribe to bridge.messages BEFORE
sending (broadcast streams don't replay; subscribing after send loses
fast echoes). Debug prints retained for diagnosability.
example/integration_test/bridge_echo_test.dart (new)
Drives the bridge directly without going through BridgeExampleApp's
UI: pump a minimal SizedBox to trigger plugin registration, then
init the bridge, run the handshake retry, send a fixed-seed 1KB
random payload, await the echo, assert byte-identical.
example/macos/{Runner.xcodeproj,Runner.xcworkspace,Podfile.lock}
Updated by `pod install` to register the bridge pod. flet_example
tracks the same files; keeping parity.
New bridge_example_macos job mirrors the existing macos job but
operates on src/serious_python_bridge/example/. Runs the same Python
version matrix (3.12 / 3.13 / 3.14) — verifies the abi3 + static-link +
inittab path works against each embedded CPython.
Steps:
1. dart run serious_python:main package app/src --platform Darwin
--python-version <ver> (builds app.zip; --bridge not needed on
macOS since the bridge is static-linked
via the darwin podspec)
2. flutter test integration_test -d macos (runs the 1KB byte-
identical echo test from
bridge_echo_test.dart)
Added bridge_example_macos to publish.needs so a failing round-trip
blocks pub.dev publication.
Linux/Windows/Android/iOS bridge_example jobs are deferred to the
next step (5c).
…ge example to throwaway workflow - ci.yml `on.push.branches` temporarily narrowed from '**' to ['main', 'master'] so iteration pushes to dart-bridge skip the ~20-30 min flet_example platform matrix. Tag pushes still fire the full workflow (release builds + publish). REVERT before merging. - test-bridge-build.yml: add test_bridge_example_macos matrix job (Python 3.12/3.13/3.14) that mirrors the new bridge_example_macos job from ci.yml without the dependency on the rest of the matrix. Renamed workflow's display name to reflect broader scope. Net effect on dart-bridge pushes: only the fast throwaway workflow runs (~3 build cells + 3 Android cross-build cells + 3 macOS bridge example cells ≈ ~3-5 min for the slowest cell vs ~30 min before).
Flutter version
---------------
.fvmrc: 3.29.3 → 3.41.7 to match flet repo.
Bridge example matrix expanded (step 5c)
----------------------------------------
ci.yml + test-bridge-build.yml now run the bridge integration test on:
- macOS (Apple path: static-link + inittab via darwin podspec)
- iOS (same Apple path; simulator launched via futureware-tech/
simulator-action — same as flet_example)
- Linux amd64 + arm64 (abi3 wheel from release_build/test_wheel_build
passed to `dart run serious_python:main package --requirements
<local-wheel>`)
- Windows (same wheel-from-artifact pattern)
All run the 3.12/3.13/3.14 Python matrix (abi3 means the same .so loads
on each). 13 cells per workflow:
3 macos + 3 ios + 6 linux (2 arches × 3 versions) + 3 windows
Linux/Windows jobs depend on release_build (ci.yml) or test_wheel_build
(throwaway) so the locally-built wheel is available. Linux/Windows are
also gated to tag pushes in ci.yml — release_build only runs on tags.
publish.needs extended with all four new bridge_example_<plat> jobs so
a failing round-trip blocks pub.dev publication.
Android deferred — its site-packages live inside libpythonsitepackages
.so (a zip-as-.so archive built by serious_python_android's gradle), so
injecting dart_bridge.abi3.so needs a different path than the desktop
wheel-as-requirement approach. Will need either:
- package_command.dart `--bridge` flag (step 7 work) that knows how
to inject .so into Android's pythonsitepackages zip, or
- A custom script step in the test workflow that patches the bundle
before the integration test runs.
Three issues from the previous CI run: 1. test_wheel_build (throwaway) didn't upload wheel artifacts — only built them and listed filenames. Bridge example Linux/Windows then failed downloading the missing artifacts. Added actions/upload- artifact step that mirrors ci.yml's release_build. 2. iOS bridge_example failed: serious_python_darwin's sync_site_packages.sh only populates dist_ios/site-xcframeworks when iOS-specific site-packages subdirs (iphoneos.arm64, iphonesimulator.arm64, iphonesimulator.x86_64) exist. Empty --requirements falls to the macOS branch which doesn't create that directory, and bundle-python-frameworks-ios.sh later fails `find dist_ios/site-xcframeworks` with `No such file or directory`. Workaround: pass `--requirements certifi` (a tiny pure-Python placeholder, the same shape as flet_example's flet==0.28.3) to trigger the iOS site-xcframeworks population. 3. test-bridge-build.yml had no workflow-level SERIOUS_PYTHON_SITE_PACKAGES env var. ci.yml sets this at the top (line 15) so all jobs inherit. The throwaway needs the same so sync_site_packages.sh knows where package_command.dart staged the bundled site-packages. The certifi workaround is fragile; a cleaner long-term fix is to patch sync_site_packages.sh to always create site-xcframeworks (with just the base python xcframeworks, no user deps required). Adding to bridge-followups.
…bi3 stub Three CI failures from the previous run, all in the Linux + Windows bridge_example matrix: Linux: Picked the wrong wheel — cibuildwheel produces both x86_64 AND i686 variants for ubuntu hosts, and `ls | head -n1` lands on dart_bridge-*manylinux*_i686.whl alphabetically. pip then refuses with "is not a supported wheel on this platform". Fix: skip i686 in [tool.cibuildwheel] so only x86_64 ships. (Also pre-emptively skip win32 for symmetry; Flet desktop is 64-bit only.) Windows: `LINK : fatal error LNK1104: cannot open file 'python314_d.lib'` — find_package(Python3 ... Development.Module) picks the version- specific debug-config import lib in Debug builds, but python-build- standalone for Windows doesn't ship a Debug variant. Fix: native/CMakeLists.txt — on WIN32, link against python3.lib (the abi3 stable-ABI stub) directly. It's version-agnostic and has no debug variant requirement, matching our Py_LIMITED_API choice in dart_bridge.c. iOS (separate flake): iOS 3.14 passed in 12m26s but iOS 3.12 and 3.13 hung past 1 hour in the "Package + run integration test" step. Adding timeout-minutes: 25 to both bridge_example_ios in ci.yml and test_bridge_example_ios in the throwaway so future hangs fail loudly within 25 min instead of consuming the 6-hour job default. Root cause TBD — likely a CocoaPods/simulator zombie since one of three Python versions succeeded.
cibuildwheel's Linux build identifier is `cp312-manylinux_i686`, not `cp312-linux_i686`, so my earlier `*-linux_i686` skip pattern didn't match and i686 wheels kept being built (alphabetically picked first by `ls | head -n1` over x86_64, then rejected by pip with "is not a supported wheel on this platform"). The broader `*_i686` matches every i686 build identifier regardless of the libc prefix (manylinux, musllinux, etc.).
…erred) Root cause is the symbol visibility split the original plan called out: the wheel-installed dart_bridge.so (cibuildwheel) and the Flutter-built libflet_bridge.so are two separate binaries with two separate copies of global_enqueue_handler_func. Python's set_enqueue_handler_func writes to the wheel's copy; Dart's DartBridge_EnqueueMessage reads libflet_bridge.so's (NULL); messages are silently dropped and the handshake echo never fires — "Python did not echo handshake within 30s". Apple platforms (macOS + iOS) don't hit this because the bridge is static-linked into the framework, so there's only one copy. Proper fix: dart_bridge_core.c + dart_bridge_shim.c, with the shim dlopening libflet_bridge.so RTLD_GLOBAL so its `extern global_enqueue_handler_func` resolves to libflet_bridge.so's copy. Documented in bridge-followups.md. For now, continue-on-error so the iOS/macOS proof-of-life survives and we get diagnostic data on the Linux/Windows symbol-split work without blocking other CI.
…n module)
Fixes the Linux/Windows/Android bridge_example handshake-timeout failure
by making Python and Dart share the SAME `global_enqueue_handler_func`
cell on every platform.
Before: the wheel-installed dart_bridge.so (cibuildwheel) and the
Flutter-built libflet_bridge.so each compiled a full copy of
dart_bridge.c — two separate binary instances, two separate copies of
the static `global_enqueue_handler_func`. Python's
set_enqueue_handler_func wrote to the wheel's copy; Dart's
DartBridge_EnqueueMessage read libflet_bridge's copy (NULL); messages
were silently dropped.
After: the core lives in libflet_bridge ONLY. The wheel ships a thin
shim that resolves the core's exports at PyInit time via runtime
symbol lookup (dlsym(RTLD_DEFAULT) → dlopen RTLD_GLOBAL fallback on
Linux/macOS, LoadLibrary+GetProcAddress on Windows). One global cell,
visible to both sides.
Changes:
- native/dart_bridge.c: drop the Python-callable methods. Exports
three symbols for the shim to resolve:
* dart_bridge_global_enqueue_handler_func (PyObject*, was static)
* dart_bridge_post_to_dart (helper that wraps Dart_PostCObject_DL
so the shim doesn't need its own copy of dart_api_dl.c)
* existing DartBridge_InitDartApiDL + DartBridge_EnqueueMessage
- native/dart_bridge_shim.c (NEW): PyInit_dart_bridge resolves the
three exports above via shim_sym_lookup(); set_enqueue_handler_func
writes through the resolved pointer; send_bytes calls through the
resolved function pointer. ImportError if libflet_bridge isn't
loaded into the process (catches misconfiguration loudly).
- python/setup.py: compile only dart_bridge_shim.c; add -ldl on Linux
for the dlopen/dlsym used by the shim. Dropped dart_api_dl.c from
the wheel's sources (the shim doesn't call Dart_PostCObject_DL
directly).
- darwin/Classes/dart_bridge_shim.c (symlink): so the Apple static
link picks both .c files into the framework. Apple's
dlsym(RTLD_DEFAULT) finds the symbols in-process (smoke-tested:
macOS bridge_example_macos still passes).
- ci.yml + test-bridge-build.yml: drop continue-on-error from
bridge_example_linux and bridge_example_windows — they should pass
now.
Verified locally on macOS: `flutter test integration_test -d macos`
still green. CI verification pending — pushing to dart-bridge will
exercise all three desktop platforms.
Windows: Previous python3.lib fix wasn't enough — Python.h still emits `#pragma comment(lib, "python<XY>_d.lib")` in Debug builds via pyconfig.h's auto-link, and python-build-standalone doesn't ship a Debug lib. Add `Py_NO_LINK_LIB` compile-time define to suppress the pragma (Python 3.12+). Our explicit target_link_libraries reference to python3.lib (the abi3 stub) is then the only libpython reference. Linux ARM64 matrix: test-bridge-build.yml's test_bridge_example_linux matrix declared python_version in the base but `arch` only via `include:` entries. GitHub Actions doesn't expand the matrix correctly for that shape — only 3 AMD64 jobs ran, no ARM64. Adding `arch: [arm64, amd64]` to the base so it cross-multiplies with python_version (6 cells). ci.yml already had this right. Android bridge_example (new): Added test_bridge_example_android matrix job to test-bridge-build.yml. Mirrors the existing flet_example Android pattern (reactivecircus/android-emulator-runner, x86_64 emulator, API 33). Targets only x86_64 to match the emulator arch — keeps CI fast. Pre-package step downloads the bridge-android-x86_64 artifact (from test_android_build) and drops the cross-compiled dart_bridge.abi3-android-x86_64.so as `dart_bridge.abi3.so` into $SERIOUS_PYTHON_SITE_PACKAGES/x86_64/. serious_python_android's gradle then zips that directory into libpythonsitepackages.so and bundles it in the APK; at runtime Python's import finds the bridge shim via the canonical filename. The shim then dlopens libflet_ bridge.so (already loaded by Dart) to resolve the core symbols — same single-cell architecture as Linux/macOS/Windows.
- test-bridge-build.yml's test_android_build was missing the actions/upload-artifact step, so test_bridge_example_android's download failed (Artifact not found). Adding the upload mirroring test_wheel_build. - Add stderr diagnostics to dart_bridge_shim's Windows code path: log LoadLibraryA error code if flet_bridge.dll lookup fails, and log GetProcAddress error if a symbol isn't found. The macOS+Linux shim split works, but Windows is still failing with a handshake timeout — instrumented to figure out whether the DLL is on the search path or symbols are exported under different names.
When dart_bridge.pyd (the Python shim, loaded into the embedded
Python by `import dart_bridge`) does `LoadLibraryA("flet_bridge.dll")`,
Windows' default DLL search starts from the calling module's
directory — which is `runner/Debug/site-packages/`, NOT
`runner/Debug/` where Flutter places plugin DLLs. So LoadLibrary
silently returns NULL, the shim raises ImportError at PyInit time,
and Dart's handshake retry loop times out at 30s.
Fix: shim_find_flet_bridge_module() now tries three strategies in order:
1. GetModuleHandleA("flet_bridge.dll") — picks up a copy already
loaded by Dart's DynamicLibrary.open (no I/O).
2. LoadLibraryA("flet_bridge.dll") — default search path.
3. Construct the absolute path next to the running .exe (via
GetModuleFileNameA(NULL)) and LoadLibrary that. This is what
Flutter's plugin loader uses to locate flet_bridge.dll.
Linux/macOS unaffected (dlopen + dlsym pathway unchanged).
Windows: Python's stderr isn't captured by flutter test on Windows, so fprintf diagnostics in the shim were invisible. Switch to file logging — shim now appends to dart_bridge_shim.log in CWD with every lookup attempt + result. New diagnostics step at end of the windows job (if: failure()) lists the runner/Debug build dir, site-packages contents, and cats the shim log so we can see exactly which lookup strategy failed. Android: Earlier `cd src/serious_python_bridge/example` ran on its own line in the emulator-runner script — each line is a fresh shell, so subsequent `dart run` saw the workspace root (no pubspec.yaml) and failed with "Found no pubspec.yaml file". Switched to flet_example's pattern: cd && command on each line.
…flatten)
Windows — the abi3 wheel dart_bridge.pyd imports python3.dll, but
serious_python_windows only bundles python3_d.dll in Debug builds.
That made the .pyd silently fail to load (no shim log written, no
ImportError surfaced through Python's stderr which flutter test
doesn't capture on Windows). Found via the new diagnostics step
which dumped runner/Debug/'s contents.
Fix: bridge plugin's windows/CMakeLists.txt downloads its own copy
of python-windows-for-dart-<ver>.zip (the same flet-dev/python-build
artifact serious_python_windows uses), extracts just python3.dll,
and adds it to serious_python_bridge_bundled_libraries. Flutter
copies it next to the .exe; Windows' DLL loader then resolves
dart_bridge.pyd's python3.dll dependency.
Android — gradle was using python-android-dart-*-<abi>.tar.gz which
only ships runtime libs (no headers, no libpython3.so), and its
include patterns matched files at the tarball's root rather than at
the install/android/<abi>/python-<X.Y.Z>/ nesting.
Fix: switched bridge plugin's android/build.gradle to the
python-android-mobile-forge-<ver>.tar.gz tarball (one download covers
all four ABIs with both headers + libpython*.so), expanded include
patterns to match the nested layout, and added an eachFile rule to
flatten install/android/<abi>/python-<X.Y.Z>/ out so the staged
files land at python-dist/<abi>/{include,lib}/... where
android/CMakeLists.txt expects them.
…p extras Previous shim_log used a relative path that landed wherever Python's cwd was at PyInit time — Python on Windows runs from app/ extraction dir, not from src/serious_python_bridge/example/ where the diagnostics step was searching. Switched to GetTempPathA-based absolute path. Added a shim_log_init() call at the very top of PyInit_dart_bridge. If the log is missing entirely from %TEMP%, PyInit never ran (the .pyd failed to load before module init). If the log has "PyInit entered" but no further lookups, the issue is elsewhere. Diagnostics step now cats $TEMP/$RUNNER_TEMP for the shim log and also runs dumpbin /dependents against the .pyd to confirm its import requirements vs what's in runner/Debug/.
Two changes to find why PyInit_dart_bridge isn't running on Windows: 1. shim_log writes to a path constructed from GetModuleFileNameA(NULL) (alongside the running .exe in runner/Debug/) instead of GetTempPathA. The latter can resolve differently between Python's embedded environment and the diagnostics shell's $TEMP, so even if the shim was logging we'd miss it. 2. Diagnostics step now runs a direct `python -c "import dart_bridge"` against the bundled Python in runner/Debug/ to surface the actual ImportError that Python sees when loading dart_bridge.pyd. If that succeeds, the .pyd loads and the issue is elsewhere; if it fails, the error message tells us exactly which dependency is missing or which symbol clash trips the loader.
…nDist Windows: Flutter's --debug config makes serious_python_windows bundle the Debug python3XY_d.dll while the cibuildwheel-built abi3 dart_bridge wheel imports python3.dll, which forwards to the Release python3XY.dll. Use --profile (Release Python) on the integration test so the embedded interpreter matches the wheel's link target. Android: extractPythonDist_<abi> still ends without libpython3.so on disk. Print the tar path, staging dir, and final tree contents so we can see whether the task is even running and what include patterns actually match.
Windows: drop the $<CONFIG:Debug>:_d> generator expressions in
serious_python_windows so the plugin always links and bundles Release
pythonXY.dll / python3.dll, regardless of Flutter's build config. Add
Py_NO_LINK_LIB to the plugin target so pyconfig.h's auto-link pragma
doesn't pull in pythonXY_d.lib in Debug Flutter builds. The cibuildwheel-
built abi3 dart_bridge wheel (and any future C-extension wheel piped
through `package_command.dart --bridge`) is Release-ABI; mixed Debug
serious_python_windows + Release wheel was the only configuration that
ever failed the bridge example. Release CRT is already shipped
unconditionally next to the .exe, so the wheel's CRT dependency is
satisfied in Debug Flutter builds too.
Side effect: removes the `flutter test --profile` workaround from the
bridge example Windows job — default Debug `flutter test` now matches the
abi3 wheel.
Android: the gradle preparePythonDist task stages headers + libpython3.so
into ${buildDir}/python-dist/<abi>, where buildDir is the *consumer
module's* build dir (e.g. example/build/serious_python_bridge/), not the
bridge plugin's source tree. android/CMakeLists.txt was hardcoded to
${CMAKE_CURRENT_SOURCE_DIR}/build/python-dist/<abi> (plugin-source-
relative) — never the same path. Thread the actual staging root from
gradle to CMake via -DPYTHON_DIST_DIR.
Windows: Py_NO_LINK_LIB is only honored by Python 3.14+'s pyconfig.h —
3.12 and 3.13 don't guard the `pragma comment(lib, "pythonXY_d.lib")`
auto-link directive with that macro, so the Debug pragma fired regardless
and the link failed with LNK1104. Add `/NODEFAULTLIB:pythonXY_d.lib` at
link time to both serious_python_windows_plugin and flet_bridge — works
across every supported Python version. Keep the Py_NO_LINK_LIB define
too, for 3.14+ cleanliness.
Android: gradle errored with "unknown property 'pythonStagingRoot'"
because the variable was defined after the `android {}` block but
referenced inside it (in the cmake.arguments expression). Hoist the
definition above `android {}`.
The Dart `_pythonReleases` registry is the source of truth for the Python version matrix, but until now CI (and any other external consumer) needed a hand-maintained lookup table to derive the full patch version + python-build release date from the short SERIOUS_PYTHON_VERSION. Last CI run failed exactly because the new SERIOUS_PYTHON_FULL_VERSION / SERIOUS_PYTHON_BUILD_DATE env vars were unset in CI, the plugin scripts fell back to their 3.14.6/20260611 defaults, and Linux's ninja then complained that the 3.12 tarball was missing libpython3.14.so.1.0. New `version` subcommand emits the registry as either a human-readable summary or `--json` (machine-readable). Self-version comes from pubspec.yaml looked up via `package_config` (no Dart constant to keep in sync). Registry rows are sorted descending in the text output to match `flet --version` ordering. `_pythonReleases` and `_PythonRelease` promoted to `pythonReleases` / `PythonRelease` so the new file can reference them. Internal callers in `package_command.dart` updated to match. CI: new composite action `.github/actions/resolve-python-vars` runs `dart run serious_python:main version --json` + `jq` to write SERIOUS_PYTHON_FULL_VERSION and SERIOUS_PYTHON_BUILD_DATE into `$GITHUB_ENV`. Inserted as a `Resolve Python version vars` step right after `Setup Flutter` in all 15 test jobs (publish job left untouched). Future Python registry bumps need no CI changes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
On a cold pub-cache, `dart run serious_python:main version --json`
prints "Building package executable..." (and "Built …") to stdout
ahead of our JSON document. jq then trips on "Building" with
"parse error: Invalid numeric literal at line 1, column 8".
The resolve-python-vars composite action now pipes the output through
`sed -n '/^{/,$p'` so jq sees the JSON document only. Warm-cache
invocations (CI cache hits, local dev) are unaffected — the sed pass
is a no-op when the first line is already `{`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Update CI workflow to use astral-sh/setup-uv@v8.2.0 instead of v6 in .github/workflows/ci.yml. Replaces three occurrences to ensure the workflow uses the newer setup-uv release.
Last CI run failed with "did not return values for SERIOUS_PYTHON_VERSION=3.12" but the composite action's debug `echo "$json"` printed nothing — couldn't tell from the log whether `dart run` errored silently or returned non-JSON. Capture stdout+stderr into a single var, print it verbatim before piping through sed/jq, and let `flutter pub get` print its own progress (was previously >/dev/null). Once we see the actual output we can decide if this is a Dart precompile hiccup, a package_config issue, or something else. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…-based sed
Last run uncovered the actual prefix: `dart run` joins "Running build
hooks..." (sometimes printed multiple times) onto the same line as our
JSON's opening `{`, so my line-based `sed -n '/^{/,$p'` matched no line
and ate the JSON entirely.
Replace it with `json="{${raw#*\{}"` — bash parameter expansion that
extracts from the first `{` byte forward, regardless of newlines. Guard
the empty-output case explicitly so the error message is unambiguous.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Update .github/workflows/ci.yml to use actions/cache@v5 (replacing actions/cache@v4) for multiple cache steps. The change applies to AVD cache steps and Flet download caches across the CI matrix to pick up fixes and improvements in the newer cache action.
Update .github/workflows/ci.yml to use futureware-tech/simulator-action@v5 (was @v4) for all 'Setup iOS Simulator' steps across multiple CI jobs, ensuring the workflow uses the newer simulator-action release.
Three example directories collapse into two:
* `flet_example/` (was `flet_ffi_example/`) — the FFI-transport demo.
Sample only; no longer runs in CI.
* `bridge_example/` — the raw-PythonBridge CI gate. Extended with
interactivity, throughput, and memory tests (details below).
* Old `flet_example/` (socket transport) deleted. Its regression
coverage is folded into bridge_example: the FFI byte transport
underneath every flet example is what bridge_example now tests
rigorously.
Reasons:
* The dart_bridge FFI transport has been green across the full
matrix for a week; the socket path was kept around as a fallback
that no Flet user will reach now that the template ships FFI.
* Maintaining two parallel example apps for the same Python+Flet
contract was duplicated work; renaming the FFI one back to
flet_example removes the "wait, which one's the recommended path"
confusion.
CI surgery in .github/workflows/ci.yml:
* Removed the five old `flet_example` jobs (macOS/iOS/Android/
Windows/Linux) — they were `if: false`-gated already.
* Removed the five `flet_ffi_*` jobs and the FLET_GIT_REF/URL env
vars that supported their shallow-clone hack.
* Un-gated the five `bridge_example_*` jobs (15-job matrix:
macOS×3 × iOS×3 × Android×3 × Windows×3 × Linux ARM/AMD64 ×3).
* Added the unified ~/.flet/cache step to each bridge_example job
(replaces the lone dist_ios cache).
* All test invocations now pass --dart-define=EXPECTED_PYTHON_VERSION
so the interactivity test can assert against the embedded
interpreter's actual version.
* `publish` job's `needs:` list pruned to just the bridge_example_*
jobs.
bridge_example transport changes:
Two independent PythonBridge channels keep the perf/memory test's
hot path free of any framing tax while interactivity stays
self-describing:
* **control** (BRIDGE_EXAMPLE_CONTROL_PORT): UTF-8 JSON frames.
Dart sends {"op": "inc"|"dec"|"version"|"mem"}. Python responds
with {"event": "count"|"version"|"mem", ...}. Used by the
interactivity test and the memory test's snapshot probes.
* **echo** (BRIDGE_EXAMPLE_ECHO_PORT): pure raw bytes, no framing.
Python's handler is a one-liner that echoes the payload verbatim.
Used by the throughput test and the memory test's hammer loop.
bridge_example tests (replace the single bridge_echo_test.dart):
* `interactivity_test.dart`: taps `Key('increment')` /
`Key('decrement')` IconButtons, asserts the counter Text updates
(1, then -1 after two decrements). When
--dart-define=EXPECTED_PYTHON_VERSION is set, asserts the version
Text matches.
* `throughput_test.dart`: 100-iter sweep across 1 KB → 16 MB
payloads. Logs min/p50/p95/mean per size + MB/s throughput.
Asserts a 50 MB/s floor at ≥1 MB to catch order-of-magnitude
regressions without flaking on slow Windows Debug runs.
Measured on M2 Pro macOS, mean throughput:
1 KB → 23 MB/s (call-overhead-bound below ~64 KB)
64 KB → 1.3 GB/s
256 KB → 2.4 GB/s
1 MB → 4.5 GB/s
4 MB → 5.2 GB/s
16 MB → 7.2 GB/s
* `memory_test.dart`: 1000 × 1 MB echo round-trips (~2 GB of bytes
moved). Snapshots Python's tracemalloc + RSS before/after via
the control channel. Asserts `traced_delta < 5 MB`.
Measured on M2 Pro macOS:
traced_delta = 0 B (Python heap unchanged across 2 GB throughput)
rss_delta = 112 MB (OS-level page residency, expected; not retention)
bridge_example Dart side (lib/main.dart):
* Two PythonBridges created in main() before runApp() so the
testable BridgeExampleHandle is available immediately.
* Counter UI (decrement / display / increment) plus a "Python
version: …" Text — same shape the old flet_example test
asserted against, so the interactivity test pattern is familiar.
* BridgeExampleHandle singleton exposes both bridges +
ValueNotifiers for counter / version so tests can drive the
transport without traversing the widget tree.
bridge_example Python side (app/src/main.py):
* Two handlers wired to the two ports: JSON dispatcher for
control, verbatim echo for the bulk channel.
* _rss_bytes() uses resource.getrusage on POSIX (macOS / Linux /
Android / iOS) and falls back to ctypes →
GetProcessMemoryInfo.WorkingSetSize on Windows.
* tracemalloc is started lazily on the first 'mem' op so it
doesn't perturb the throughput test.
Renamed flet_example internals updated:
* pubspec.yaml: name flet_ffi_example → flet_example.
* integration_test import path matches the rename.
* README rewritten without the "compared to legacy flet_example"
framing — there is no other one to compare to now.
* Python boot log filename: flet_ffi_boot.log → flet_boot.log.
* Android Kotlin package, iOS bundle id, etc. left at their old
com.example.fletExample values to avoid Pods churn.
Replaces the old single-bridge echo description with:
* The actual current shape — two PythonBridge channels (JSON control +
raw echo) and what each is used for.
* Doc-grade descriptions of the three integration tests so a reader
knows what each covers before opening the .dart file.
* The 2026-06-12 macOS / M2 Pro perf and memory baseline (throughput
table, memory deltas, narrative on what the numbers mean).
Keeping perf numbers next to the tests that produce them means there's
one place to look when the next libdart_bridge or serious-python bump
changes the baseline.
CI run 27440632153 caught this: `flutter test integration_test` over the whole directory passes the first file then trips "Error waiting for a debug connection: The log reader stopped unexpectedly, or never started" / "Unable to start the app on the device" on every subsequent file. Local runs worked because I'd been invoking each file separately by habit. In CI we relied on the directory-wide form, which reuses one Dart VM session across files — bridge_example's per-test `SeriousPython.run()` + two `PythonBridge` instance creation + Python process spawn isn't a clean enough teardown for the next file's `runApp()` to take over the VM. Spawning a fresh `flutter test` process per file sidesteps the issue entirely. Applied uniformly to all five matrices: macOS, iOS, Android, Windows, Linux. Each now runs three explicit invocations: interactivity_test, throughput_test, memory_test. Affected runs: macOS×3, iOS×3, Android×3, Linux ARM×3 (and likely Linux AMD64 / Windows on a longer iteration count — the failure order was platform-dependent based on file-traversal order).
Run 27441055242 had 3-of-6 Linux jobs fail with "xvfb-run: error: Xvfb failed to start" on the second or third flutter test invocation. The pattern was randomly distributed across python versions (3.12, 3.13, 3.14) and arches (AMD64, ARM64) — the signature of a display-number race when xvfb-run is called back to back: the previous Xvfb hadn't fully released its display before the next xvfb-run tried to claim one. Start one Xvfb on :99 at the top of the test step (with a kill-on-EXIT trap), then run all three test files against the same DISPLAY. Also bumped the post-startup settle from "none" to a fixed 1-second sleep before the first flutter test runs (no xdpyinfo on the runner to do a proper readiness check, and 1s is plenty for Xvfb to start listening).
Add comprehensive documentation (docs/dedicated-data-channels.md) for the new DataChannel feature implemented in Flet 0.86.0. The doc explains the problem, architecture, public Python/Dart APIs, cross-mode transport choices, muxed wire format, backpressure pattern, concurrency considerations, empirical performance baselines, and known limitations, and links to implementation files and examples.
Run 27456825168 emitted Node 20 warnings for two actions: - gradle/actions/setup-gradle@v3: bumped to @v5. v5 is the last fully MIT-licensed major (v6 split the caching code into a separate gradle-actions-caching component under proprietary Terms of Use) and is on Node 24. The bump should also resolve the 'Cache service responded with 400' restore failures, which were the v3 caching code hitting the new GitHub Actions Cache v2 backend. - futureware-tech/simulator-action@v5: upstream still ships node20 in action.yml and there is no Node 24 release. Set the workflow-level FORCE_JAVASCRIPT_ACTIONS_TO_NODE24 env to make the runner execute node20-declared JS actions on Node 24 (documented opt-in from the Node 20 deprecation notice). Leaving the windows-latest NOTICEs in place — those are informational (latest is moving to windows-2025-vs2026 on 2026-06-15, no action needed).
CI was emitting `windows-latest requests are being redirected to windows-2025-vs2026 by June 15, 2026` NOTICE on every Windows job. Pin explicitly to the destination image instead of relying on the alias so the runner image can't shift under us silently.
`getPlatformVersion` was the example method from `flutter create --template=plugin` — a sanity-check that returns a string like "macOS 14.5.0". Nothing in this workspace nor flet itself ever called `SeriousPython.getPlatformVersion()`; the only consumers were the scaffold-generated plugin tests for Linux and Windows. Drop it everywhere before the dart-bridge release. Touches all six packages: - Top-level facade + platform-interface + method-channel impl: drop the declaration / forwarding wrapper. - Linux & Windows Dart impls: drop the override and the now-unused `methodChannel` field; trim the corresponding flutter/services and flutter/foundation imports. - Linux native plugin (C): drop the `get_platform_version()` function, collapse the method-call handler to a `NotImplemented` stub, and remove the now-orphan `<sys/utsname.h>`, `<cstring>`, and `_plugin_private.h` includes. Delete `_plugin_private.h` (only ever declared this function) and `linux/test/` (only tested this function; not referenced by CMakeLists). - Windows native plugin (C++): same shape — drop the case, the `<VersionHelpers.h>` + `<sstream>` includes, and the entire `windows/test/` scaffold. - Darwin Swift: drop the `getPlatformVersion` case; the `getResourcePath` case stays. Tighten the file docstring accordingly. - Android Java: drop the `getPlatformVersion` if-arm; `getAppVersion` / `getNativeLibraryDir` stay. flutter analyze: clean across all six packages (the 6 remaining warnings are pre-existing path-dep / missing-asset notes from local dev setup, unrelated to this change). Technically a public API break — `SeriousPython.getPlatformVersion()` is part of the published Dart API — but folded in here with the dart-bridge wire-format break already in flight.
Replace the ~6 independently hardcoded copies of the Python/Pyodide/ dart_bridge versions with a single source of truth: python-build's date-keyed manifest.json. serious_python pins one release date and commits generated snapshots of it; native builds read the committed snapshots offline, and CI guards against drift. - bin/gen_version_tables.dart: fetch python-build's manifest for the pinned pythonReleaseDate (or --release-date / --manifest) and write the committed snapshots: lib/src/python_versions.dart + a python_versions.properties in each native package. - lib/src/python_versions.dart (generated): the canonical table, moved out of package_command.dart; imported by package/version/configure commands. - Native configs (Android build.gradle, Darwin podspec, Linux/Windows CMakeLists) read the committed table with one layered resolution: SERIOUS_PYTHON_VERSION -> table -> per-field escape hatch -> error on unknown. - prepare_ios.sh/prepare_macos.sh take dart_bridge_version as a 4th arg from the table (drift unified to 1.2.3) instead of a hardcoded default. - New `configure` command stages the embedded Darwin interpreter for the selected version; `package` calls it so a version switch takes effect on a bare rebuild (no `rm -rf Pods`). - CI: "Version tables in sync with manifest" drift-check; drop the now- redundant resolve-python-vars action; key the flet cache on the snapshot. - Tests: run_example version_test (numpy ABI canary + version assertion via a main.py probe) and a gated python-version-switching workflow (sequential 3.12->3.13->3.14->3.12 on one VM, macOS + iOS). - Docs: CONTRIBUTING "Python runtime versions" bump flow; README manifest model + `configure`.
Bump the pinned Flutter to 3.44.2 (.fvmrc) and address the breaking changes: - Regenerate all four example apps (bridge/flask/run/flet) to the genuine 3.44.2 template across android/ios/macos/linux/windows: Kotlin DSL Gradle (build.gradle.kts/settings.gradle.kts) with built-in kotlin-android, AGP 8.x, SceneDelegate, the new linux/runner layout, and regenerated Podfiles. - serious_python_android: AGP 7.3.0 -> 8.11.1, gradle-download-task 4.1.2 -> 5.6.0, compileSdk 31 -> 36, Java 8 -> 17. - Replace the removed android.bundle.enableUncompressedNativeLibs with useLegacyPackaging + keepDebugSymbols; drop the NDK pin (use flutter.ndkVersion). - Podfiles: restore use_modular_headers! (required for the static serious_python_darwin framework to link); app-sandbox disabled in profiles. - prepare_ios.sh/prepare_macos.sh: version-aware dist extraction guard (a stale dist_* from a previous Python version caused C-extension ABI "unknown slot ID" errors); re-extract when the version marker mismatches. - Update example READMEs (package commands, Kotlin-DSL packaging guidance); ignore .fvm/.
Switching the bundled Python version between builds is a rare scenario, and a clean rebuild is the natural, expected way to handle it (flet build now wipes its build dir on a version change). The dedicated machinery to make a bare in-place switch take effect without a clean was disproportionate and couldn't fully cover the downstream Xcode/CocoaPods incremental bundle copy anyway. Remove: - the `configure` command and `stageDarwinRuntime` (+ its call in `package`) - the run_example version_test + its main.py version probe - the gated python-version-switching workflow - the `configure` docs Kept (these are anti-drift / correctness, not switching): the manifest-driven single-source version resolution, and the version-aware `rm -rf dist` marker guard in prepare_ios.sh/prepare_macos.sh — that guard is what makes a clean build pick up a new version, since the embedded dist lives in the shared pub-cache and a build-dir clean doesn't touch it.
Bump all six federated packages 2.0.0 -> 3.0.0 (pubspecs + darwin podspec + android build.gradle) and add 3.0.0 CHANGELOG sections. 3.0.0 highlights since 2.0.0: - In-process Python via the dart_bridge FFI transport (PythonBridge); the Python lifecycle is absorbed into the dart_bridge native library on every platform. - Requires Flutter 3.44.2 (Android: AGP 8.11.1 / compileSdk 36 / Java 17). - Python versions resolved from a committed snapshot of python-build's date-keyed manifest.json; SERIOUS_PYTHON_VERSION is the single knob. - Add `serious_python:main version [--json]`; remove the scaffold getPlatformVersion. Also restores each ## 2.0.0 section to exactly what the v2.0.0 tag published (the post-tag date-keyed-scheme bullets move into 3.0.0).
* DartBridge gains `isPythonInitialized` (true on Android process reuse where the OS kept this OS process alive across a Dart VM restart) and `signalDartSession(Map<String, int>)` (fires Python's registered session-restart handlers with the new native port numbers). * Both use a soft `lookupOrNull` against the underlying C symbols, so pre-1.3.0 libdart_bridge binaries still load cleanly — calls into the new wrappers become safe no-ops (`isPythonInitialized` returns false, `signalDartSession` is dropped on the floor). Decouples the Dart/Python rollout from the libdart_bridge release cadence. * Re-export `DartBridge` from `package:serious_python/bridge.dart` so embedders (notably the `flet build` template's `native_runtime.dart`) don't need a direct dependency on `serious_python_platform_interface`. Consumer wiring lands in flet's build template + flet.app runtime; this commit is just the lower-level FFI surface.
Regenerated via `dart run serious_python:gen_version_tables` after the python-build manifest at release 20260614 was updated to point at libdart_bridge 1.3.0 (https://github.com/flet-dev/dart-bridge/releases/tag/v1.3.0). This activates the Dart/Python FFI bindings + flet build template changes already on this branch — Android process-reuse and native stdout/stderr → logcat redirection now wire up end-to-end on the next Android build. No code changes here; just the regenerated version tables.
Regenerated via `dart run serious_python:gen_version_tables` after the python-build manifest at release 20260614 was updated to point at libdart_bridge 1.3.1 (https://github.com/flet-dev/dart-bridge/releases/tag/v1.3.1). 1.3.1 fixes: * `__android_log_write` is now resolved via explicit `-llog` linkage, unblocking Android Python 3.12 (whose libpython doesn't transitively pull in liblog like 3.13/3.14 do). Should unstick the previously failing bridge_example test on the Android-Py3.12 CI matrix entry. * `print(x)` no longer produces blank logcat / os_log entries after every real line — pure-whitespace writes that come from CPython's trailing-newline sub-call are now skipped on the native log branches. Desktop passthrough is unchanged. No code changes here; just the regenerated version tables.
Re-published Android binaries with explicit liblog linkage. 1.3.1's CMakeLists.txt fix was a no-op on Android (that platform builds via a hand-rolled clang line in the dart-bridge CI workflow, not CMake), so the shipped Android Py 3.12 binary still missed liblog and crashed at app start. 1.3.2's workflow adds -llog to the cross-compile and re-releases all platform binaries. Linux / Windows / Apple binaries are content-identical to 1.3.1. Regenerated via dart run serious_python:gen_version_tables. Release: https://github.com/flet-dev/dart-bridge/releases/tag/v1.3.2
…egacyPackaging) (#210) * android: migrate plugin build.gradle to Kotlin DSL Faithful, behavior-preserving port of the per-ABI task graph (gradle-download-task, tarTree/untar, copyOpt, zipSitePackages, dart-bridge download+rename, packaging, abiFilters) to build.gradle.kts. Verified green: run_example builds and runs on the arm64 emulator with identical jniLibs and Python output (numpy works). Prerequisite for the native-split tasks. * android: add _sp_bootstrap.py native-module finder (Phase E) sys.meta_path finder that resolves relocated CPython extension modules from their .soref markers (jniLibs lib loaded by basename). Reads markers via frozen zipimport.get_data (zip-resident) or open() (extracted), imports only builtin/frozen machinery so it runs before any native is resolvable. Host-tested: zip + extracted-dir probes, idempotent install, pure imports fall through. Not yet wired into the build. * android: split native modules to jniLibs, pure code to stored zips (Phase A/B) The Kotlin split tasks replace zipSitePackages: they relocate every tagged CPython extension .so (stdlib lib-dynload + site-packages) to jniLibs/<abi>/lib<mangled>.so (dotted name, '.'->'-', readable & injective), leaving a .soref marker (content = lib name) at the module's path in ABI-common stored zips. stdlib bundle is cracked post-untar; libpythonbundle.so/libpythonsitepackages.so no longer ship. Pure code -> stdlib.zip/sitepackages.zip (assets, noCompress); allowlisted packages (SERIOUS_PYTHON_ANDROID_EXTRACT_PACKAGES) -> extract.zip, excluded from sitepackages.zip. _sp_bootstrap.py embedded at stdlib.zip root. Verified by artifact inspection: 53 stdlib + 19 numpy natives mangled in jniLibs, markers present with matching libs, zips stored, no fake-zip .so. Runtime wiring (D/F) pending, so not yet runnable end-to-end. * android: runtime for native-mmap flow — first end-to-end green (Phase D) - AndroidPlugin.java: getFilesDir/extractAsset/unzipAsset (AssetManager + java.util.zip) to copy the stored zips to disk and unpack extract.zip. - serious_python_android.dart run(): copy stdlib.zip/sitepackages.zip once (version-keyed), unpack extract.zip; sys.path = [modulePaths, programDir, extract, sitepackages.zip, stdlib.zip]; PYTHONHOME=base. No fake-zip extraction. - _sp_bootstrap.py: resolve soname to an absolute origin under nativeLibraryDir when present (legacy packaging; CPython prepends ./ to a no-slash origin which breaks bare-soname load). - build.gradle.kts: inject interim sitecustomize.py (installs finder during site); declare split jniLibs/assets outputs + always-run so AGP's native-libs merge re-packages incrementally (no flutter clean needed). Verified on arm64 emulator: run_example runs fully — bz2, sqlite, and numpy all work, with pure code from stored zips (zipimport) and ALL native modules loaded via the finder from jniLibs. Legacy packaging still on (natives extracted); dropping it for mmap is next. * android: drop useLegacyPackaging — mmap natives from the APK (goal achieved) The finder now loads extension modules via Bionic's APK zip-path (base.apk!/lib/<abi>/<soname>) when libs aren't extracted (modern packaging). AndroidPlugin exports ANDROID_APK_NATIVE_PREFIX (sourceDir + abi); _sp_bootstrap prefers an extracted nativeLibraryDir copy (legacy) else the APK zip-path. The run_example app sets useLegacyPackaging=false. Proven on arm64 emulator: program runs fully (bz2, sqlite, numpy), ZERO native extraction (no .so in app storage or nativeLibraryDir), and /proc/<pid>/maps shows 32 r-xp executable mappings backed by base.apk — i.e. native modules are mmap'd directly from the APK. Pure code loads from stored zips via zipimport. * android: wire SERIOUS_PYTHON_DART_BRIDGE_DIST override; finder install via dart-bridge shim (F) - copyDartBridge_$abi uses a local dir of cross-compiled libdart_bridge-android-<abi>-py<ver>.so when SERIOUS_PYTHON_DART_BRIDGE_DIST is set (mirrors SERIOUS_PYTHON_BUILD_DIST), bypassing the GitHub download for dev iteration on the bridge. - Drop the interim sitecustomize injection: the dart-bridge Android shim now installs the finder before site (proven on emulator: green with finder installed solely by the shim). Re-enable the sitecustomize fallback for bridges without the shim (commented in the task). This branch now requires the dart-bridge android-native-mmap bridge (via the override until released). * android: bootstrap audit probe + clean stale ABI jniLibs dirs - _sp_bootstrap.install(): report any extension module already imported at install time (loaded during core-init before the finder) — verified empty on emulator, so PyConfig core-init pulls only builtin/frozen modules (F's pre-site install is safe). - cleanStaleAbis: remove jniLibs/<abi> dirs for ABIs not in the current set (e.g. stale armeabi-v7a 3.12 leftovers when building 3.14) so old fake-zip .so aren't packaged. Audits on arm64 emulator + AAB: importtime (no pre-finder natives), ABI-split (per-ABI natives, ~33MB pure payload shared ONCE in assets vs per-ABI duplication before), multi-ABI markers resolve in x86_64, allowlist partition+extract+import, 16KB-aligned. * Regenerate version tables from python-build 20260614 (dart_bridge 1.4.0) dart run serious_python:gen_version_tables --release-date 20260614 — the 20260614 manifest now pins dart_bridge_version 1.4.0 (the full-API/PyConfig Android bridge that installs the native-module finder before site). Python versions unchanged. Default build now downloads the released v1.4.0 bridge; verified green on the arm64 emulator (sqlite, numpy) with no env overrides. SERIOUS_PYTHON_DART_BRIDGE_DIST remains as a dev escape hatch. * docs: drop useLegacyPackaging consumer instructions; document native-mmap (CHANGELOGs) - run_example/flask_example READMEs: remove the useLegacyPackaging/keepDebugSymbols packaging block — consumers need no special native packaging now, just minSdk 23+. Document SERIOUS_PYTHON_ANDROID_EXTRACT_PACKAGES for path-hungry packages. - run_example app build.gradle.kts: drop the packaging block entirely (modern packaging is AGP's default at minSdk 23+). Verified green on emulator with natives still mmap'd from base.apk (32 r-xp maps), zero extraction. - serious_python + serious_python_android CHANGELOGs (3.0.0): replace the useLegacyPackaging language with the native-mmap design; bump bundled dart_bridge to 1.4.0. * flask_example: add INTERNET permission to main manifest (fix release builds) The Flask app binds a local socket server, so it needs android.permission.INTERNET in all build types. It was only in the debug/profile manifests, so release builds hit 'PermissionError: [Errno 1] Operation not permitted' on socket() (the app isn't in the inet group). Verified: release APK now grants INTERNET and the Flask server starts (Running on http://127.0.0.1:55001) — flask/werkzeug from the zip, _socket via the finder. * android: split-aware APK native prefix (Play Store AAB) + drop dead getNativeLibraryDir - ANDROID_APK_NATIVE_PREFIX now points at whichever installed APK actually contains lib/<abi>/ — base.apk for single-APK builds, the per-ABI config split for Play Store AAB installs (where base.apk has no native libs). Detected by probing each of sourceDir + splitSourceDirs for the always-present libdart_bridge.so. - Remove the unused getNativeLibraryDir method-channel handler (run() uses getFilesDir). Verified with bundletool: built the AAB, build-apks + install-apks the device splits (base.apk + split_config.arm64_v8a.apk), ran green (numpy). adb-root /proc/<pid>/maps shows 33 r-xp mappings from split_config.arm64_v8a.apk, 0 from base.apk, 0 extracted — natives mmap'd directly from the config split. * docs(readme): add 'How packaging works' section (per-platform layout) Document, per platform, where the stdlib and site-packages live, how native extension modules are shipped (Android: jniLibs mmap'd from the APK via a custom importer; iOS: xcframeworks + AppleFrameworkLoader; macOS: universal .so; Linux/Windows: on-disk; Web: Pyodide wheels), architecture handling, and how the user's app.zip is produced and extracted at runtime. Replace the stale Android note (enableUncompressedNativeLibs / extractNativeLibs) with the new no-config guidance (minSdk 23+).
Flutter no longer produces an x86 ABI on Android, so every x86 reference is dead. Builds target arm64-v8a + x86_64 (plus armeabi-v7a on Python 3.12); x86_64 (emulator / Intel macOS) and armeabi-v7a are untouched. - package_command.dart: drop the "x86" wheel platform-tag map entry and the `|| arch.key == "x86"` arm of the 32-bit skip condition. - serious_python_android/build.gradle.kts: remove the whole vestigial keepDebugSymbols block. It was a leftover from the old fake-.so-zip scheme: this com.android.library only packages the downloaded libdart_bridge.so (the libpython*.so patterns match nothing it owns), the real native modules land in the consuming app's jniLibs and are real ELF that mmap fine when stripped, and the README/CHANGELOG already state native-mmap needs no keepDebugSymbols. - flask_example: drop the stale useLegacyPackaging + keepDebugSymbols block (the only example still on the old scheme); minSdk 24 → modern packaging applies. Verified: built + ran on an arm64 device, Flask serves from Python with native deps memory-mapped from the APK, APK ships arm64-v8a + x86_64 only. Also syncs flask_example's pubspec.lock to the 3.0.0 path-dep versions.
- Fix the bundled dart_bridge version in the darwin/linux/windows CHANGELOGs: they said 1.2.3, but all three python_versions.properties pin 1.4.0 (the android CHANGELOG was already correct). - Document the x86 (32-bit Intel) Android ABI removal in the core CHANGELOG. - Document the removal of the `configure` command / bare in-place version-switching machinery (a clean rebuild now handles version switches) as a breaking change in the core CHANGELOG.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
serious_python 3.0.0
A major release headlined by an in-process Python runtime over the
dart_bridgeFFI transport, plus a Flutter 3.44.2 toolchain bump and manifest-driven version resolution.In-process Python (dart_bridge FFI)
SeriousPython.runruns the embedded interpreter in-process through thedart_bridgebridge instead of a socket transport. The Python lifecycle (init / run / teardown) is absorbed into thedart_bridgenative library on every platform —dart_bridge.xcframework(iOS/macOS),libdart_bridge.so(Android/Linux),dart_bridge.dll/dart_bridge.pyd(Windows).PythonBridgeAPI: a MsgPack control channel plus dedicated binary data channels between Dart and Python (see thebridge_exampleapp).dart_bridge1.2.3.Flutter 3.44.2
compileSdk36, Java 17, Kotlin-DSL Gradle;useLegacyPackaging+keepDebugSymbolsreplace the removedandroid.bundle.enableUncompressedNativeLibs.Manifest-driven versions
flet-dev/python-build's date-keyedmanifest.json(generated bydart run serious_python:gen_version_tables).SERIOUS_PYTHON_VERSIONis the single knob; full version / build date / Pyodide / dart_bridge derive from it (SERIOUS_PYTHON_FULL_VERSION/SERIOUS_PYTHON_BUILD_DATE/DART_BRIDGE_VERSIONremain escape hatches).python_versions.properties; a CI drift-check keeps the snapshots in sync.20260614), Pyodide 0.27.7 / 0.29.4 / 314.0.0.Also
dart run serious_python:main version [--json].~/.flet/cacheon all platforms.getPlatformVersionfrom the platform plugins.pyemscripten-2025.0-wasm32.Release
Bumps all six federated packages 2.0.0 → 3.0.0 with 3.0.0 CHANGELOG sections. Each
## 2.0.0section is restored to exactly what thev2.0.0tag published. Not tagged/published — review only.