feat(flutter): 0.9.0 iOS, macOS, and Android support

[sebasbad]

Summary

Consolidates all fixes and enhancements required for seamless mobile & desktop (iOS, macOS, and Android NDK) integration, resource safety, and long-reasoning model support in llama_cpp_dart 0.9.0 into a single unified pull request.

---

Key Changes

1. Platform & Plugin Declarations (pubspec.yaml, ios/llama_cpp_dart.podspec, android/build.gradle)

• Added ffiPlugin platform configuration for ios, macos, and android.
• Added vendored Podspec configuration pointing to dynamic Llama.xcframework with module definitions and pod target xcconfigs.
• Added Android Gradle Plugin integration (android/build.gradle, android/src/main/AndroidManifest.xml) with AGP 8.0+ namespace resolution fallback.

2. Native C++ Binaries & Toolchains (android/src/main/jniLibs/, tool/build_apple_xcframework.sh)

• Android NDK Native C++ Binaries (arm64-v8a & x86_64 for emulators):
  • Compiled native shared object libraries (libllama.so, libggml-base.so, libggml-cpu.so, libmtmd.so, libggml.so) using Android NDK toolchain (API level 26 / Oreo floor).
  • Debug symbols stripped via llvm-strip to keep binary footprint lightweight.
• Apple XCFramework:
  • Packaging script produces a self-contained dynamic Llama.xcframework with universal iOS simulator (arm64 + x86_64) and universal macOS (arm64 + x86_64) slices.
  • Normalized executable binary naming (Llama.framework/Llama) and CFBundleIdentifier format.

3. Mobile-First API Extensions (lib/src/)

• Added ContextParams.mobile preset tuned for memory constraints on mobile devices.
• Added estimateVramBytes() helper to LlamaModel.
• Added isDisposed flag for status checks.
• Added support for cancelling ongoing generations in EngineChat.

4. Deterministic C++ VRAM Teardown (lib/src/isolate/)

• Ensures worker isolate shutdown explicitly disposes C++ native pointers (LlamaModel, LlamaContext, MultiModalInstance) to prevent memory/VRAM leaks.

5. Context Shift Support for Reasoning Models (lib/src/isolate/)

• Exposed shiftPolicy and shift named parameters in EngineChat.generate() and GenerateChatCommand, enabling opt-in auto context sliding for models with high token output.

6. Documentation (example/README.md)

• Added documentation for running probe CLI examples (engine_stream.dart, chat_two_turn.dart, version_info.dart).

---

Verification & Testing

• Android Physical Device & Emulator: Verified clean build, NDK library dynamic linking, and inference on Android API 26+.
• Physical iOS Device & macOS Release: Verified full build, dynamic framework embedding, and runtime execution in release Flutter apps.
• Probe Scripts Executed: Tested example/probes/engine_stream.dart and example/probes/chat_two_turn.dart using Qwen2.5-0.5B model; verified multi-turn conversation memory, Jinja template rendering, token streaming, and clean VRAM deallocation.

[netdur]

Thanks @sebasbad could you review the latest release (0.9.0-dev.12) and check whether it covers the features you requested?

Note the packaging landed differently from your PR: Android libraries are bundled through a native-assets build hook rather than checked-in jniLibs/ + Gradle, and Apple uses a SwiftPM manifest rather than a vendored podspec (I've opted out of CocoaPods). The API additions, ContextParams.mobile, estimateVramBytes(), isDisposed, chat-level shiftPolicy/shift, and deterministic native teardown are all in

Let me know if anything else is needed

[sebasbad]

Thanks @netdur! I tested upgrading to 0.9.0-dev.12 and confirmed all requested features (ContextParams.mobile, estimateVramBytes(), isDisposed, chat shift policy, and deterministic teardown) are cleanly included.

One quick note: 0.9.0-dev.12 has environment.sdk: ^3.10.0 in its pubspec.yaml. The current stable Flutter SDK release (Flutter 3.35 / Dart 3.9.x) fails dependency resolution when fetching 0.9.0-dev.12:

Because flutter_image_to_text_app depends on llama_cpp_dart >=0.9.0-dev.12 which requires SDK version ^3.10.0, version solving failed.

Would it be possible to adjust the SDK lower bound constraint in llama_cpp_dart (e.g. to >=3.5.0 <4.0.0 or >=3.9.0) in a minor dev patch so apps on stable Flutter toolchains can resolve it?

[sebasbad]

Thanks @netdur! We upgraded our local toolchain to Flutter stable / Dart 3.12.2 to support the native-assets prerelease. Testing integration now.

[sebasbad]

Closing in favor of draft PR #111.

Context:
PR #110 was opened against v0.9.0-dev.10. Upstream v0.9.0-dev.12 introduced SwiftPM (Package.swift), which caused build conflicts in downstream Flutter apps combining llama_cpp_dart with CocoaPods plugins (e.g. google_mobile_ads).

We synced with upstream/main and configured dual CocoaPods + SwiftPM support in #111 so both package managers resolve seamlessly.