feat(apple/flutter): 0.9.0 iOS/macOS integration, mobile API, deterministic dispose & context shift support - #109
Closed
sebasbad wants to merge 1 commit into
Closed
Conversation
sebasbad
force-pushed
the
feat/0.9.0-ios-macos-flutter-support
branch
from
July 31, 2026 23:17
ebc73bc to
060f171
Compare
This was referenced Jul 31, 2026
sebasbad
marked this pull request as draft
July 31, 2026 23:44
sebasbad
force-pushed
the
feat/0.9.0-ios-macos-flutter-support
branch
from
August 1, 2026 00:09
cd6d175 to
82ca8e1
Compare
sebasbad
marked this pull request as ready for review
August 1, 2026 00:09
…nistic dispose & context shift support Consolidate mobile & Apple ecosystem enhancements for llama_cpp_dart 0.9.0 release: - Platform Support: Declare Flutter ffiPlugin support for iOS and macOS in pubspec.yaml and ios/llama_cpp_dart.podspec. - Apple XCFramework: Build script fixes for self-contained dynamic Llama.xcframework (universal simulator arm64+x86_64, universal macos arm64+x86_64, executable name fix, bundle id normalization). - Mobile-First API: Added ContextParams.mobile preset, estimateVramBytes(), model.isDisposed, and engine worker chat cancel. - Resource Safety: Deterministic C++ VRAM teardown for model, context, and multimodal instances on isolate shutdown. - Reasoning Models: Exposed ContextShiftPolicy in EngineChat.generate() for auto context sliding on reasoning/thinking models.
sebasbad
force-pushed
the
feat/0.9.0-ios-macos-flutter-support
branch
from
August 1, 2026 00:17
82ca8e1 to
c82fd85
Compare
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Consolidates all fixes and enhancements required for seamless mobile & desktop (iOS/macOS) integration, resource safety, and long-reasoning model support in
llama_cpp_dart0.9.0.Key Changes
1. Platform & Plugin Declarations (
pubspec.yaml,ios/llama_cpp_dart.podspec)ffiPluginplatform configuration foriosandmacos.Llama.xcframeworkwith module definitions and pod target xcconfigs.2. Apple XCFramework Build Script (
tool/build_apple_xcframework.sh)Llama.xcframeworkwith universal iOS simulator (arm64+x86_64) and universal macOS (arm64+x86_64) slices.Llama.framework/Llama) and normalizedCFBundleIdentifierformat.3. Mobile-First API Extensions (
lib/src/)ContextParams.mobilepreset tuned for memory constraints on mobile devices.estimateVramBytes()helper toLlamaModel.isDisposedflag for status checks.EngineChat.4. Deterministic C++ VRAM Teardown (
lib/src/isolate/)LlamaModel,LlamaContext,MultiModalInstance) to prevent VRAM memory leaks.5. Context Shift Support for Reasoning Models (
lib/src/isolate/)shiftPolicyandshiftnamed parameters inEngineChat.generate()andGenerateChatCommand, enabling opt-in auto context sliding for models with high token output.6. Documentation (
example/README.md)engine_stream.dart,chat_two_turn.dart,version_info.dart).Verification & Testing
example/probes/engine_stream.dartandexample/probes/chat_two_turn.dartusing Qwen2.5-0.5B model; verified multi-turn conversation memory, Jinja template rendering, token streaming, and clean VRAM deallocation.