Skip to content

fix: 0.9.0 Deterministic C++ VRAM Teardown on Worker Isolate Shutdown (Issue #5) - #7

Closed
sebasbad wants to merge 1 commit into
feat/0.9.0-mobile-first-apifrom
fix/0.9.0-deterministic-dispose
Closed

fix: 0.9.0 Deterministic C++ VRAM Teardown on Worker Isolate Shutdown (Issue #5)#7
sebasbad wants to merge 1 commit into
feat/0.9.0-mobile-first-apifrom
fix/0.9.0-deterministic-dispose

Conversation

@sebasbad

Copy link
Copy Markdown
Owner

Summary

Fixes Issue #5 on top of feat/0.9.0-mobile-first-api (PR #6):

  • Previously, runEngineWorker in lib/src/isolate/worker.dart was explicitly skipping calling context.dispose(), model.dispose(), and multimodal.dispose() during ShutdownCommand.
  • This left the native C++ llama_model* and llama_context* pointers and Metal GPU buffers in C++ heap memory when engine.dispose() was called, causing subsequent model load operations in long-running app processes (like mobile apps) to fail with failed to load model.
  • _shutdown now explicitly disposes context, model, and multimodal before completing ShutdownCommand.

Closes #5.

@sebasbad

Copy link
Copy Markdown
Owner Author

Superseded by upstream PR netdur#109

@sebasbad sebasbad closed this Jul 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant