Open source C/C++ engine for high-performance local LLM inference and on-device AI.
-
Updated
Jul 14, 2026 - C++
#
model-runtime
Here are 6 public repositories matching this topic...
Data-only launcher contracts and runtime-readiness surfaces for PCCX™ model integration.
python cli workflow runtime descriptor launcher inference compatibility diagnostics gemma readiness edge-ai kv260 llm model-runtime ai-accelerator evidence-gated pccx pccx-v002
-
Updated
May 31, 2026 - Python
A complete inference runtime for open-weight large language models, enabling efficient execution through streaming weights, quantization, and memory-aware scheduling
machine-learning deep-learning numpy inference quantization inference-engine memory-mapping inference-optimization kv-cache ai-inference cpu-inference large-language-models llm model-runtime safetensors llm-inference gguf streaming-inference gguf-runner streaming-inference-engine
-
Updated
Aug 3, 2026 - Python
GPT-OSS B20 Local Execution. Lightweight local environment for running it with Python 3.12 and CUDA acceleration. - Run GPT-OSS B20 entirely offline - Optimize text generation with GPU - Enable fast, secure inference on consumer hardware.
text-generation performance-optimization gpu-optimization edge-ai inference-acceleration secure-inference model-runtime minimal-setup llm-inference open-source-llm local-execution offline-inference privacy-preserving-ai consumer-gpu gpt-oss-b20 lightweight-environment
-
Updated
Aug 13, 2025 - Python
Portable, provider-neutral model invocation contracts, adapters, and conformance tools
-
Updated
Aug 2, 2026 - TypeScript
Windows-first local LLM runtime written in Go. Custom model loading, GGUF conversion, local chat serving, optional CUDA integration.
-
Updated
May 28, 2026 - Go
Improve this page
Add a description, image, and links to the model-runtime topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the model-runtime topic, visit your repo's landing page and select "manage topics."