LocalAI version:
4.8.2 (a633fae08dc3a1f0cdf75235b1a44be098235f52)
Environment, CPU architecture, OS, and Version:
Linux 7.1.8-arch1-3 #1 SMP PREEMPT_DYNAMIC Tue, 11 Aug 2026 09:16:08 +0000 x86_64 GNU/Linux
Describe the bug
When I start LocalAI, it downloads over a gigabyte of model gallery data/configs to support fast VRAM estimation. It does this every time the service is started, so if there's any caching supposed to happen here, it's not working. It also doesn't look like I can disable this behavior via config. As more models are added, this amount will keep increasing.
I was able to workaround this by only enabling my own custom galleries or disabling galleries entirely.
To Reproduce
Start a local-ai run service with default gallery config.
Expected behavior
LocalAI should cache the gallery data it downloads as best it can whenever possible.
Logs
I observed this when the download finished:
Aug 12 13:02:25 INFO gallery caches warmed estimates=299 variants=52 of=300 took=3m4s
Yes, my internet connection is pretty slow. LocalAI fully saturates it for just over 3 minutes and I have several instances and restart then a lot for testing. If I remove the built-in model and backend galleries this takes only a few seconds.
LocalAI version:
4.8.2 (a633fae08dc3a1f0cdf75235b1a44be098235f52)
Environment, CPU architecture, OS, and Version:
Linux 7.1.8-arch1-3 #1 SMP PREEMPT_DYNAMIC Tue, 11 Aug 2026 09:16:08 +0000 x86_64 GNU/Linux
Describe the bug
When I start LocalAI, it downloads over a gigabyte of model gallery data/configs to support fast VRAM estimation. It does this every time the service is started, so if there's any caching supposed to happen here, it's not working. It also doesn't look like I can disable this behavior via config. As more models are added, this amount will keep increasing.
I was able to workaround this by only enabling my own custom galleries or disabling galleries entirely.
To Reproduce
Start a
local-ai runservice with default gallery config.Expected behavior
LocalAI should cache the gallery data it downloads as best it can whenever possible.
Logs
I observed this when the download finished:
Yes, my internet connection is pretty slow. LocalAI fully saturates it for just over 3 minutes and I have several instances and restart then a lot for testing. If I remove the built-in model and backend galleries this takes only a few seconds.