feat(tts): support multi-reference personalities - #11620
Open
localai-org-maint-bot wants to merge 1 commit into
Open
feat(tts): support multi-reference personalities#11620localai-org-maint-bot wants to merge 1 commit into
localai-org-maint-bot wants to merge 1 commit into
Conversation
Saved profiles previously resolved to one audio path and transcript, so cloning backends could not use several examples of one personality. Store ordered audio and transcript pairs while preserving the legacy first-reference fields. Fish Speech and audio.cpp receive all pairs, including on distributed workers. Other backends retain their single-reference behavior. Assisted-by: Codex:gpt-5
localai-org-maint-bot
force-pushed
the
cron/issue-11617-voice-personalities-20260820
branch
from
August 20, 2026 07:18
b15e9b2 to
3797753
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR lets one saved voice personality contain up to ten ordered audio and transcript pairs. It preserves the existing first-reference fields for clients and backends that only support one reference. Fish Speech and audio.cpp receive every pair, including through distributed workers.
The Voice Library UI can add and remove reference pairs. The REST and MCP creation shapes accept the same ordered list, and the documentation and Swagger output describe the new response metadata.
Closes #11617
Notes for Reviewers
The audio.cpp pin advances to upstream merge commit
3ce3d037, which adds themulti_reference_condoption. Single-reference profiles and non-Fish cloning backends keep their current behavior.Verification:
go test ./core/services/voiceprofilego test ./core/http/endpoints/localai -ginkgo.focus="Voice profile endpoints"go test ./core/services/nodes -ginkgo.focus="FileStagingClient TTS references"go test ./pkg/mcp/localaitools/...npm --prefix core/http/react-ui run buildnpm --prefix core/http/react-ui run lint:inline-stylespython3 -m py_compile backend/python/fish-speech/backend.pyThe full nodes suite was not used as the acceptance signal because its shared database fixture fails in this environment; the focused staging suite passes.
Signed commits
gpgis unavailable in this cron environment.)