Fish Speech backend using ROCM doesn't work

[ALameLlama]

LocalAI version:
localai/localai:master-gpu-hipblas
sha256:f3ff922777a09ec7f63093df7482ebb15d3f94c0f8611d49143fe455b4f7144d

Environment, CPU architecture, OS, and Version:
I'm doing this on unraid 7.3, using portainer.
AMD Ryzen 9 5900X
Radeon AI PRO R9700
Local AI is setup using distribution mode with nodes.

Describe the bug
I tried to use the fish speech backend and this is the error I get failed to route model with internal loader: routing model fishaudio/s2-pro: loading model fish-speech-s2-pro: loading model fishaudio/s2-pro on node rocinante-worker: Failed to load model: No module named 'fish_speech.inference_engine'

this is the docker log

Aug 18 10:47:58 DEBUG GRPC stderr id="fish-speech-s2-pro#0-0.0.0.0:50051" line="CUDA is not available" caller={caller.file="/build/pkg/model/process.go"  caller.L=305 } 
Aug 18 10:47:58 DEBUG GRPC stderr id="fish-speech-s2-pro#0-0.0.0.0:50051" line="Using device: cpu, precision: torch.float32, compile: False" caller={caller.file="/build/pkg/model/process.go"  caller.L=305 } 
Aug 18 10:47:58 DEBUG GRPC stderr id="fish-speech-s2-pro#0-0.0.0.0:50051" line="Loading model from: /models/fish-speech-s2-pro/.artifacts/huggingface/637e4c7713f137cd3cd5969badcf6fa2e2fedd6781c55e4fc5c1a586a64726a0/snapshot" caller={caller.file="/build/pkg/model/process.go"  caller.L=305 } 
Aug 18 10:47:58 DEBUG GRPC stderr id="fish-speech-s2-pro#0-0.0.0.0:50051" line="[ERROR] Loading model: ModuleNotFoundError: No module named 'fish_speech.inference_engine'" caller={caller.file="/build/pkg/model/process.go"  caller.L=305 } 
Aug 18 10:47:58 DEBUG GRPC stderr id="fish-speech-s2-pro#0-0.0.0.0:50051" line="Traceback (most recent call last):" caller={caller.file="/build/pkg/model/process.go"  caller.L=305 } 
Aug 18 10:47:58 DEBUG GRPC stderr id="fish-speech-s2-pro#0-0.0.0.0:50051" line="  File \"/backends/rocm-fish-speech/backend.py\", line 201, in LoadModel" caller={caller.file="/build/pkg/model/process.go"  caller.L=305 } 
Aug 18 10:47:58 DEBUG GRPC stderr id="fish-speech-s2-pro#0-0.0.0.0:50051" line="    from fish_speech.inference_engine import TTSInferenceEngine" caller={caller.file="/build/pkg/model/process.go"  caller.L=305 } 
Aug 18 10:47:58 DEBUG GRPC stderr id="fish-speech-s2-pro#0-0.0.0.0:50051" line="ModuleNotFoundError: No module named 'fish_speech.inference_engine'" caller={caller.file="/build/pkg/model/process.go"  caller.L=30

when I try and view the trace for details and click backend log it just has

Model not loaded on any worker

fish-speech-s2-pro isn't currently loaded on any node in the cluster. Check the Nodes page to see which models are running where.

To Reproduce
I try and create a TTS with the fish-speech backend, I get this error

Expected behavior
For it to work

Logs

Aug 18 10:47:58 DEBUG GRPC stderr id="fish-speech-s2-pro#0-0.0.0.0:50051" line="CUDA is not available" caller={caller.file="/build/pkg/model/process.go"  caller.L=305 } 
Aug 18 10:47:58 DEBUG GRPC stderr id="fish-speech-s2-pro#0-0.0.0.0:50051" line="Using device: cpu, precision: torch.float32, compile: False" caller={caller.file="/build/pkg/model/process.go"  caller.L=305 } 
Aug 18 10:47:58 DEBUG GRPC stderr id="fish-speech-s2-pro#0-0.0.0.0:50051" line="Loading model from: /models/fish-speech-s2-pro/.artifacts/huggingface/637e4c7713f137cd3cd5969badcf6fa2e2fedd6781c55e4fc5c1a586a64726a0/snapshot" caller={caller.file="/build/pkg/model/process.go"  caller.L=305 } 
Aug 18 10:47:58 DEBUG GRPC stderr id="fish-speech-s2-pro#0-0.0.0.0:50051" line="[ERROR] Loading model: ModuleNotFoundError: No module named 'fish_speech.inference_engine'" caller={caller.file="/build/pkg/model/process.go"  caller.L=305 } 
Aug 18 10:47:58 DEBUG GRPC stderr id="fish-speech-s2-pro#0-0.0.0.0:50051" line="Traceback (most recent call last):" caller={caller.file="/build/pkg/model/process.go"  caller.L=305 } 
Aug 18 10:47:58 DEBUG GRPC stderr id="fish-speech-s2-pro#0-0.0.0.0:50051" line="  File \"/backends/rocm-fish-speech/backend.py\", line 201, in LoadModel" caller={caller.file="/build/pkg/model/process.go"  caller.L=305 } 
Aug 18 10:47:58 DEBUG GRPC stderr id="fish-speech-s2-pro#0-0.0.0.0:50051" line="    from fish_speech.inference_engine import TTSInferenceEngine" caller={caller.file="/build/pkg/model/process.go"  caller.L=305 } 
Aug 18 10:47:58 DEBUG GRPC stderr id="fish-speech-s2-pro#0-0.0.0.0:50051" line="ModuleNotFoundError: No module named 'fish_speech.inference_engine'" caller={caller.file="/build/pkg/model/process.go"  caller.L=30

[localai-org-maint-bot]

I’m picking this up. I’ll reproduce the fish-speech editable-install relocation failure on current master, add regression coverage for runtime importability, and fix the packaging path without changing the backend’s CPU, CUDA, or ROCm dependency selection.