Skip to content

Add Qwen3 Coder 480B MLX serving - #1

Open
mweingartner wants to merge 1 commit into
jrepp:mainfrom
mweingartner:add-qwen3-coder-480b-mlx
Open

Add Qwen3 Coder 480B MLX serving#1
mweingartner wants to merge 1 commit into
jrepp:mainfrom
mweingartner:add-qwen3-coder-480b-mlx

Conversation

@mweingartner

Copy link
Copy Markdown

Summary

  • add Qwen3-Coder-480B-A35B sidecars for MLX 4-bit and UD-Q4_K_XL GGUF
  • add MLX server routing through run-server.py and llama-swap model generation
  • extend model download support for Hugging Face snapshots and clean markdown/pre-commit lint issues

Validation

  • pre-commit run --all-files
  • python3 -m py_compile download_model.py route-config.py run-server.py lm_launcher/.py llama_swap/.py
  • python3 route-config.py --validate
  • python3 download_model.py --list
  • served local Qwen3-Coder-480B MLX snapshot on 127.0.0.1:8001 and verified /v1/chat/completions returned "ready"

Note: route-config validation still reports warnings for planned model artifacts that are not present locally.

@mweingartner
mweingartner force-pushed the add-qwen3-coder-480b-mlx branch from 3aff207 to 1247984 Compare May 24, 2026 23:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant