MLX-Serve：在本机运行模型与编码助手

Native LLM inference server for Apple Silicon. OpenAI + Anthropic API compatible. No Python. Zig backend, Swift frontend macOS app with c…

MLX-Serve Run open models on your Mac: chat, coding agents, images, video, music, speech and 3D. One native app and one local server that speaks the OpenAI, Anthropic and Ollama APIs. It runs offline, needs no Python and no account, and it's MIT licensed. Download for Mac · Homebrew · iPhone · Linux · mlxserve.com · 简体中文 Install The app. Download MLX-Serve, drag it to Applications and open it. It needs macOS 26.2 or later on Apple Silicon, suggests a model that fits your Mac, and updates itself. Homebrew. brew tap ddalcu/mlx-serve https://github.com/ddalcu/mlx-serve brew trust ddalcu/mlx-serve brew install --cask mlx-serve # the app brew install mlx-serve # or only the command-line server The terminal, if you know Ollama: mlx-serve run gemma4 # downloads Gemma 4 E4B, serves it and chats right here mlx-serve pull gemma4:26b # download only, resumable mlx-serve serve # serve everything you downloaded; models load when asked for The server listens on http://localhost:11234. Point any OpenAI, Anthropic or Ollama client at it, or open that address in a browser for the built-in web console. iPhone MLX-Serve puts the same engine on your iPhone: chat with Gemma 4, clone your voice, and make pictures and music, all on the phone and offline. It needs an A17 Pro-class iPhone or newer with iOS 26. Linux and Windows Every release has Linux downloads for NVIDIA GPUs: x64 for RTX 20 to 50 cards, arm64 for DGX Spark, GH200 and Jetson Thor. On Windows with an NVIDIA card, one script builds it. Details are in docs/building.md. What it does Coding agents on a local model. mlx-serve launch claude starts Claude Code against your Mac, set up with the model's real context window. Codex, OpenCode, pi, Grok and others work the same way. Integrations Pictures, video, music, speech and 3D. FLUX.2, Qwen-Image, Krea, MOSS-Image, LTX-Video, MiniMax-H3, ACE-Step, YuE2, Qwen3-TTS with voice cloning, and Hunyuan3D, in the app or over HTTP. The app Transcripts and meeting notes. Speech to text in 30 languages, with word timings and who said what. The menu bar records a call and writes the notes. API Models bigger than your Mac. GLM-5.3-Flash, MiMo-V2.6-Flash and DeepSeek-V4.1-Flash run on a 128 GB Mac by reading their experts from the SSD. Two Macs on a Thunderbolt 5 cable can run one model together, and Macs on your network can use each other's models. CLI · FAQ An assistant that lives on your Mac. Chat with your documents, talk to it hands-free, give agents a sandboxed shell and MCP tools, or schedule tasks in plain English. The app The APIs your tools already use. OpenAI chat, completions and Responses, Anthropic Messages and Ollama, all on one port, with streaming, tool calls, JSON schemas, vision and reasoning. API What can my Mac run? The app suggests a first model for your Mac's memory: Memory Suggested model 16 GB Gemma 4 E4B 18–24 GB Gemma 4 12B 32 GB Qwen 3.8 27B 36 GB Qwen 3.8 27B 6-bit 48–64 GB Qwen 3.8 27B 8-bit 96 GB and up Qwen3.8-Flash-Next (125B, about 6B active per token) 128 GB Also GLM-5.3-Flash, MiMo-V2.6-Flash and DeepSeek-V4.1-Flash, streamed from the SSD 256 GB Also GLM-5.3 (744B), streamed from the SSD Thousands more run too: any MLX model with a supported architecture, and any GGUF. The tier list ranks the popular ones, and docs/models.md lists every architecture. Speed Decode speed in v26.10.3 on an M5 Ultra with 256 GB, on a code prompt, with speculative decoding on: Model Tokens per second Qwen3.6 35B-A3B, 4-bit 424 Qwen3.8 27B, 4-bit 242 Gemma 4 E4B, 4-bit 199 Gemma 4 26B-A4B, 4-bit 186 Qwen3.8-Flash-Next, 4.7 bits 175 On an M4 Max, the same models decode 10% to 51% faster than they did in v26.8. Every release is measured the same way in benchmarks.md; speculative decoding and tuning are in docs/performance.md. Docs Integrations: Claude Code, Codex, OpenCode, pi, Zed and more HTTP API: OpenAI, Anthropic, Ollama and the media endpoints CLI and server flags The Mac app, with the media tour Supported models Write a plugin without rebuilding the server: model, media, quant Performance and tuning Building from source and NVIDIA GPUs FAQ and the changelog Build from source You need Xcode 26.2 or later and Homebrew: git clone --recurse-submodules https://github.com/ddalcu/mlx-serve && cd mlx-serve ./app/build.sh The script fetches the pinned Zig, builds MLX and llama.cpp, and signs the app ad hoc. There's no Python anywhere in the build. Server-only builds are in docs/building.md. Community Questions, ideas and things you made: the community chat on X and GitHub issues. Builds and deep dives go out on YouTube and X. Thank you Thank you to the sponsors who back mlx-serve: @jcprichard, @skudinov, @davidfekke, @lojza3d, @cpko, @d-b, @alinselea, Johnny Dang and @R0xr1te. If it's useful to you, you can support it on GitHub Sponsors or Buy Me a Coffee. Nothing is paywalled: it's MIT now and stays MIT. Thank you also to everyone who sends code, reports and ideas. The contributors page shows the code side, and the changelog thanks people release by release. Acknowledgements mlx-serve builds on MLX, mlx-c, mlx-lm, llama.cpp, antirez/ds4, mlx-stream, Sushi, jinja.cpp, nlohmann/json, stb_image, libwebp, Hugging Face tokenizers and Zig. It runs the open models their authors published, and the app uses the Anthropic and MCP Swift SDKs. Licenses and required notices are in NOTICE. If we missed you, please open a PR. Star history If mlx-serve is useful to you, a star helps other people find it. License MIT, see LICENSE. Bundled third-party code keeps its own license: NOTICE lists it with the required attributions, and LICENSE-APACHE-2.0 has the Apache license text.