QVAC：跨平台本地 AI SDK

Open-source local AI SDK - run AI on-device with no cloud, no API keys. Supports GGUF, RAG, image, music, and video generation, speech-to…

Local AI – SDK & Model Provider Run LLMs, speech, vision, image/video generation, and more on any device. Build mobile and desktop apps, or serve local models to your favorite AI tools. Website • Docs QVAC lets you run a comprehensive range of AI workloads locally using open models across Linux, macOS, Windows, Android, and iOS. QVAC provides: SDK for building local-first AI applications and systems in JavaScript/TypeScript and Python. HTTP server for using QVAC as a local model provider. Its OpenAI-compatible API lets you connect AI tools such as OpenCode and OpenClaw, or any other compatible tool. Why QVAC Local-first: run AI offline with inference optimized for commodity hardware, from consumer apps and embedded systems to on-premises deployments. Privacy and control: keep data local, own the AI system. No cloud or third-party APIs required. One SDK, all of AI: a comprehensive range of AI capabilities through one interface. Cross-platform: one codebase for Linux, macOS, Windows, Android, and iOS, using JavaScript/TypeScript or Python. Peer-to-peer: fetch AI models directly between peers and build unstoppable internet systems, like BitTorrent or IPFS, but for AI. Quickstart Load a model and run inference locally in a few steps. Pick your path. JavaScript Run your first example using the JS/TS SDK. Create the examples workspace: mkdir qvac-examples cd qvac-examples npm init -y && npm pkg set type=module Install the SDK: npm i @qvac/sdk Create qvac.config.json to enable client and server logs during the run: { "loggerConsoleOutput": true, "loggerLevel": "info" } Create the quickstart.js script: import { loadModel, LLAMA_3_2_1B_INST_Q4_0, completion, unloadModel } from '@qvac/sdk'; try { const modelId = await loadModel({ modelSrc: LLAMA_3_2_1B_INST_Q4_0, onProgress: (p) => { const mb = (n) => (n / 1e6).toFixed(1); const line = `▸ Downloading ${p.percentage.toFixed(0)}% (${mb(p.downloaded)}/${mb(p.total)} MB)`; process.stderr.write(process.stderr.isTTY ? `\r${line}` : `${line}\n`); if (p.percentage >= 100) process.stderr.write('\n'); }, }); const history = [{ role: 'user', content: 'Explain quantum computing in one sentence' }]; const result = completion({ modelId, history, stream: true }); for await (const token of result.tokenStream) { process.stdout.write(token); } await unloadModel({ modelId }); } catch (error) { console.error('✖', error); process.exit(1); } Run the quickstart script: QVAC_CONFIG_PATH=./qvac.config.json node quickstart.js You'll see the model download first. Then QVAC will stream the response tokens and print them to the terminal. Python Run your first example using the Python SDK. Create the examples workspace: mkdir qvac-examples-py cd qvac-examples-py python -m venv .venv source .venv/bin/activate Install the package (self-contained — bundles the QVAC worker and Bare runtime, no Node.js required): # Replace <version> with the release you want, e.g. sdk-v0.17.0: pip install tetherto-qvac-sdk \ -f https://github.com/tetherto/qvac/releases/expanded_assets/sdk-v<version> Create the quickstart.py script: import asyncio import sys from tetherto.qvac_sdk import Client, completion, load_model, unload_model from tetherto.qvac_sdk.models import LLAMA_3_2_1B_INST_Q4_0 def print_progress(p): line = f"▸ Downloading {p.percentage:.0f}% ({p.downloaded / 1e6:.1f}/{p.total / 1e6:.1f} MB)" print(line, end="\r" if sys.stderr.isatty() else "\n", file=sys.stderr) if p.percentage >= 100: print(file=sys.stderr) async def main(): async with Client() as client: t = client.transport try: model_id = await load_model( t, model_src=LLAMA_3_2_1B_INST_Q4_0, on_progress=print_progress ) run = completion( t, model_id=model_id, history=[ {"role": "user", "content": "Explain quantum computing in one sentence"}, ], ) async for event in run.events: if event.type == "contentDelta": sys.stdout.write(event.text) sys.stdout.flush() print() await unload_model(t, model_id) except Exception as error: print(f"✖ {error}", file=sys.stderr) return 1 return 0 if __name__ == "__main__": sys.exit(asyncio.run(main())) Run the quickstart script: python quickstart.py You'll see the model download first. Then QVAC will stream the response tokens and print them to the terminal. HTTP server Launch the server with the CLI, then use QVAC as model provider for OpenAI-compatible tools like OpenCode and OpenClaw. Install the CLI globally (this also installs @qvac/sdk as a transitive dependency): npm install -g @qvac/cli Create the examples workspace: mkdir qvac-server cd qvac-server Create the qvac.config.json declaring one model to serve: { "serve": { "models": { "my-llm": { "model": "QWEN3_600M_INST_Q4", "default": true, "config": { "ctx_size": 8192 } } } } } Start the server (bound to 127.0.0.1:11434 by default): qvac serve openai The model downloads on first start and is preloaded into memory. You'll see progress in the server output. From another terminal, hit it with any OpenAI-compatible client. A minimal curl: curl http://localhost:11434/v1/chat/completions \ -H 'Content-Type: application/json' \ -d '{ "model": "my-llm", "messages": [{"role": "user", "content": "Explain quantum computing in one sentence"}] }' The response comes back as a single JSON payload with the model's answer. Add "stream": true to the body to get an SSE stream instead. Point your AI tool at the server: open its model provider settings and add a new OpenAI-compatible provider with base URL http://localhost:11434/v1, any string as the API key, and my-llm as the model name. [!IMPORTANT] Setup varies by tool, and we ship dedicated plugins for some of them (like OpenCode and OpenClaw) that run the server for you. See Connect AI tools to QVAC for details. ⭐ If QVAC saves you from shipping yet another cloud dependency, give it a star, it helps other developers find the project! AI capabilities Task Description Text generation LLM inference for text generation and chat via Fabric LLM. Text embeddings Vector embedding generation for semantic search, clust