Soup：简化大模型微调

Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.

🌍 English | Türkçe | العربية | 日本語 | 中文 Soup Fine-tune and post-train LLMs in one command. No SSH, no config hell. Website · Quick Start · Web UI · Config · Docs · Commands · Models · Discord · Telegram · Product Hunt Soup turns the pain of LLM fine-tuning into a simple workflow. One config, one command, done. pip install "soup-cli[train]" # add [train] to fine-tune; bare `soup-cli` is the light CLI soup init --template chat soup train Fine-tune an 8B model on a 4 GB laptop GPU. Layer streaming keeps the frozen base out of VRAM and feeds it to the GPU one decoder layer at a time. Measured on an RTX 3050 Laptop 4 GB: Llama-3.1-8B-Instruct + NF4 at 119.6 tok/s at 952 MHz, 3.32 GB peak (pre-repair) — bit-exact against a normal resident run, and reproduced independently on an H100 at 113.00 tok/s in the same 3.32 GB. (Both figures were measured on v0.72.2, before the v0.73.0 correctness repair that cost −4.8% at 32B. The post-repair run, on another RTX 3050 Laptop 4 GB (@umran666, #1103), reads 208.6 tok/s at 1935–1957 MHz in a 2.40 GB peak. That is 59% of its same-session GEMM ceiling against 68%: about 14% slower per unit clock, so the raw 1.74x gain is the clock, not a speedup. Large-layer streaming also changed the memory pattern between the two rows, so even the fraction is not like-for-like — the record.) Opt-in (stream_layers: true) and still BETA — how it works · all measurements · paper · check it yourself on a free Colab T4 (caps the process to 4 GB, then asserts a streamed model is bit-identical to a normal one) Llama-3.1-8B-Instruct + NF4, LoRA, batch 1, seq 512 on an RTX 3050 Laptop 4 GB — 3.32 GB peak, 119.6 tok/s at 952 MHz (pre-repair, v0.72.2; post-#331: 208.6 tok/s at 1935 MHz in 2.40 GB, not directly comparable, @umran666, #1103). Full video (90s) Why Soup? Training LLMs is still painful. Even experienced teams spend 30-50% of their time fighting infrastructure instead of improving models. Soup fixes that. Zero SSH. Never SSH into a broken GPU box again. One config. A simple YAML file is all you need. Auto everything. Batch size, GPU detection, quantization — handled. Works locally. Train on your own GPU with QLoRA. No cloud required. What's New v0.77.0 — what v0.76 warned about is now refused. 31 config fields that nothing reads, and three deprecated values, no longer load with a warning: the load stops with one line per setting that names it and says what to delete. A config that loaded on v0.76, with warnings or silently, can stop loading on v0.77: run soup doctor --config soup.yaml first. Breaking: staged fields and deprecated values are refused. Among the fields: training.lr_groups, early_stop_patience, forgetting_eval_steps, checkpoint_eval_*, quantize_ref_model, long_context_grpo, vision_grpo, data.split_thinking and the data.video_* keys. training.rl_checkpoint_include_ref_model and rl_checkpoint_include_rollout_buffer were staged after v0.76.0, so no release warned about them first. The values: data.remove_unused_columns: true, training.quantization: 4bit / 8bit (or load_in_8bit: true) on a task that never quantises, and soup ab --beta. Fourteen settings v0.76 ignored without a word are refused too, such as use_mod or neftune_alpha where nothing applies them, expand_layers with a LoRA rank, 8bit or vision SFT on Unsloth, and lora.rank_pattern on MLX or Unsloth: see the CHANGELOG upgrade notes. Breaking: Python 3.11–3.13, no AWQ or GPTQ export, a token for two soup serve routes. Python 3.10 is no longer supported (end of life 2026-10-01); on 3.10, pip keeps resolving v0.76.x. soup export --format awq / gptq and soup quantize --to awq / gptq are removed, because both upstream projects are archived: use gguf, onnx, tensorrt, bitnet or tq1_0, or stay on v0.76.x. /v1/tools/python and /v1/tools/bash answer 401 unless the server was started with --tool-auth-token (or SOUP_TOOL_AUTH_TOKEN), on every bind, and /v1/tools/python answers 501 where the host has no OS-level isolation, as bash already did. DPO, ORPO, SimPO and KTO run natively on Apple Silicon. task: dpo, orpo, simpo and kto with backend: mlx train through Soup's own MLX loop on TRL's objectives (ORPO and SimPO with no reference model). MLX text SFT gains native DoRA, training.eval_steps, training.seed and the loss watchdog. lora.use_rslora is now applied on MLX and Unsloth, so an rsLoRA run there trains at the scale it asked for, and its results change. Layer streaming reaches DeepSeek-V3-family MoE. Soup's layer streaming (an implementation of L2L, Pudipeddi et al. 2020) now streams model_type: deepseek_v3 with resident-vs-streamed parity forward and backward, and dequantises FP8 block-quantised checkpoints at shard time. Kimi-K2-Base and -Instruct are admitted behind a field-by-field check (a tiny K2-shaped checkpoint streams bit for bit on CPU; nothing has been trained on K2 itself). training.stream_expert_layout: per_expert streams one routed expert at a time: on one RTX 5070 Laptop 8 GB, the same Moonlight-16B LoRA config peaked at 1.0 GB of GPU memory against 3.1 GB fused (NF4 experts against bf16, different cache tiers), at about 2.7x the step time (record). training.stream_recompute: off / auto keeps streamed activations instead of recomputing them, with bit-identical gradients: on the same card's RAM tier a Mistral-7B NF4 step is 22% faster at 512 tokens and 8.5% at 2048 (record). Opt-in (stream_layers: true), BETA. Provenance you can check. soup registry push --data <file> records the training file's SHA-256, and --idempotency-key makes a push safe to retry; on the transformers backend, resident, layer-streamed and multi-GPU runs record the Hub commit of the base they loaded; soup ship --require-provenance checks the evidence against the adapter, task and source bytes, suite version and verdict policy it was stamped with. Also new. soup --version; pip install "soup-cli[documents]" for PDF and DOCX in soup data ingest; soup merge --low-memory; GRPO can score with an LLM judge (training.reward_fn: judge