A neighborhood developer, GnLOLot, has printed a 1B mannequin that runs totally on native {hardware}. The mannequin is MiniCPM5-1B-Claude-Opus-Fable5-Pondering, with GGUF builds for llama.cpp-compatible runtimes. It wants no API key and makes no cloud calls.
The Proposed Mannequin
The mannequin is constructed on openbmb/MiniCPM5-1B. That base is an actual, documented launch from OpenBMB. It’s a dense 1.08B-parameter mannequin utilizing a typical LlamaForCausalLM structure. It has 24 layers, grouped-query consideration, and a 131,072-token context size. OpenBMB experiences 1B-class open-source SOTA inside its personal comparability set.
The bottom already ships a local pondering template. Reasoning is toggled by way of enable_thinking, giving each a Assume and a No Assume mode. The by-product mannequin retains that template and MiniCPM5’s tool-call format.
On high of that base, the developer utilized a fine-tune. The cardboard states the mannequin is ‘additional fine-tuned on Fable 5 knowledge’ to enhance coding and instruction following. The GGUF card repeats this as ‘post-trained on Fable 5 knowledge.’
How it’s truly constructed
The described methodology will not be classical distillation. You don’t shrink the unique mannequin. As a substitute you generate many conversations with a trainer mannequin. You seize its replies and reasoning traces as textual content. You then supervised-fine-tune a smaller base mannequin on these traces.
This distinction is essential for accuracy. Classical distillation transfers sign from a trainer’s logits or weights. Nobody has entry to Claude’s weights or logits. So that is supervised fine-tuning on generated outputs, not weight-level distillation. OpenBMB’s personal base mannequin, against this, makes use of a documented On-Coverage Distillation stage between its personal trainer and pupil checkpoints.
The sensible impact is that the 1B mannequin learns to mimic response format and elegance. It doesn’t take up the trainer’s underlying functionality. A 1B parameter finances can not maintain frontier-scale reasoning.
The specs that take a look at
The context window is 128K tokens, inherited from the bottom config.json (131,072). The GGUF repository ships 4 quantizations. Q4_K_M is roughly 657MB and is labeled the smallest footprint. Q5_K_M is roughly 751MB. Q8_0 is roughly 1.1GB and is the maintainer’s advisable default. F16 is roughly 2.1GB.
The ‘657MB footprint’ is the smallest quant, not the default construct. The mannequin hundreds immediately in llama.cpp, Ollama, LM Studio, jan, and KoboldCpp.
Interactive: how the construct works
The explainer beneath walks the construct pipeline, the footprint tradeoffs, and the trustworthy cut up of what a fine-tune can and can’t carry over.
Learn how to run it
The GGUF card provides a one-line path by way of Ollama:
ollama run hf.co/GnLOLot/MiniCPM5-1B-Claude-Opus-Fable5-Pondering-GGUF:Q4_K_MThe identical repository paperwork llama.cpp, LM Studio, jan, and KoboldCpp. Really useful sampling for Assume mode is temperature=0.9, top_p=0.95. The mannequin might emit reasoning blocks earlier than the ultimate reply, which downstream apps can strip.
Key Takeaways
- The mannequin is a supervised fine-tune of OpenBMB’s MiniCPM5-1B on Claude Fable 5 traces, not a weight-level distillation.
- Actual specs: 128K context, GGUF quants from ~657MB (Q4_K_M) to ~2.1GB (F16), Q8_0 the advisable default.
- High-quality-tuning on outputs transfers format and elegance, not frontier reasoning or broad information.
- No benchmarks or coaching dataset are printed, so functionality claims are at the moment unverifiable.
- Apache-2.0 covers the bottom weights solely; coaching on Claude outputs raises a licensing query the cardboard leaves open.
Michal Sutter is a knowledge science skilled with a Grasp of Science in Information Science from the College of Padova. With a strong basis in statistical evaluation, machine studying, and knowledge engineering, Michal excels at reworking complicated datasets into actionable insights.
