{"name":"llama-modchip","repo":"synchopate/llama-modchip","url":"https://midorreal.com/project/synchopate-llama-modchip","repository":"https://github.com/synchopate/llama-modchip","description":"Zero-GPU CPU inference for 30B-284B MoE models (Qwen3.8 176.5B cold-cache certified in 28.26/48.35 GiB Cgroup v2, DeepSeek-V4 284B, Nemotron-30B) + RouteCrack 2.0.","facts":{"license":{"value":"Apache-2.0","source":"https://github.com/synchopate/llama-modchip","checked":"22 Nov 2026"},"language":{"value":"Python","source":"https://github.com/synchopate/llama-modchip","checked":"22 Nov 2026"},"homepage":{"value":"https://synchopate.github.io/","source":"https://synchopate.github.io/","checked":"22 Nov 2026"},"last_release":{"value":"v3.1.1, 20 Nov 2026","source":"https://github.com/synchopate/llama-modchip/releases","checked":"22 Nov 2026"},"releases":{"value":"13","source":"https://github.com/synchopate/llama-modchip/releases","checked":"21 Nov 2026"},"contributors":{"value":"5","source":"https://github.com/synchopate/llama-modchip/graphs/contributors","checked":"21 Nov 2026"},"archived":{"value":"no","source":"https://github.com/synchopate/llama-modchip","checked":"22 Nov 2026"}},"stars":1401.0,"writeup":{"what_it_is":"llama-modchip is a set of ggml/llama.cpp kernel patches, quantization tools, and runtime utilities for running sparse Mixture-of-Experts language models in system RAM on CPU, without a GPU. The README reports results for a 284B DeepSeek-V4-Flash model and 30B Nemotron models.","audience":"Users running large MoE language models locally on CPU with 16–64 GB of RAM","claims":[{"kind":"specific","claim":"Runs a 284B DeepSeek-V4-Flash model in 64 GB RAM at 9.1–14.7 tok/s decode","excerpt":"fits in 62.5 GiB usable Linux RAM (57.61 GiB VmRSS) with zero NVMe swap","status":"not_checked"},{"kind":"specific","claim":"Achieves 44.36 tok/s prefill for the 284B model on a 16-core Zen 5 CPU","excerpt":"44.36 tok/s</b><br><sub>284B CPU prefill (16C Zen 5)","status":"not_checked"},{"kind":"specific","claim":"Restores KV cache slots from /dev/shm in about 13.1 ms","excerpt":"13.1 ms</b><br><sub>/dev/shm KV slot restore","status":"not_checked"},{"kind":"specific","claim":"Reduces the DeepSeek-V4-Flash checkpoint to 56.56 GiB at 1.71 bpw","excerpt":"reduces the checkpoint to `56.56 GiB` (`60.73 GB`)","status":"not_checked"},{"kind":"specific","claim":"Requires custom kernels, so unpatched llama.cpp builds reject the Fractal-64 models","excerpt":"Unpatched `llama.cpp`, Ollama, or LM Studio binaries will reject them with `unknown tensor type 40`","status":"not_checked"}],"alternatives":[],"written_by":"AI, from the project's README","date":"2026-11-09"},"why_now":null,"owner_supplied":null,"score":{"verdict":"real","hype":95,"reality":70,"gap":25,"momentum":null,"confidence":60,"version":"score-2.0","rule":"verdict-2.0","calculated_at":"2026-11-22T08:00:00+00:00","verdict_source":"rule"},"early_read":null,"changes":[{"date":"2026-11-20T06:38:03+00:00","text":"Latest release v3.1.1","source":"https://github.com/synchopate/llama-modchip/releases"},{"date":"2026-11-02T14:24:06+00:00","text":"First release","source":"https://github.com/synchopate/llama-modchip/releases"},{"date":"2026-10-29T10:17:33+00:00","text":"Repository created","source":"https://github.com/synchopate/llama-modchip"}],"cite":"llama-modchip is a GitHub project at https://github.com/synchopate/llama-modchip written mainly in Python, described there as \"Zero-GPU CPU inference for 30B-284B MoE models (Qwen3.8 176.5B cold-cache certified in 28.26/48.35 GiB Cgroup v2, DeepSeek-V4 284B, Nemotron-30B) + RouteCrack 2.0\".","attribution":"Data from Mid or Real, https://midorreal.com/project/synchopate-llama-modchip, checked 22 November 2026. Attribution with a link is required.","still_real":null}