{"name":"deepseek-v3-jax","repo":"Vaibhaav-Tiwari/deepseek-v3-jax","url":"https://midorreal.com/project/vaibhaav-tiwari-deepseek-v3-jax","repository":"https://github.com/Vaibhaav-Tiwari/deepseek-v3-jax","description":"DeepSeek V3 (MLA, DeepSeekMoE, aux-loss-free balancing, MTP) in ~200 lines of pure JAX, trained on a MacBook Air.","facts":{"license":{"value":"MIT","source":"https://github.com/Vaibhaav-Tiwari/deepseek-v3-jax","checked":"21 Nov 2026"},"language":{"value":"Python","source":"https://github.com/Vaibhaav-Tiwari/deepseek-v3-jax","checked":"21 Nov 2026"},"homepage":{"value":"https://vaibhaav.in/writing/deepseek-v3-jax/","source":"https://vaibhaav.in/writing/deepseek-v3-jax/","checked":"21 Nov 2026"},"last_release":{"value":"v1.0, 14 Nov 2026","source":"https://github.com/Vaibhaav-Tiwari/deepseek-v3-jax/releases","checked":"21 Nov 2026"},"releases":{"value":"1","source":"https://github.com/Vaibhaav-Tiwari/deepseek-v3-jax/releases","checked":"19 Nov 2026"},"contributors":{"value":"3","source":"https://github.com/Vaibhaav-Tiwari/deepseek-v3-jax/graphs/contributors","checked":"19 Nov 2026"},"archived":{"value":"no","source":"https://github.com/Vaibhaav-Tiwari/deepseek-v3-jax","checked":"21 Nov 2026"}},"stars":260.0,"writeup":{"what_it_is":"A pure JAX implementation of the DeepSeek-V3 architecture in about 190 lines, including Multi-head Latent Attention, DeepSeekMoE with auxiliary-loss-free balancing, and multi-token prediction. It was pretrained from scratch on public-domain detective fiction on a MacBook Air.","audience":"Readers who want to study or learn from a small, readable DeepSeek-V3 implementation.","claims":[{"kind":"specific","claim":"Implements the architecture from the DeepSeek-V3 technical report in about 190 lines of pure JAX","excerpt":"The architecture from the [DeepSeek-V3 technical report]","status":"not_checked"},{"kind":"specific","claim":"Caches 144 numbers per token per layer instead of 480 with Multi-head Latent Attention","excerpt":"**480 → 144** numbers per token per layer here","status":"not_checked"},{"kind":"specific","claim":"Uses 1 shared expert and 16 routed experts, with top-4 routing per token","excerpt":"1 shared expert that always runs + 16 routed experts, top-4 per token","status":"not_checked"},{"kind":"specific","claim":"Has 36.2M parameters, 13.6M of them active per token","excerpt":"36.2M parameters, 13.6M active per token","status":"not_checked"},{"kind":"specific","claim":"Trained for 12,000 steps on a MacBook Air M5 with 24 GB of memory","excerpt":"12,000 steps, ~98M tokens (~4.5 epochs), ~2.6 h","status":"not_checked"}],"alternatives":[],"written_by":"AI, from the project's README","date":"2026-11-15"},"why_now":null,"owner_supplied":null,"score":{"verdict":"too_early","hype":94,"reality":36,"gap":58,"momentum":null,"confidence":60,"version":"score-2.0","rule":"verdict-2.0","calculated_at":"2026-11-22T07:00:00+00:00","verdict_source":"rule"},"early_read":{"estimate":true,"lean":"too_close","gap_low":25,"gap_high":67,"rule":"early-2.0"},"changes":[{"date":"2026-11-14T14:59:40+00:00","text":"Latest release v1.0","source":"https://github.com/Vaibhaav-Tiwari/deepseek-v3-jax/releases"},{"date":"2026-11-13T14:59:13+00:00","text":"Repository created","source":"https://github.com/Vaibhaav-Tiwari/deepseek-v3-jax"}],"cite":"deepseek-v3-jax is a GitHub project at https://github.com/Vaibhaav-Tiwari/deepseek-v3-jax written mainly in Python, described there as \"DeepSeek V3 (MLA, DeepSeekMoE, aux-loss-free balancing, MTP) in ~200 lines of pure JAX, trained on a MacBook Air\".","attribution":"Data from Mid or Real, https://midorreal.com/project/vaibhaav-tiwari-deepseek-v3-jax, checked 22 November 2026. Attribution with a link is required.","still_real":null}