{"name":"NaiveRT","repo":"NaiveAI-Labs/NaiveRT","url":"https://midorreal.com/project/naiveai-labs-naivert","repository":"https://github.com/NaiveAI-Labs/NaiveRT","description":"NaiveRT: a self-contained TP8 inference runtime for the Naive N series on eight Blackwell GPUs with UMMA support. One persistent CUDA graph per rank runs draft, eight-row verify, top-p sampling and cache commit with no host in the loop: 1,700 tokens/s at batch size 1, one-second peaks above 2,000.","facts":{"license":{"value":"Apache-2.0","source":"https://github.com/NaiveAI-Labs/NaiveRT","checked":"22 Nov 2026"},"language":{"value":"Python","source":"https://github.com/NaiveAI-Labs/NaiveRT","checked":"22 Nov 2026"},"releases":{"value":"0","source":"https://github.com/NaiveAI-Labs/NaiveRT/releases","checked":"21 Nov 2026"},"contributors":{"value":"1","source":"https://github.com/NaiveAI-Labs/NaiveRT/graphs/contributors","checked":"21 Nov 2026"},"archived":{"value":"no","source":"https://github.com/NaiveAI-Labs/NaiveRT","checked":"22 Nov 2026"}},"stars":177.0,"writeup":{"what_it_is":"NaiveRT is an inference runtime for the Naive-N0.5-Flash model on eight Blackwell GPUs. It uses CUDA kernels, speculative decoding, and GPU-driven execution to generate tokens, and exposes an Anthropic Messages-compatible API.","audience":"Developers running Naive-N0.5-Flash on eight-GPU Blackwell systems for single-stream inference","claims":[{"kind":"specific","claim":"Measured device generation rate of 1,719 tokens/s across requests in an HTML/SVG benchmark with thinking disabled","excerpt":"NaiveRT achieved **1,719 device tokens/s across requests**","status":"not_checked"},{"kind":"specific","claim":"Peak one-second rate of 2,122 tokens/s in the same benchmark with thinking disabled","excerpt":"with a **peak one-second rate of 2,122 tokens/s**","status":"not_checked"},{"kind":"specific","claim":"Requires eight Blackwell GPUs with UMMA (tcgen05) support and peer-to-peer access, with the supplied build targeting compute capability 10.3a","excerpt":"The supplied build targets compute capability **10.3a**","status":"not_checked"},{"kind":"specific","claim":"Targets bitwise-identical tokens, logits, and KV state for fixed inputs, checkpoint, seed, and supported configuration","excerpt":"NaiveRT targets bitwise-identical tokens, logits, and KV state.","status":"not_checked"},{"kind":"specific","claim":"Processes one active sequence per instance, with additional requests queued and rejected with HTTP 529 when the queue is full","excerpt":"The server processes **one active sequence per instance**.","status":"not_checked"}],"alternatives":[],"written_by":"AI, from the project's README","date":"2026-11-12"},"why_now":null,"owner_supplied":null,"score":{"verdict":"too_early","hype":57,"reality":0,"gap":56,"momentum":null,"confidence":60,"version":"score-2.0","rule":"verdict-2.0","calculated_at":"2026-11-22T08:00:00+00:00","verdict_source":"rule"},"early_read":{"estimate":true,"lean":"too_close","gap_low":17,"gap_high":52,"rule":"early-2.0"},"changes":[{"date":"2026-09-27T07:43:04+00:00","text":"Repository created","source":"https://github.com/NaiveAI-Labs/NaiveRT"}],"cite":"NaiveRT is a GitHub project at https://github.com/NaiveAI-Labs/NaiveRT written mainly in Python, described there as \"NaiveRT: a self-contained TP8 inference runtime for the Naive N series on eight Blackwell GPUs with UMMA support. One persistent CUDA graph per rank runs draft, eight-row verify, top-p sampling and cache commit with no host in the loop: 1,700 tokens/s at batch size 1, one-second peaks above 2,000\".","attribution":"Data from Mid or Real, https://midorreal.com/project/naiveai-labs-naivert, checked 22 November 2026. Attribution with a link is required.","still_real":null}