{"name":"spinf-serve","repo":"spinfinc/spinf-serve","url":"https://midorreal.com/project/spinfinc-spinf-serve","repository":"https://github.com/spinfinc/spinf-serve","description":"Bulk classification and decision inference on one GPU: the content is read once, every question about it is scored in the same pass. OpenAI-compatible, Apache 2.0","facts":{"license":{"value":"Apache-2.0","source":"https://github.com/spinfinc/spinf-serve","checked":"21 Nov 2026"},"language":{"value":"Python","source":"https://github.com/spinfinc/spinf-serve","checked":"21 Nov 2026"},"homepage":{"value":"https://spinf.com","source":"https://spinf.com","checked":"21 Nov 2026"},"last_release":{"value":"0.1.2, 29 Oct 2026","source":"https://github.com/spinfinc/spinf-serve/releases","checked":"21 Nov 2026"},"releases":{"value":"2","source":"https://github.com/spinfinc/spinf-serve/releases","checked":"21 Nov 2026"},"contributors":{"value":"1","source":"https://github.com/spinfinc/spinf-serve/graphs/contributors","checked":"21 Nov 2026"},"archived":{"value":"no","source":"https://github.com/spinfinc/spinf-serve","checked":"21 Nov 2026"}},"stars":21.0,"writeup":{"what_it_is":"spinf-serve is an open-source (Apache 2.0) inference server for bulk classification and decision inference on one 24 GB NVIDIA GPU. It reads a piece of content once and returns a probability over the options given for each question asked about it, instead of generated text. It exposes an OpenAI-compatible API.","audience":"Teams running bulk classification or decision scoring on their own GPU hardware","claims":[{"kind":"specific","claim":"Reads content once and answers many questions about it in the same pass","excerpt":"reads a piece of content once and answers many questions about it in the same pass","status":"not_checked"},{"kind":"specific","claim":"Runs on one 24 GB NVIDIA GPU with an OpenAI-compatible API","excerpt":"It runs on one 24 GB NVIDIA GPU and serves an\nOpenAI-compatible API","status":"not_checked"},{"kind":"specific","claim":"Processed 1,344 questions in 2.7 seconds on one RTX 4090 in the project's benchmark, versus 6.7 seconds for vLLM","excerpt":"| the 1,344 questions | **2.7 s** | 6.7 s |","status":"not_checked"},{"kind":"specific","claim":"Batch-invariant: a content's scores are bit-identical whether scored alone or in a batch, with the fp8 checkpoints","excerpt":"With the fp8 checkpoints, a content's scores are bit-identical\nalone or packed with thousands of others","status":"not_checked"},{"kind":"broad","claim":"Throughput is built as the priority rather than single-request latency","excerpt":"it is built for throughput rather than for\nthe latency of a single request","status":"too_broad"}],"alternatives":[],"written_by":"AI, from the project's README","date":"2026-10-31"},"why_now":null,"owner_supplied":null,"score":{"verdict":"overhyped","hype":96,"reality":38,"gap":58,"momentum":null,"confidence":85,"version":"score-2.0","rule":"verdict-2.0","calculated_at":"2026-11-22T07:00:00+00:00","verdict_source":"rule"},"early_read":null,"changes":[{"date":"2026-10-29T04:31:47+00:00","text":"Latest release 0.1.2","source":"https://github.com/spinfinc/spinf-serve/releases"},{"date":"2026-10-25T06:37:14+00:00","text":"First release","source":"https://github.com/spinfinc/spinf-serve/releases"},{"date":"2026-10-25T05:29:59+00:00","text":"Repository created","source":"https://github.com/spinfinc/spinf-serve"}],"cite":"spinf-serve is a GitHub project at https://github.com/spinfinc/spinf-serve written mainly in Python, described there as \"Bulk classification and decision inference on one GPU: the content is read once, every question about it is scored in the same pass. OpenAI-compatible, Apache 2.0\".","attribution":"Data from Mid or Real, https://midorreal.com/project/spinfinc-spinf-serve, checked 22 November 2026. Attribution with a link is required.","still_real":null}