{"name":"llm-verbosity","repo":"gbeltramo/llm-verbosity","url":"https://midorreal.com/project/gbeltramo-llm-verbosity","repository":"https://github.com/gbeltramo/llm-verbosity","description":"Checks to evaluate LLM response verbosity","facts":{"license":{"value":"MIT","source":"https://github.com/gbeltramo/llm-verbosity","checked":"21 Nov 2026"},"language":{"value":"Python","source":"https://github.com/gbeltramo/llm-verbosity","checked":"21 Nov 2026"},"releases":{"value":"0","source":"https://github.com/gbeltramo/llm-verbosity/releases","checked":"21 Nov 2026"},"contributors":{"value":"1","source":"https://github.com/gbeltramo/llm-verbosity/graphs/contributors","checked":"21 Nov 2026"},"archived":{"value":"no","source":"https://github.com/gbeltramo/llm-verbosity","checked":"21 Nov 2026"}},"stars":1.0,"writeup":{"what_it_is":"llm-verbosity is a tool that measures how verbose an LLM API is. It reports elapsed time, thinking tokens, answer tokens, generated tokens and total tokens on easy, mid and hard Q&A datasets (MATH-500, GPQA-Diamond and HLE-Diamond). It supports OpenRouter and a budget-style agent API.","audience":"Researchers and developers evaluating LLM response length and token usage","claims":[{"kind":"specific","claim":"Measures thinking tokens, answer tokens, generated tokens and total tokens on easy, mid and hard Q&A","excerpt":"Measure how verbose an LLM API is: **elapsed time, thinking tokens, answer (post-thinking)","status":"not_checked"},{"kind":"specific","claim":"Writes 100 seeded questions per dataset","excerpt":"Writes 100 seeded questions per dataset to `data/*.parquet` (zstd).","status":"not_checked"},{"kind":"specific","claim":"Requests are retried with exponential backoff on transport errors and certain HTTP status codes","excerpt":"are retried with exponential backoff on transport errors,","status":"not_checked"},{"kind":"specific","claim":"Token counts come from the API response and are not estimated from text length","excerpt":"Everything comes from the API response (`usage` block, engine timings).","status":"not_checked"}],"alternatives":[],"written_by":"AI, from the project's README","date":"2026-11-12"},"why_now":null,"owner_supplied":null,"score":{"verdict":"mid","hype":42,"reality":22,"gap":21,"momentum":null,"confidence":60,"version":"score-2.0","rule":"verdict-2.0","calculated_at":"2026-11-22T09:00:00+00:00","verdict_source":"rule"},"early_read":null,"changes":[{"date":"2026-10-27T09:37:28+00:00","text":"Repository created","source":"https://github.com/gbeltramo/llm-verbosity"}],"cite":"llm-verbosity is a GitHub project at https://github.com/gbeltramo/llm-verbosity written mainly in Python, described there as \"Checks to evaluate LLM response verbosity\".","attribution":"Data from Mid or Real, https://midorreal.com/project/gbeltramo-llm-verbosity, checked 22 November 2026. Attribution with a link is required.","still_real":null}