MIDREAL
Menu

Backtest

Status: Passed. One run per version, with the pass rules written down before we saw the result. The current scoring version, score-2.0, was tested on 2026-10-08. The earlier version, score-1.0, was tested first and is kept below as history.

What it tested (score-2.0)

Does adding package usage to Reality hurt the ranking? We drew 200 projects at random from 3,867 candidates that peaked in 7-day star gain between 2024-01-01 and 2024-06-05. This is a fresh set, not the one used for score-1.0. Each was scored using only data that existed at the time, then checked a year later. The pass rule, written first: rank projects that were abandoned a year later at least as well as score-1.0 does, on the same projects.

The result (score-2.0)

It passed: the score-2.0 AUROC is at least the score-1.0 AUROC.

The earlier result (score-1.0, history)

Does the Hype Gap predict which projects were later abandoned better than raw star growth does? We took 198 projects that peaked in the first half of 2024 and scored them using only data that existed at the time. 53 were abandoned.

That pass rule: AUROC of 0.60 or more with the interval above 0.50, and a lower bound above zero against star velocity. Both held.

Where it is weak

See also the methodology.