Click any column header to sort. Highlighted rows mark highest score, lowest latency, and best score per second.
Judge issues are model-output mistakes found by the judge. One result can have multiple issues; run failures are shown separately as statuses.
ChirperBench checks whether local Ollama models can clean up dictated transcripts without treating the dictated words as instructions to execute.
The suite stresses command-like text, dictated questions, email requests, markdown, URLs, code identifiers, spelling corrections, mixed formatting, and cases where no change is needed.
Scores run from 0 to 100. Passing means the judge accepted the cleaned transcript as meeting the case target.
Judge issues are model-output mistakes, not necessarily process crashes. One result can have several judge issues, such as answering a dictated question, refusing a command-like transcript, leaking spoken edits, or inventing extra text.