<p>Existing benchmarks measure solve rate and token output/cost, but do not penalize excess code, which can complicate codebases over time. The goal of this benchmark is to assess this aspect, particularly for long-term projects.</p><p>Methodology: Parsimony utilizes patches published by existing benchmarks (DeepSWE, SWE-bench Verified, SWE-bench Live) and counts "coding units," which include syntax elements such as statements, calls, names, and literals. It does not rely on lines of code, thus avoiding rewards for overly compact code. Deleted code is subtracted from the total, allowing for a measurement of net coding units.</p><p>As of now, GPT-6 and Claude 5.5 do not have public patches available, so the author conducted tests on 3–10 tasks using their own subscriptions, represented as faded circles in the results, which are not ranked.</p><p>Feedback and contributions regarding models are encouraged, particularly public patch exports for newer models.</p><p>GitHub repository: <a href="https://github.com/La5u/Parsimony" rel="nofollow">https://github.com/La5u/Parsimony</a></p><p>This project is not affiliated with any of the benchmarks mentioned.</p>
✓ AI-Debiased Article
4
Wire-neutral
provisional
✓ No loaded language, vague sourcing, or framing detected.
Parsimony: An Open-Source Benchmark for Evaluating AI Model Code Efficiency
Parsimony is an open-source benchmark designed to evaluate the efficiency of AI models in terms of the amount of code they generate. It measures 'coding units' from existing benchmarks and aims to provide insights into how excess code can affect long-term project viability.
Annotating as
No note attached
on this article.
Original vs. Neutral
Original Headline
Show HN: Parsimony – an open-source benchmark for how much code AI models add
Neutral Headline
Parsimony: An Open-Source Benchmark for Evaluating AI Model Code Efficiency