Critique of Demo-Benchmarks in AI Model Evaluation

GPT Astra was released recently, leading to a surge of social media posts showcasing various creative outputs, including recreating Minecraft, painting in MS Paint, and generating a pelican riding a bicycle. These outputs are referred to as demo-benchmarks, which are visually appealing and easy to understand but may not accurately reflect the capabilities of AI models. The author argues that these benchmarks can be easily optimized for, making them ineffective for true evaluation of model performance. They suggest that a benchmark should be challenging and not easily perfected. The discussion highlights that smaller models can sometimes outperform larger ones in evaluations, as seen with Thinking Machines' Inkling Small scoring closely to its more advanced counterpart on the Artificial Analysis Intelligence Index. The article suggests that while demo-benchmarks generate engaging content, they do not provide a reliable measure of a model's true capabilities. Alternatives like LiveBench and ARC-AGI are mentioned as potential solutions for more effective evaluation, as they involve unpredictable testing scenarios. However, the author acknowledges that demo-benchmarks remain popular due to their immediate visual impact, despite their limitations in assessing model performance.