A model launch usually comes with a chart showing the new model on top. That is the point of the chart. Five questions help you read it fairly.
- Who ran the test?
Results from the company are claims. Results from an outside lab, or from several, are evidence.
- Can anyone repeat it?
A public benchmark can be rerun. A private evaluation cannot.
- What is the comparison?
Check which rival models are on the chart, which versions, and which are missing.
- What did it cost to get that score?
A result that needs ten attempts, or a huge amount of compute, is not the same as one that works first time.
- What does the price do next?
Introductory prices end. Work out your cost at the standard rate before you build on it.
