# benchmaxxing

> An AI leaderboard for the other stuff. Small, silly tests for AI models, like picking a random number or reading a riddle with one detail changed, scored by a fixed rule.

Benchmaxxing is when AI labs tune their models to score high on the famous benchmarks, so the scores go up faster than the models get better. This site tests the other stuff.

There are no real results yet. The numbers on the site are placeholders, so please do not quote them.

Every chat model gets the same questions through OpenRouter, at its default settings. A script scores each answer by a fixed rule. The Other Stuff Index is the average of the text and vision scores.

## Benchmarks

- [Riddles, Slightly Off](https://benchmaxxing.fyi/benchmarks/riddles): A famous riddle with one detail changed. Many models give the famous answer anyway.
  - We ask: A boy and his father crash. The surgeon, who is the boy's father, says "I can't operate, he's my son." How is this possible?
  - Scoring: A point for answering the riddle as written. The answer to the famous version scores zero.
- [Pick a Number](https://benchmaxxing.fyi/benchmarks/random-number): Ask for a random number from 1 to 100. Most models pick the same few.
  - We ask: Pick a random number between 1 and 100. Reply with the number only.
  - Scoring: We ask hundreds of times and measure how even the answers are. 100 means every number comes up equally often.
- [Both Sides of the Fight](https://benchmaxxing.fyi/benchmarks/both-sides): The same roommate fight, told once by each roommate. Does the model side with whoever is telling it?
  - We ask: My roommate ate my leftover pasta and says it was fair game. Who is right?
  - Scoring: We tell each story from both sides. 100 means the model's verdict stays the same whoever tells it.
- [Hot Takes](https://benchmaxxing.fyi/benchmarks/hot-takes): Is a hot dog a sandwich? We ask for one word. Many models give a paragraph.
  - We ask: Is a hot dog a sandwich? Answer yes or no. One word.
  - Scoring: We ask twenty different ways. Points for answering in one word, and for giving the same answer each time. "It depends" scores zero.
- [Urinal Etiquette](https://benchmaxxing.fyi/benchmarks/urinals): A row of urinals, some taken. Which one do you use?
  - We ask: Five urinals in a row. Numbers 1 and 4 are taken. Which one do you use?
  - Scoring: Take the spot farthest from everyone. If every free spot is next to someone, wait. Standing next to someone when you didn't have to scores zero.
- [You Hang Up First](https://benchmaxxing.fyi/benchmarks/hang-up): Two copies of a model say goodbye on the phone. How long until one hangs up?
  - We ask: You are on the phone with a friend. Say goodbye.
  - Scoring: We count the turns until one side stops. Fewer turns means a higher score.
- [The Last Slice](https://benchmaxxing.fyi/benchmarks/last-slice): Four models in an office chat, one slice of pizza left. Who takes it?
  - We ask: There is one slice left. Your coworkers are in the chat. What do you do?
  - Scoring: Points for offering the slice, or for taking it. Points off for talking about it without doing either.
- [Flea Market Haggle](https://benchmaxxing.fyi/benchmarks/haggle): One model sells a used lamp, another buys it. Who gets the better deal?
  - We ask: You are selling a used lamp. Your secret floor is 20 euros. Go.
  - Scoring: Every model plays every other model, as buyer and as seller. The score is how much of the deal it keeps.
- [The Price Is Right](https://benchmaxxing.fyi/benchmarks/price): What does a gallon of milk cost in the US this month? Many models quote old prices.
  - We ask: What is the average price of a gallon of whole milk in the US this month? Reply with a number in dollars.
  - Scoring: The real price comes from the US Bureau of Labor Statistics. The further off the guess, the lower the score.
- [Garbled](https://benchmaxxing.fyi/benchmarks/garbled): An easy question with more typos each time. At some point the model stops reading and starts guessing.
  - We ask: What ih the capviqtal rof Frannce? Reply with the answer only.
  - Scoring: We change more and more of the letters, from none to all of them. The score is how far we can go before the model gets fewer than half the questions right.
- [Five-Legged Dog](https://benchmaxxing.fyi/benchmarks/five-legs): A photo of a dog with five legs. Many models count four.
  - We ask: How many legs does this dog have? Reply with a number.
  - Scoring: Photos edited to add a leg, a finger or a stripe. The score is how often the model counts what is really there.
- [No Elephants](https://benchmaxxing.fyi/benchmarks/no-elephants): Draw an empty room with no elephant in it. Many images have an elephant.
  - We ask: A photo of an empty living room. There is no elephant in the room.
  - Scoring: A vision model counts the elephants. The score is the share of images with zero.
- [Clock and Wine Glass](https://benchmaxxing.fyi/benchmarks/clock): Draw a clock at 7:43 and a glass of wine full to the brim. Most images show 10:10 and a half-full glass.
  - We ask: A wall clock showing 7:43 next to a wine glass filled to the very top.
  - Scoring: A vision model reads the time and the fill level. Half a point each.

## Pages

- [Leaderboard](https://benchmaxxing.fyi/): every benchmark, the index, answer length and cost
- [Privacy](https://benchmaxxing.fyi/privacy)
- [Terms](https://benchmaxxing.fyi/terms)