Questions, comparisons and evidence

The same model with Exa and Brave

For each row, take the questions successfully read on both sides, then collect each side's distinct recommendations. Overlap is the shared count divided by the union of the two lists. The denominator is tools, not mentions or repeated calls. Each question was asked once per combination.

What was available to compare

There were 468 intended question–combination slots. Two executions produced 457 successful answers; the product extraction successfully read 456. An extraction that returned an empty list after reading an answer is observed data. A failed or missing reading is a gap and is excluded from comparisons.

Qwen + Brave had nine timeouts. Mistral + Exa had two unsuccessful answers. MiniMax + Brave had one failed product extraction. Grok and GLM's first attempts failed at the account limit; their successful answers came from the second execution. We compare their unchanged prompt text with the first execution's answers.

All 325 pair comparisons

The post's summary means use 55 Exa pairs, 55 Brave pairs, 40 pairs of different models on Brave and built-in search, and 11 same-model Exa/Brave pairs. Every pair is weighted equally within its stated comparison group. Each pair uses its own intersection of successfully read questions; populations can differ. There are no confidence intervals or causal estimates here.