# Choosing varied results, and whether the reader matters

Keep the most relevant results and fill the rest with the least similar ones. This buys topic coverage at half the relevance cost of the standard method. The idea that a model reader needs more variety than a person did not hold up.

Author: Murali Sid (https://linkedin.com/in/muralisid)
Source: https://www.agenticarchitectureskills.com/patterns/diversity-and-consumer
Published: 2026-08-23
Updated: 2026-10-15
Licence: CC BY-SA 4.0, https://creativecommons.org/licenses/by-sa/4.0/
Cite as: Murali Sid. Choosing varied results, and whether the reader matters. Agentic Architecture Skills. https://www.agenticarchitectureskills.com/patterns/diversity-and-consumer

---


After a search finds candidates, something must pick the ten that reach the agent. The ten most similar often repeat the same point. Ten deliberately different ones cover more ground, but may include passages that do not answer the question. We asked which selection rule gives the best trade. The answer: keep the most relevant results at the top, then fill the last places with the least similar candidates. We call this the harvest. It matched the coverage of the standard method, MMR, at half the relevance cost. It also let in fewer junk passages. We also tested whether a model reader needs more variety than a person. That idea did not survive a fair judge.

This choice is the last step before material enters the agent's [context](/ladder/context). It is part of how agents use [knowledge and memory](/memory). It sits at the retrieve stage of the [data-to-memory pipeline](/architecture/data-to-memory), which the [agent data engineering layer (R14)](/layers/r14-agent-data-engineering) builds.

## The test: questions where the sub-topics are known [#the-test-questions-where-the-sub-topics-are-known]

Each of 120 queries has several distinct facts. Each candidate document states at most one of them. So we can count exactly which sub-topics a selection covers. We did not find sub-topics by clustering. That would have been circular, because one rule under test selects by cluster.

First, candidates are cut to the most relevant half, as a first-pass filter would do. Then a pool of answerless documents is mixed in. We call these distractors. Each rule then selects ten.

Three scores tell the rules apart. <Term k="coverage metrics">S-recall</Term> is the share of known sub-topics covered. <Term k="ndcg">nDCG\@10</Term> measures relevance. The distractor rate is the share of answerless documents let in.

We tested seven rules.

* Relevance only: the ten most relevant.
* <Term k="diversity selection">MMR</Term> (maximal marginal relevance), at lambda 0.7 and 0.3. It is the standard rule, built into most vector stores. It trades relevance for variety across the whole list. A lower lambda means more variety.
* DPP (determinantal point process), a maths method that picks a spread-out set.
* Cluster round-robin: one result from each cluster in turn.
* The harvest, at 20 and 40 percent. Keep the most relevant results, then fill that share of the ten places with the least similar candidates.

<ResultChart
  kind="scatter"
  title="Coverage against relevance for seven selection rules"
  subtitle="120 built queries, ten results each, seed 13. Up is more sub-topic coverage. Right is more relevance."
  xLabel="Relevance (nDCG@10)"
  yLabel="Sub-topic coverage (S-recall)"
  points="[
  { label: 'Relevance only', x: 0.827, y: 0.372, anchor: 'end' },
  { label: 'MMR, lambda 0.7', x: 0.802, y: 0.405 },
  { label: 'Harvest 20%', x: 0.808, y: 0.567, emphasis: true },
  { label: 'Cluster round-robin', x: 0.774, y: 0.545, anchor: 'end' },
  { label: 'Harvest 40%', x: 0.781, y: 0.686, emphasis: true },
  { label: 'MMR, lambda 0.3', x: 0.729, y: 0.686, anchor: 'end' },
  { label: 'DPP', x: 0.699, y: 0.746, anchor: 'end' },
]"
  caption="The two harvest settings sit above and to the right of MMR and cluster round-robin at similar coverage. The DPP reaches the most coverage and pays the most relevance for it."
  source="Author-reported experiment, August 2026. The comparison and limitations are described on this page."
/>

| Rule                      | S-recall | Alpha-nDCG | nDCG\@10 | Distractor rate |
| ------------------------- | -------- | ---------- | -------- | --------------- |
| Relevance only            | 0.372    | 0.542      | 0.827    | 0.168           |
| MMR, lambda 0.7           | 0.405    | 0.566      | 0.802    | 0.197           |
| MMR, lambda 0.3           | 0.686    | 0.695      | 0.729    | 0.285           |
| DPP                       | 0.746    | 0.706      | 0.699    | 0.328           |
| Cluster round-robin       | 0.545    | 0.626      | 0.774    | 0.229           |
| Harvest, 20 percent quota | 0.567    | 0.619      | 0.808    | 0.198           |
| Harvest, 40 percent quota | 0.686    | 0.677      | 0.781    | 0.236           |

Alpha-nDCG is a ranking score that rewards coverage and discounts repeats. Read along the rows. The 40 percent harvest and MMR at lambda 0.3 reach the same coverage, 0.686. Against relevance-only ranking that is a gain of 0.315 for the harvest and 0.314 for MMR, from the unrounded scores. The table shows rounded values. The harvest pays 0.046 of relevance for it. MMR pays 0.098. The harvest also lets in fewer distractors: 0.236, against 0.285 for MMR and 0.328 for the DPP.

The reason shows in how the rules work. MMR and the DPP spread out the whole list, so even the top drifts away from relevance. The harvest keeps a relevant top and spends only the tail. All coverage gains over relevance-only ranking are significant at p=0.0001 after Holm correction. So are all the relevance costs.

An earlier version of this test reported coverage as nearly free. It was wrong. A coding bug put two distractors in each task instead of forty. So nothing could be lost, and every relevance score sat at its maximum. The corrected trade is the one above.

## The reader test: does the right amount of variety depend on who reads? [#the-reader-test-does-the-right-amount-of-variety-depend-on-who-reads]

We expected it would. The effect shrank to nothing under a fair judge.

The pool, the queries and the writing model stayed the same throughout. Only the selection rule and the reader changed. For a model reader, the selection is evidence for a model. It must write an account of everything the material covers. For a person, the same selection becomes a short brief they will read once and act on. Sixty tasks, two rules (relevance and harvest) and two readers give 240 generated texts.

We predicted a difference between readers, not a gain for everyone. The harvest should raise coverage more for the model reader than for the person.

Three judges scored coverage of the known sub-topics in each text. A judge is a model that grades another model's output. One judge came from the same company as the writing model. One was a larger model from that company. One came from a different company.

<ResultChart kind="dots" title="The reader effect, as each judge measured it" subtitle="The harvest's coverage gain for the model reader minus its gain for the person. 60 tasks, with 95 percent bootstrap intervals. A positive value supports the claim." categories="['Same-company judge, small', 'Same-company judge, large', 'Different-company judge']" series="[{ name: 'Reader difference', values: [0.071, 0.047, 0.008] }]" intervals="[{ low: 0.022, high: 0.125 }, { low: -0.024, high: 0.117 }, { low: -0.039, high: 0.056 }]" baseline="0" min="-0.06" max="0.14" ticks="[-0.05, 0, 0.05, 0.1]" xLabel="Reader difference (model-reader gain minus person gain, judged coverage)" caption="+0.071 (p=0.009), +0.047 (p=0.211), +0.008 (p=0.763). The estimate shrinks as the judge moves away from the writer's own company. It vanishes with the judge from another company. That pattern points to a flaw in the judging, not a real effect. It is not disproved. All three estimates are positive, and 60 tasks cannot resolve an effect this small. But no evidence can be cited for it." source="Author-reported experiment, August 2026. The comparison and limitations are described on this page." />

One finding from the same study needs no judge, and it survived every check. We counted words in each output that no selected document supports. Harvesting reduced this unsupported content.

* For the brief for a person: -0.067 (p=0.0018).
* For the account written for a model: -0.048 (p=0.067).

A selection that covers more of the material gives the writer less reason to invent. We did not predict this. It does not depend on a judge. It is the most solid result of the reader study.

## A worked example [#a-worked-example]

A utility planner asks an agent for every known cause of a substation fault. That question has several answers, so a varied set of ten helps. The harvest keeps the strongest matches at the top and adds different causes below them. A question with one answer, such as the date of the last inspection, gains nothing from variety. This example shows how to apply the rule. We tested it on built questions, not on utility records.

## What we conclude [#what-we-conclude]

<PrincipleCard eyebrow="Verdict" title="An efficient selection rule, and an unsupported reader rule" kind="evidence">
  Within a filtered pool, the harvest buys sub-topic coverage more cheaply than MMR or a DPP. Its summaries also contain fewer unsupported statements. That half stands. The reader half does not. No evidence for tuning variety to the reader survives a judge from another company. We withdraw that claim. If you already use MMR or a DPP, set its balance per workload on measured coverage. Do not set it per reader on the strength of these results.
</PrincipleCard>

## Evidence and limits [#evidence-and-limits]

<EvidenceBadge status="mixed-evidence" /> The selection half is measured on built ground truth. The reader half depends on model judges and is unresolved.

The queries were built for the test, not taken from real users. Sixty tasks are too few to detect a small reader effect. We ran this ourselves. Nobody else has repeated it yet.

## Next step [#next-step]

See how all these tests were kept honest, in [how to test a context design](/patterns/how-to-test-a-context-design). Or go back to the [nine context patterns](/patterns).

<details>
  <summary>
    Explore this topic
  </summary>

  * Carbonell and Goldstein, MMR, SIGIR 1998. Diverse multi-answer retrieval with determinantal point processes, COLING 2022. DIVA, NAACL 2025. The Distracting Effect, ACL 2025. The Power of Noise (SIGIR 2024) and its retraction (SIGIR 2026). All are on the [reading list](/patterns/reading-list).
  * Alpha-nDCG, Clarke et al., SIGIR 2008. Sub-question coverage, Salesforce and Georgia Tech, Oct 2024. Both explain why coverage needs coverage scores, not recall.
  * The experiment ran in August 2026.
</details>