Grounding and abstention

Retrieval puts text in front of the model. Nothing so far obliges the model to use it, and nothing stops it filling gaps from training. Grounding is the set of instructions and checks that make the retrieved passages the only permitted source.

The instruction matters and is not sufficient. "Answer only from the passages" moves behaviour a long way, and it is a request, not a control. Toggle the grounding rules off in the lab and compare — the difference is usually visible, and it is never a guarantee.

The harder case is the question nothing answers. Retrieval always returns something: ask about trade discounts and it hands back the three nearest passages, which are about delivery charges and warranties. A model given three passages and a question will usually try.

Q: Do you offer a trade discount for interior designers?

  0.34  Shipping - Delivery charges
  0.27  Warranty - Warranty period
  0.26  Returns - Furniture and large items

Nothing in the corpus answers this. Every score is below 0.4.

That trace is from this lab, and it contains the answer to the problem. The scores knew. A floor — refuse to generate when the best hit falls below a threshold — catches the unanswerable case before a token is spent, and it does not depend on the model choosing to be honest.

• Set a similarity floor and tune it on questions you know are unanswerable.
• Ask for the passage number the answer came from. An answer that cannot cite is one you can reject automatically.
• Make "I could not find this" an acceptable output, in the prompt and in whatever measures the system.
• Watch the abstention rate. Zero means the system is answering things it should not.