RAG Hallucinations in News: Define When Evidence Is Too Thin

News applications must inform the user when no meaningful answer is found. Returning a confidence paragraph with citations is not enough. The product needs a way to explain what was found, what is unknown, and why a claim was not made.

A paper recently published on arxiv.org on October 21, 2026, provides an excellent opportunity to create this functionality. The paper, Data-Void Exploits in Web-Searching LLMs, examines cases where relevant information may be completely lacking. The researchers experimented on web-searching models, not on Webz.io or similar production news applications.

There are two different types of failures here. First, does the answer represent the captured text? Second, assuming the answer is reported faithfully, is that text sufficient to validate the claim? A model can parrot a misleading answer faithfully. In that case, calling the answer a hallucination is a smoke screen to avoid talking about the quality of the source. The workflow describes how to address the insufficiency of the evidence.

Assumption is the mother of all…failures. Describe the claim the application is supposed to verify. Determine the organization, event, place, and time in the window being questioned. Distinguish the claim reported from the underlying claim that may be true.

Consider a hypothetical question: has production at factory X stopped this week? Financial difficulty of the company may be reported in a news article but may not substantiate a production stoppage claim. The application should not quietly parrot one question for the other.

State the evidence needed to support that claim. One team might require a statement from the operator or a clearly attributed report describing the stoppage. These are proposed product rules, not a definitive source hierarchy. The appropriate requirement depends on the decision the answer will support.

Keep the scope of the search visible

Retain the query, filters and time of the search in an investigation record. State any known constraints on the question, like the language searched and the publication window. Retain permitted source material to allow a reviewer to understand the decision.

If the result of a query is inadequate, state the next query that will likely resolve that deficiency. For the factory example, maybe a different company name in the local language is searched. Give the reason for that query.

Do not broaden search constraints without a stopping rule. A wider query can change the focus of the search rather than answer the original question. Set a reasonable limit on the number of searches and state that limit to a reviewer.

There are limits to public reporting and gaps in collection. An empty result in a defined search should not be stated as nothing happened.

Examine the basis for the claim

For every claim state the basis for it. Determine if the basis is from a direct observation, is attributed to another, or is simply repeated. Preserve the attribution in the response.

Many articles can lead back to one original article. Be sure to record the relationships between the source articles. A set of rewritten articles should not automatically satisfy the need for independent reporting.

This review also needs a space for uncertainty. When the source of a claim cannot be verified, describe the situation in the internal record. Do not call a publisher malicious when the application was not able to find much information. Thin evidence is a case for limiting a conclusion, not an attack.

Choose an answer status deliberately

Give the product explicit outcomes rather than forcing every search into a yes-or-no answer. One of the possible outcomes is a supported answer with appropriate attribution. Another possible outcome is a brief summary of what the source reported. A third possible outcome is an insufficient evidence answer.

For the supplier example, one possible outcome is to summarize a report that describes financial difficulty. The reviewed evidence does not validate a factory stoppage. Keeping the answer brief helps avoid drawing unsubstantiated conclusions.

The application needs to give a reason for withholding the claim. One of the possible reasons could be that matching entities is still uncertain, event timing is still uncertain, or that there is no passage to validate the event. Avoid a confidence percentage that gives no specific reasons for why a claim could not be validated. Reviewers need to see the specific reason for which they need to investigate.

Each time a primary statement is not found, there are still ways to make the system useful to the user. The scope should be explained, along with the question that is left unanswered. A next step should be provided that is consistent with the overall flow of the system. This next step may be a check for an operator update, or an analyst review.

It is a mistake to either provide an accusation, or reassure the user, when evidence for a primary statement is unavailable. This may have different impact to a business user, and could cause the user to take unjustified actions, but in both cases this exceeds what is warranted, given the evidence. Explain only what is warranted, and no more.

A question should be documented for what information would allow a review to be reopened, and an owner of that review should be identified.

Use filters provided by Webz.io to define the news evidence the system wants to review.

Look over the returned material to analyze the claim and its context. The application needs its own answer policy, review process and list of unanswered questions. Retrieval supplies candidate evidence; it does not validate every conclusion drawn from that evidence.

Allow a small set of permitted review examples: supported claims, uncertain claims and claims for which there is no sufficient evidence. Instruct reviewers to consider the support and the response as a unit. Instruct reviewers to analyze examples where several articles repeat the same statement.

## Preserve the decision for review

Preserve the final answer status, sources reviewed and the rationale for any limitations. Identify the reviewer and date. If the decision changes due to new information, preserve the previous decision and explain the change.

The ultimate goal is a news application that can be useful, even if not every question is settled. A clear insufficient evidence response gives users something a fluent unsupported response cannot: an honest boundary around the decision they are about to make.