Deep-search AI agents can recognize that evidence deserves additional checking and still fail to escape a poisoned information environment. A new benchmark finds that verification behavior alone is often insufficient to make an agent revise a manipulated claim before producing its final recommendation.
The study, “Evaluating Deep-Search Agents under Hierarchical Web Evidence Poisoning,” was submitted to arXiv on September 5 by Zhongan Bi and 13 co-authors. Their HAE-GEO benchmark evaluates ten search-augmented agents against progressively more persuasive web poisoning using a controlled corpus of 72,039 clean pages and 770 poisoned pages at each attack level, spanning eight product categories and 154 brands.
Unlike benchmarks that stop after asking whether malicious content was retrieved or cited, HAE-GEO follows the agent’s entire trajectory. It asks whether the system encounters the poisoned evidence, recognizes the need to investigate it, searches for additional information, revises what it has already accepted and ultimately recovers the correct recommendation.
The central result is a warning for the emerging “deep research” model of AI search: more searching can improve final resistance, but searching and verification do not automatically produce better evidence recognition or successful self-correction. The problem becomes particularly severe when several sources appear to confirm the same false claim.
HAE-GEO measures what happens after an agent sees poisoned evidence
Most evaluations of generative engine optimization attacks focus on a relatively short chain. An attacker publishes manipulated content, the search system retrieves it and the benchmark records whether the model cites or endorses the targeted source.
HAE-GEO extends the evaluation beyond that first failure.
A deep-search agent is supposed to do more than answer from the first page it finds. It can issue additional searches, open sources, scrape pages and continue investigating before committing to a recommendation. In principle, that agentic behavior should create opportunities to detect contradictions and recover from bad evidence.
The benchmark therefore treats poisoning as a process rather than a binary outcome. Exposure to an attack is only the beginning. The important question is whether the agent can correct course after exposure.
The controlled web contains 72,039 clean pages
The researchers construct a controlled corpus containing 72,039 clean pages alongside manipulated pages designed to influence product and brand recommendations.
The benchmark covers eight product categories and 154 brands, giving agents a sufficiently broad environment in which to search rather than presenting one isolated malicious document directly in the prompt.
At each of the three poisoning levels, the evaluation includes 770 poisoned pages. This detail matters because the attack becomes progressively more persuasive across levels rather than simply adding one static malicious page to the same environment.
The agents interact with this corpus through a multi-turn Search-Scrape interface. They can search, inspect results and gather evidence over several steps before producing a final answer.
That setup gives the researchers visibility into the reasoning process at the behavioral level: not hidden chain-of-thought, but observable actions such as whether an agent searches again, opens additional sources or changes its final recommendation.
The benchmark escalates poisoning through three levels
HAE-GEO is hierarchical because the manipulated evidence becomes harder to resist as the attack progresses.
Level 1 uses direct assertion. The poisoned content presents the target claim relatively plainly, giving the agent a chance to identify a suspicious or unsupported statement and seek confirmation elsewhere.
Level 2 introduces contextual camouflage. The manipulated claim is embedded in a more convincing informational context, making it less conspicuous as an attack.
Level 3 creates apparent corroboration. Multiple pieces of evidence make the false narrative look independently supported.
The progression is designed to model a crucial property of the open web: misinformation becomes more persuasive when it does not appear isolated.
The “corroboration trap” is where recognition breaks down
The researchers identify apparent corroboration as one of the most important weaknesses across the ten evaluated agents.
When several sources seem to support the same claim, evidence recognition deteriorates. The agent becomes less likely to treat the information as suspicious because agreement across sources normally functions as a useful credibility signal.
That creates what the paper calls a corroboration trap.
Deep-search systems are often encouraged to verify information by checking multiple sources. But that heuristic assumes the sources are sufficiently independent. If an attacker can place the same manipulated narrative across several pages, the agent may interpret coordinated repetition as independent confirmation.
More sources then become part of the attack rather than part of the defense.
Search depth does not guarantee evidence quality
One of the paper’s most important findings is that agentic search improves final resistance without necessarily improving evidence recognition or utility.
This distinction challenges a common assumption about deep-search agents. Giving a model additional search steps can help because it creates more opportunities to encounter clean evidence. But the agent still needs to interpret conflicts correctly.
An agent can browse extensively while continuing to privilege the wrong sources. It can also find contradictory evidence but fail to update the conclusion it formed earlier.
Search capability and epistemic reliability are therefore different properties.
A system that performs ten searches is not automatically safer than one that performs three. What matters is how it evaluates independence, credibility and contradiction among the results it finds.
Verification is not the same as recovery
HAE-GEO’s trajectory-based design exposes another gap that simpler benchmarks can miss: an agent can verify without recovering.
Verification means the system takes additional actions to check a claim. Recovery means those actions ultimately lead it away from the poisoned conclusion and back to the correct answer.
The researchers find that these stages frequently separate.
An agent may notice uncertainty, launch another search and read more pages. Those behaviors look encouraging in a trace. But if the newly retrieved evidence repeats the poisoned claim—or if the agent fails to weigh contradictory clean evidence strongly enough—the final answer can remain manipulated.
Counting verification actions alone can therefore overstate safety.
Defensive prompts make agents check more sources
The researchers also test whether explicit defensive prompting can improve resistance.
The idea is intuitive: instruct the agent to be skeptical, verify claims and seek independent evidence before committing to a recommendation. Such instructions are attractive because they can be deployed without retraining the model or rebuilding the search stack.
HAE-GEO finds that defensive prompting does change behavior. Agents verify more.
But the increase in checking rarely translates into successful recovery. The agent performs the requested safety procedure without reliably reaching a safer conclusion.
This is a familiar failure mode in AI evaluation: procedural compliance can look like robustness even when the downstream outcome remains wrong.
“Check multiple sources” is an incomplete safety rule
The corroboration result suggests that a better defensive instruction cannot simply ask the model to find more sources.
A robust agent needs to ask whether those sources are independent.
Three pages repeating the same unsupported statement may provide less evidence than one primary document contradicting all three. A reseller page, an affiliate comparison and an AI-generated review may also trace back to the same original claim even though their domains are different.
Source diversity therefore needs to be evaluated at a deeper level than URL count.
Agents may need signals about provenance, publication relationships, citation chains, ownership, temporal ordering and whether several documents are derivative versions of the same underlying information.
Without that context, the instruction to corroborate can be weaponized by anyone able to manufacture apparent consensus.
The benchmark focuses on consumer recommendation agents
The HAE-GEO setup is motivated by search-augmented LLMs increasingly used to make consumer decisions.
Product recommendations are a particularly relevant test case because users may delegate substantial research to an agent: comparing specifications, assessing brands, reading reviews and deciding which option best meets a need.
Those workflows also create economic incentives for manipulation. If a publisher can make an agent believe that one brand is superior, the resulting recommendation may influence purchasing behavior even when the user never sees the poisoned pages directly.
The agent becomes an intermediary between the web’s evidence ecosystem and the consumer.
That changes the threat model from conventional SEO spam. The attacker no longer necessarily needs to persuade a human reader. It may be enough to persuade the agent that performs the reading for them.
Deep research creates both resilience and a larger attack surface
Agentic search is often presented as an answer to the limitations of one-shot retrieval. If the model is uncertain, it can keep researching until it finds stronger evidence.
HAE-GEO shows why that capability is double-edged.
Additional search steps can expose the agent to clean information that helps it resist manipulation. But every additional retrieval is also another opportunity to encounter poisoned content.
If the information environment contains coordinated false corroboration, deeper browsing can reinforce the attack rather than correct it.
The relevant safety question is therefore not simply how much research the agent performs. It is whether the research policy increases the diversity and independence of evidence rather than repeatedly sampling the same manipulated narrative.
Agents may need to model evidence relationships, not just evidence relevance
Search systems are traditionally optimized around relevance: does this document answer the query?
Evidence poisoning introduces a second dimension: how should this document change confidence in a claim given the other documents already observed?
Two highly relevant pages are not necessarily two independent pieces of evidence. One may copy the other. Both may cite the same press release. They may be controlled by the same publisher or generated from the same source.
A deep-search agent that treats each retrieval as a separate vote can therefore become vulnerable to duplication and coordinated content networks.
Future defenses may need to build evidence graphs that represent provenance and dependency rather than merely ranking pages individually.
Recovery requires the agent to revise an adopted belief
Another reason verification can fail is that recovery is cognitively harder than detection.
Once an agent has incorporated a claim into its working answer, later evidence must do more than introduce uncertainty. It must cause the system to revise the internal conclusion guiding subsequent searches and synthesis.
This creates a form of path dependence. Early poisoned evidence can shape which follow-up queries the agent asks, which documents it considers relevant and how it interprets later contradictions.
A defensive system may therefore need explicit mechanisms for belief revision: identifying which claims are disputed, maintaining competing hypotheses and reopening conclusions when higher-quality evidence appears.
Simply adding “verify your answer” to the prompt may not be enough.
Final resistance can hide weak intermediate reasoning
The paper also warns against evaluating agents only on their final recommendation.
An agent may produce the correct final answer while demonstrating poor evidence recognition along the way. Another may recognize the poisoning correctly but fail to convert that insight into recovery.
Those systems have different failure modes even if a single end-point metric makes them look similar.
HAE-GEO combines deterministic behavioral measures with six semantic rubric dimensions to capture more of this trajectory.
That approach is valuable for safety research because defenses need to target the stage that actually fails. Retrieval filtering addresses exposure. Provenance analysis addresses recognition. Better revision policies address recovery. A single accuracy score cannot reveal which intervention is needed.
The study is controlled, not a measurement of the live web
The benchmark’s controlled corpus is a strength for causal evaluation but a limitation for real-world generalization.
The researchers know exactly which pages are clean and which are poisoned, and they can escalate the attack systematically across three levels. That makes it possible to compare agent behavior under repeatable conditions.
The open web is substantially messier. Search rankings change, sources disappear, commercial engines apply proprietary spam systems and information can be partially true rather than cleanly correct or poisoned.
Real attackers can also adapt to agent behavior, create new domains, exploit social platforms and coordinate across formats that a text-focused controlled corpus may not reproduce.
HAE-GEO therefore does not establish a universal poisoning rate for production ChatGPT, Gemini, Perplexity or other deep-search products. It demonstrates failure patterns under a benchmark designed specifically to expose the verification-to-recovery gap.
Ten agents provide breadth, but production systems keep evolving
Evaluating ten agents gives the study a broader foundation than a single-model demonstration.
The recurring patterns across systems suggest that the corroboration problem is not merely one model’s idiosyncrasy. Evidence recognition degrades under apparent consensus, agentic search helps final resistance more than it helps recognition, and defensive prompts increase checking without reliably creating recovery.
Still, deep-search architectures are evolving quickly. Commercial systems can combine proprietary ranking, browsing tools, source-quality classifiers, memory, hidden system prompts and specialized verification components that are difficult to reproduce in a public benchmark.
The paper should therefore be read as a stress test of an important class of agents, not a permanent ranking of current consumer products.
For GEO security, poisoning one page may be less powerful than manufacturing consensus
The hierarchical design points toward a strategic implication for defenders.
A single blatantly manipulated page is comparatively easy to challenge. Coordinated evidence is more dangerous because it exploits one of the core heuristics of research itself: independent confirmation increases confidence.
If generative-search manipulation matures, attackers may therefore focus less on making one page overwhelmingly persuasive and more on creating a network of superficially independent pages that tell the same story.
That would make conventional domain-level filtering less effective. The system would need to determine whether apparent agreement reflects genuinely independent evidence or coordinated publication.
The corroboration trap turns web-scale duplication from a quality problem into an AI-agent security problem.
The next generation of defenses needs to optimize for recovery
HAE-GEO’s strongest contribution is the distinction between checking and correcting.
A safety layer can make an agent more cautious. A prompt can tell it to verify. A search tool can give it more opportunities to browse. None of those mechanisms guarantees that the agent will abandon a poisoned conclusion once it has formed one.
That suggests future benchmarks and defenses should treat recovery as a first-class objective.
A resilient agent should recognize conflicting evidence, identify which sources are independent, assign greater weight to stronger provenance and explicitly revise earlier conclusions when warranted. It should also know when the information environment is too contaminated to support a confident recommendation.
Those capabilities go beyond “search more.” They require the agent to reason about the structure of evidence.
Deep search is not the same thing as deep verification
The promise of deep-search agents is that they can perform the research work a human would otherwise do manually. HAE-GEO shows why browsing depth alone does not deliver that promise.
The ten evaluated agents operate across a controlled environment of 72,039 clean pages and 770 poisoned pages per attack level covering 154 brands. As poisoning becomes more persuasive, particularly when multiple sources appear to corroborate the same false claim, their evidence recognition weakens.
Defensive prompts encourage more verification, but verification rarely becomes recovery. Agentic search improves final resistance without reliably improving the agent’s ability to recognize poisoned evidence or extract greater utility from its investigation.
For AI-search developers, that is a more demanding safety target than simply adding another fact-checking step. The agent must not only look for more evidence. It must understand when several sources are actually one story repeated many times—and be willing to reverse itself when the evidence it trusted first turns out to be wrong.