A preregistered field experiment with 1,100 Google users has produced unusually strong causal evidence for a concern publishers have mostly had to measure through observational traffic data: forcing users into Google AI Mode substantially reduces the likelihood that they click through to the wider web.
The study, submitted to arXiv on August 18, randomly assigned participants to one of three Google Search experiences: normal Search, Search with AI features suppressed, or a condition that redirected searches into AI Mode. Over the seven-day experimental period, assignment to AI Mode reduced external click-through rate by 18.8 percentage points compared with current Google Search. Exposure to the no-AI treatment moved in the opposite direction, increasing external click-through by an estimated 8.8 points.
The experiment also complicates the argument that lower outbound traffic is simply the price users willingly pay for a better search experience. Participants assigned to AI Mode conducted fewer daily search sessions, reported lower trust in information found on Google, and rated the experience lower on usefulness, satisfaction, agency and personalization or relevance. They were also 11.2 percentage points more likely to use a competing search engine such as Bing, DuckDuckGo or Yahoo.
The results are consequential, but the limits matter. The treatment lasted only seven days, participants were U.S.-based Chrome users who already used Google as their primary search engine, and the sample skewed younger and more educated than the general population. More importantly, a Google HTML change during the experiment partially broke the researchers' attempt to hide AI Overviews in the no-AI condition. The authors account for that failure statistically, but it makes the cleanest causal evidence stronger for forced AI Mode than for complete removal of Google's AI features.
This was a randomized field experiment, not another traffic correlation
The paper, titled “AI in Search Reduces Publisher Referrals Without Improving User Experience: Experimental Evidence,” was authored by Stephanie T. Wang, Jeffrey Gleason, Yakov Bart, Christo Wilson and Danaé Metaxa, with affiliations at the University of Pennsylvania and Northeastern University. The researchers preregistered their hypotheses and ran the intervention through a browser extension while participants used Google in their everyday browsing.
That design distinguishes the study from the growing number of SEO analyses that compare click-through rates for queries with and without AI Overviews. Observational studies can identify strong associations, but queries that trigger AI features may differ systematically from queries that do not. A randomized experiment can isolate the effect of the search experience itself more directly.
Participants first spent three days in a baseline period using current Google Search. They were then randomly assigned for seven days to Current Search, No AI Search or AI Mode Search. Current Search made no changes. The no-AI treatment attempted to hide AI Overviews and redirect AI Mode searches back to general search. The AI Mode treatment redirected all Google searches into AI Mode.
The researchers initially enrolled 1,444 people. Following the preregistered analysis plan, behavioral treatment effects were estimated among the 1,100 participants who made at least one search during the experiment and were eligible for the post-study survey. A total of 956 completed that survey.
Forced AI Mode reduced external click-through by 18.8 percentage points
The headline publisher result is unusually clear. Assignment to AI Mode reduced click-through to external websites by 18.8 percentage points relative to Current Search, with a 95% confidence interval from -22.2 to -15.3 points and a p-value below 0.001.
This is a percentage-point treatment effect, not an 18.8% relative decline. That distinction is important when translating the finding into headlines or forecasting traffic. The experiment estimates how much the probability of an external click changed under the treatment compared with the control experience; it does not say every publisher would lose exactly 18.8% of Google visits if AI Mode became universal.
The effect also extended across several types of external destination. Assignment to AI Mode reduced the fraction of users clicking news sites by 12.5 percentage points, Reddit by 21.2 points and Wikipedia by 9.9 points. Ad clicks fell by 42.7 points, although the authors note that AI Mode did not surface ads at the time of the experiment, making that result highly dependent on the product configuration tested.
The cross-domain pattern matters because it suggests the traffic effect was not confined to one category of website. AI Mode kept more attention inside the generated search experience while reducing visits to multiple kinds of external sources.
Removing AI features increased external CTR by an estimated 8.8 points
The no-AI condition produced the inverse result: exposure to Search without the AI features increased external click-through by an estimated 8.8 percentage points, with a 95% confidence interval of 2.3 to 15.3 points. The effect was statistically significant at p=0.008.
This estimate needs more methodological care than the AI Mode number. The browser extension successfully hid about 90% of AI Overviews on the first intervention day, but Google changed the HTML used for AI Overviews while the study was running. That broke the extension's detection logic, and its success rate eventually fell to zero.
Across the experiment, only 51.1% of AI Overviews were successfully hidden. The median participant in the no-AI condition had half of their AI Overviews blocked. Because the preregistration anticipated possible non-compliance, the researchers report local average treatment effect estimates based on actual exposure to the intervention rather than treating assignment as equivalent to complete AI removal.
The correct interpretation is therefore not that a perfectly AI-free version of Google definitively raises CTR by exactly 8.8 points for every user. It is that, within this experiment and the authors' compliance-adjusted model, successful exposure to the no-AI intervention increased external click-through by that estimated amount.
Google itself changed during the experiment
The HTML failure is more than a technical footnote. It illustrates a recurring problem for independent research on live platforms: the treatment can change underneath the researchers. A browser extension may be carefully tested and preregistered, only for a product update to alter the DOM structure days later.
In this case, the failure weakened treatment compliance for AI Overview removal but did not similarly undermine the AI Mode intervention. The researchers report that 94.7% of searches in the AI Mode group were successfully routed to AI Mode, making the forced-AI condition substantially closer to the intended experimental treatment.
That asymmetry should shape how the results are discussed. The 18.8-point AI Mode estimate is an intent-to-treat effect from randomized assignment with high compliance. The 8.8-point no-AI estimate depends more heavily on the statistical adjustment for partial compliance. Both are reported by the paper, but they are not methodologically identical.
AI Mode reduced daily search sessions instead of increasing them
The authors preregistered a hypothesis that AI Mode would increase the number of search sessions, reflecting Google's public argument that AI features can encourage users to ask more and more complex questions. The experiment found the opposite.
Assignment to AI Mode reduced search sessions by 0.92 sessions per day relative to Current Search, with a 95% confidence interval from -1.30 to -0.55 and p<0.001. The decline was especially pronounced among heavy Google users, defined in the study as people with more than four daily sessions before treatment.
At the same time, AI Mode increased the length of each session by 0.43 minutes. Users therefore spent somewhat longer inside an individual AI-mediated session while starting fewer search sessions overall and clicking out to the broader web less often.
That combination is strategically important. A search product can increase time spent inside a session without increasing overall search frequency or external exploration. Engagement metrics that focus only on session duration could therefore tell a very different story from publisher referral metrics.
More users went to Bing, DuckDuckGo or Yahoo
The experiment also tested substitution behavior. Assignment to AI Mode increased the fraction of participants who searched on a competing engine—Bing, DuckDuckGo or Yahoo—by 11.2 percentage points. The 95% confidence interval ranged from 6.4 to 16.0 points, with p<0.001.
Participants assigned to AI Mode also reported a higher intention to switch to Bing, increasing by 1.25 points on a seven-point scale. By contrast, the researchers did not detect evidence that the no-AI treatment changed use of competing engines or long-term switching intentions.
This does not mean AI Mode will cause an 11.2-point loss in Google's market share. The experiment deliberately forced one group to use AI Mode for nearly all Google searches, which is much more aggressive than the optional experience many users encounter in normal conditions. The result instead shows that under this forced treatment, some users responded by seeking a more conventional search experience elsewhere.
Trust fell rather than rising
Assignment to AI Mode reduced reported trust in information found on Google by 0.34 points on a seven-point scale. The result was statistically significant, with a 95% confidence interval from -0.47 to -0.20.
The broader experience measures moved in the same direction. AI Mode reduced perceived usefulness by 0.59 standard deviations, satisfaction by 0.73 standard deviations and agency by 0.66 standard deviations. Perceived personalization and relevance fell by 0.42 standard deviations, contrary to the researchers' preregistered prediction that AI Mode would improve it.
Exposure to the no-AI treatment, meanwhile, produced no detectable change in trust, usefulness, satisfaction, agency or personalization. Within the limits of the experiment, removing AI features increased outbound clicking without producing a measurable deterioration in these reported experience outcomes.
That finding directly addresses the publisher-versus-user tradeoff often invoked in debates about zero-click search. If users were substantially happier with AI Mode, reduced referrals could be interpreted as the result of a product delivering more value directly. This experiment did not find that compensating improvement over the one-week treatment period.
Participants' comments reveal why forced AI Mode could frustrate users
The researchers qualitatively coded open-ended feedback from 309 participants in the AI Mode condition. Negative sentiment was the largest single category at 33.6%, compared with 29.3% positive and 13.7% mixed.
Loss of control and agency appeared in 17.6% of responses, while 15.3% mentioned difficulty navigating to specific websites. Another 13.4% cited limited links or source diversity. Smaller groups complained about overly verbose answers or raised accuracy and hallucination concerns.
Positive feedback was real as well. Fourteen percent mentioned efficiency or time savings, and 6.2% praised summarization. The qualitative results therefore do not show universal rejection of AI search. They show a split experience in which convenience for some users coexisted with frustration among others, particularly when AI Mode was imposed on searches that might otherwise have been simple navigational tasks.
Forced AI Mode is not the same as today's voluntary AI Mode usage
This distinction is essential for interpreting the study. During the baseline period, participants used AI Mode for only 0.6% of their searches. In the treatment group, the extension attempted to route essentially every Google search into AI Mode and succeeded 94.7% of the time.
The experiment therefore estimates the effect of a dramatically more AI-centric Google experience, not the average impact of the current mix in which users can still perform conventional searches and selectively enter AI Mode. That is precisely what makes it useful for thinking about a possible future in which conversational search becomes more central, but it limits claims about today's aggregate Google traffic.
The authors frame AI Mode as Google's direction of travel and study what happens when that interface becomes the default treatment. Publishers should treat the results as evidence about product design and user behavior under that condition, not as a direct measurement of current real-world AI Mode market penetration.
The sample also limits generalization
Participants were recruited in the United States through Prolific and a Northeastern University work-study program. Eligibility required being at least 18, using Chrome as the primary browser and using Google as the primary search engine.
The final analysis sample skewed younger and highly educated: 75% were under 45 and 87% had at least some college education. The authors also report that 58% identified as Democrats and 20% as Republicans. Gender and racial distributions were closer to U.S. Census patterns.
These characteristics do not invalidate randomization inside the experiment, but they affect external validity. Search behavior among older users, mobile-first users, non-U.S. populations, Safari users or people who already prefer alternative search engines could differ. The treatment also lasted only one week, leaving open the possibility that behavior changes as people adapt to AI Mode over longer periods.
The experiment challenges simplistic claims about “better clicks”
Google has argued publicly that AI features can create higher-quality visits by helping users understand a topic before they click. This study does not directly measure conversion quality after a referral, so it cannot confirm or refute that specific claim.
What it does establish is that the number of opportunities for external visits can fall substantially under AI Mode. A publisher could theoretically receive fewer but more qualified clicks, but those clicks would need to become sufficiently more valuable to offset the reduction in referral probability.
For ad-supported publishers, where pageviews themselves contribute directly to revenue, that threshold may be difficult. For ecommerce, SaaS or lead-generation sites, the economics could be different if AI referrals convert at materially higher rates. The study measures traffic behavior, not downstream revenue per visit.
This is why SEO reporting should avoid collapsing click volume and click quality into one claim. The experiment provides causal evidence about the first. The second requires conversion and revenue data from the receiving websites.
The results make citation visibility an incomplete publisher metric
Generative search has encouraged a new generation of visibility metrics focused on whether a brand or publisher is cited inside AI answers. This experiment demonstrates why citation presence alone cannot describe economic impact.
A source can contribute information to an AI-generated answer without receiving a visit. If the interface satisfies the user's need on Google, the publisher may gain attribution or brand exposure while losing the click that traditionally funded content production or moved the visitor into a conversion funnel.
Publishers therefore need to measure AI-era search across several layers: inclusion as a source, prominence of the citation, outbound click-through, quality of the resulting visit and eventual commercial outcome. A rise in citations can coexist with a fall in referrals.
The strongest finding is the gap between traffic loss and user benefit
The paper's title emphasizes that AI in search reduced publisher referrals “without improving user experience,” and the experimental results support that framing for the outcomes measured during this seven-day treatment. Forced AI Mode produced a large decline in external CTR while trust, satisfaction, usefulness, agency and perceived relevance also declined.
At the same time, the study should not be stretched beyond its design. It does not prove that every future version of AI Mode will produce the same effects. Google can change the interface, citation design, model quality, speed and integration with traditional results. Users can also adapt as conversational search becomes more familiar.
What the experiment provides is a rigorous snapshot of a specific product configuration in March 2026. Under those conditions, forcing Google users into AI Mode reduced external click-through by 18.8 percentage points, cut daily search sessions and sent more participants to rival engines. Successful exposure to the no-AI intervention increased external CTR by an estimated 8.8 points, although a live Google HTML change weakened that treatment.
For publishers and SEO teams, the implication is more substantial than another zero-click correlation. Search-interface design can causally redistribute attention away from the open web, and that redistribution does not automatically come with a measurable improvement in user trust or satisfaction. As Google moves further toward AI-mediated search, the question is no longer only which pages its systems cite. It is how much of the journey still requires users to visit those pages at all.