Websites have spent roughly three decades being designed around one assumption: the visitor is a person.

That assumption is starting to break.

AI agents can already browse websites, query APIs and perform multi-step tasks. Cloudflare is now working on infrastructure that goes further, giving agents persistent identities and programmable wallets that could allow them to transact online within limits established by their owners.

Cloudflare calls the broader idea the Agentic Internet: a web that needs to become readable, discoverable, callable and payable not only for humans, but for software acting on their behalf.

For publishers and website owners, however, the search implications may be even more interesting.

Cloudflare and OpenAI announced a research pilot in July exploring whether network-level signals can help AI search systems discover and index web content more efficiently. The signals being tested include content freshness, actual page changes and traffic quality.

That is worth watching closely.

Traditional search discovery has largely revolved around crawling pages and deciding when they should be crawled again. An AI-heavy web creates a different problem: machines are both discovering information and increasingly becoming visitors themselves.

Cloudflare says more than half of AI crawler traffic it observes involves re-fetching pages that have not changed. Knowing that a page actually changed could therefore become a useful discovery signal rather than repeatedly crawling everything just to find out.

This also changes how we should think about AI visibility. The question may eventually move beyond “Can an AI cite my website?”

It could become: Can an agent understand the site, identify what changed, determine what it can do there and interact with it successfully?

That is why we included AI agents as website users among the research directions in our new public AI visibility experiments repository.

Search was built around humans looking for webpages. The next version of the web may also need to work for machines looking for information — and increasingly, looking to act.

Sources: Cloudflare's Agentic Internet research and its research pilot with OpenAI.
Cloudflare–OpenAI research pilot · Cloudflare's Agentic Internet article