Google made a small but technically important change to its crawling documentation on September 17, 2026: Mediapartners-Google should no longer be understood simply as the AdSense crawler. Google now explicitly says that crawling preferences addressed to this user agent can affect multiple advertising-related Google products, with both Google AdSense and Google Ad Manager given as examples.
The change is documented in Google's official Crawling Documentation Changelog. Google describes the update as a generalization of the Mediapartners-Google documentation and says it renamed the relevant section from “AdSense” to “Mediapartners-Google.” The reason is unusually clear: rules addressed to this crawler affect several Google advertising products rather than AdSense alone.
The crawler itself is not new — Google's definition is broader
This is not the launch of a new crawler and Google is not announcing a new user-agent token. The technical identifier remains Mediapartners-Google. What changed is the documentation around its scope, correcting an interpretation that could lead publishers and infrastructure teams to treat the crawler as relevant only to AdSense.
Google's current special-case crawler documentation says that Mediapartners-Google visits participating sites to provide relevant ads. More importantly, the “Affected products” section now states that crawling preferences directed at the Mediapartners-Google user agent affect various ad-related Google products, explicitly naming Google AdSense and Google Ad Manager as examples.
robots.txt rules can therefore have wider advertising consequences
The practical issue is robots.txt configuration. Google documents Mediapartners-Google as a special-case crawler with its own user-agent token, meaning publishers can write rules specifically for it. A site might, for example, allow or disallow selected paths for Mediapartners-Google independently of the rules intended for other crawlers.
After the September 17 clarification, those rules should be reviewed with a wider product scope in mind. A publisher that previously interpreted a Mediapartners-Google directive as an AdSense-only choice may actually be influencing crawling used by other Google advertising products as well. Google Ad Manager is the concrete additional example Google now provides.
That does not mean every Google advertising product necessarily uses the crawler in exactly the same way, nor does the documentation enumerate every affected product. Google's wording is intentionally broader: it says “various ad-related Google products” and gives AdSense and Google Ad Manager as examples. It would therefore be inaccurate to turn the documentation update into an exhaustive list that Google itself has not published.
Mediapartners-Google has unusual robots.txt behavior
There is another technical detail worth noting. In the special-case crawler documentation, Google states that the global User-agent: * group is ignored for Mediapartners-Google. Sites that want to specify crawling preferences for this crawler should therefore use the dedicated Mediapartners-Google token rather than assuming a global rule will necessarily govern it in the same way as ordinary crawlers.
This is one reason seemingly minor documentation changes can matter operationally. robots.txt files are often inherited over years, copied between properties or maintained by teams whose original rationale for a rule has disappeared. A directive added long ago under the assumption that it controls an “AdSense bot” may now deserve a second look if the site also relies on Google Ad Manager or other Google advertising infrastructure.
This is a crawling clarification, not a Google Search ranking change
The update should also be kept separate from Google Search crawling and ranking. Mediapartners-Google appears in Google's special-case crawler documentation because it serves advertising-related purposes. Googlebot remains the crawler associated with Google Search and other search products. Nothing in the September 17 changelog says that changing Mediapartners-Google rules will alter organic Search rankings.
That distinction is especially important because Google's crawler documentation now lives in a broader crawling-infrastructure area rather than exclusively inside Search Central. When Google migrated the documentation in November 2025, it explained that its crawling infrastructure is shared across many products beyond Search, including Shopping, News, Gemini and AdSense. The new Mediapartners-Google clarification is a good example of why that broader framing matters: a crawler name or rule can belong to an ecosystem larger than the product with which publishers historically associated it.
Publishers should audit assumptions, not blindly rewrite robots.txt
The appropriate response is not to rush out and change every robots.txt file containing Mediapartners-Google. Existing directives may be intentional and correct. The useful action is to audit the assumption behind them. If a rule was created specifically because a team wanted to control AdSense crawling, the team should now consider whether the same rule is appropriate for Google Ad Manager and the broader set of advertising products covered by Google's updated definition.
Publishers should also document why dedicated crawler rules exist. A technically valid robots.txt directive can become a business problem when its effect is broader than the person maintaining it realizes. Google's September 17 update makes that risk easier to see by explicitly connecting Mediapartners-Google to multiple advertising products.
This is a micro-update rather than a major platform announcement, but it is exactly the kind of documentation change that can alter how an old technical configuration should be interpreted. Mediapartners-Google is not merely shorthand for “the AdSense crawler.” Google now wants publishers to understand it as an advertising crawler whose rules can reach across products — including Google Ad Manager — and robots.txt audits should reflect that broader scope.