Bing has announced that it is changing the user-agent string that identifies itself as Bingbot. There will now be two user agents, one for the desktop and one for the mobile crawler.
The new user agent provides more information, including the latest version of Edge.
user agent
A user agent is a message from software that tells the web server what software device is accessing it.
A user agent can tell a website whether it is a browser or a search engine bot.
Some website software and SaaS applications rely on user agents to identify search bots and perform actions such as blocking them from accessing search or registration pages.
Bing user agent change is a transition
Microsoft will continue to use the old bingbot crawler user agent until fall 2022, a vague date.
However, the Bing Webmaster Tools URL Inspection tool has switched to the new bingbot user-agent.
Some apps and software may need to be updated, so this gave them a few months to update their software so they could identify bingbot accurately.
But there is no need to update robots.txt, as new versions can continue to be addressed as bingbot in robots.txt.
Robots.txt does not need to be updated to accommodate the new bingbot user-agent string.
However, for software and services that do partial matching, such as phpBB, the code does not need to be updated to accommodate the new user agent.
The code will continue to work if the publisher, application or software uses bingbot/ to identify bingbots with partial maps.
old and new user agent
Here is the old bingbot user agent:
Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
The new bingbot user-agents are an improvement, as they recognize the desktop and mobile versions of the Microsoft Bing web crawler.
These are the new bingbot user agents:
desktop
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) Chrome/W.X.Y.Z Safari/537.36
mobile
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
According to the official Microsoft Bing announcement:
“We will carefully test sites before switching them to our new user agent Bing Webmaster Tools URL Check has started live URL testing with the new desktop user agent to help you investigate potential issues.
If you encounter any challenges, feel free to contact us. “
