Hopefully you’ve heard of IndexNow by now.
we have it covered announcement In October and December, I invited Fabrice Canel, Principal Program Manager at Microsoft Bing, to on the SEJ show Talk about this revolution in how search engines discover new and updated content.
But what exactly is it, and what do you need to know about SEO?
Here’s what we know so far.
Why we need IndexNow
Search engines have long discovered content in the same way.
Back in 1993, jumping station Be the first Internet discovery tool to send bots to follow (or crawl) links from one website to another and create a map or index of content.
Over the next few years, other search engines adopted this process as a way to find new content.
This process is contained in an algorithm that also takes into account content, relevance, and other factors — and what we know as SEO.
It didn’t take long for SEO-savvy web publishers to realize the importance of indexing in ensuring high search engine rankings. In fact, without indexing content, a website would not generate organic traffic.
But here’s the catch: Unless you’re a large, high-traffic site, it can take days or even weeks for search engines to crawl your site and index new pages. Heck, even if you’re a large e-commerce site or marketplace, it can take weeks for engines to fetch newly arrived or out-of-stock products – especially when relying on basic scraping for discoverability.
At the same time, you may have lost significant traffic to your competitors.
To simplify the process, Microsoft Bing released a new initiative called IndexNow in October 2021.
So what is IndexNow?
IndexNow is an open source protocol that allows website publishers to instantly index in participating search engines, updating results based on the latest content changes.
In short, it’s a simple ping that informs search engines that a URL and its contents have been added, updated, or removed.
By eliminating the need for exploratory crawls, IndexNow aims to improve overall internet efficiency.
How does IndexNow work?
Search engines obtain index data in two different ways: pull and push.
Pull indexing occurs when search engines visit your website to request pages and collect data from servers.
This is how search engines have traditionally worked.
Push indexing is when a web publisher or content management system notifies search engines that pages have been added or removed or other changes have been made.
This is where IndexNow is truly groundbreaking – it allows all URLs submitted to any IndexNow-enabled search engine to be simultaneously submitted to all other search engines using the protocol.
You can read more about the technical aspects How IndexNow works here.
Which search engines use IndexNow?
At launch, Bing and Yandex were using IndexNow.
A small number of people are also reportedly testing the protocol, according to an unnamed expert.
So far, DuckDuckGo has not announced any plans to adopt IndexNow.
It’s worth noting, however, that in addition to using its own DuckDuckBot web crawler to compile search results, the privacy-focused search engine draws information from as many as 400 sources.
in view of Bing and Yandex are the sourcesyou can also see updated content appearing in DuckDuckGo faster thanks to IndexNow.
Even Bing’s biggest competitor, Google, the most important of all search engines, has Confirm that they will test the IndexNow protocol.
While they haven’t announced whether they’ll adopt it, the fact that they’re experimenting with it shows a nod to its capabilities.
Google’s biggest concerns with IndexNow appear to be related to sustainability and efficiency.
Google is currently using HTTP/2, an efficient underlying data transfer protocol for more than half of their crawls, and may eventually decide this is more suitable for their needs.
Or, if they decide they need an API-based approach, they might decide to build their own alternative — and it’s not a foregone conclusion that they will.
But not everyone is involved.
There are many in the SEO community who argue that this protocol is not needed, claiming that XML sitemaps already do what IndexNow is trying to do.
You can read more about the debate here.
How do you use IndexNow?
Using IndexNow to notify search engines of your web content updates is a simple process:
- Generate API key – This is submitted with the URL to ensure domain ownership.You can use an online key generator tool to do this, e.g. found here.
- Host API key – Your API key is hosted in the root directory in txt format.
- Submit URL with parameters – You can submit URLs individually or in bulk. Submit your key location as a parameter.
- Check your submission – Using the Bing Webmaster Tools Portal, you can verify which URLs have been submitted and discovered.
For examples and more in-depth explanations, you can visit IndexNow website.
More advanced users can use Python for bulk indexing and URL submission automation.
This requires a basic Understanding of Python syntax Familiarity with Python libraries and packages.
We have a handy step by step A guide to walk you through the process here.
Which tools sync with IndexNow?
Any web developer can take advantage of IndexNow using the process outlined here, but what’s really exciting is that you don’t need to know HTML for HDMI to get the benefits.
Many large companies, including LinkedIn, MSN, and GitHub, are planning to migrate to IndexNow.
In addition, other search solution providers such as BotoxOnCrawl, Onely and Yext are adopting the protocol.
Content management systems (CMS) are also joining.
WordPress already provides a IndexNow plugin. Duda Support This proactive approach to web scraping, Wix plans to incorporate soon.
Bing recently announced Rank Math and One stop search engine optimization IndexNow integration, as well.
Content Delivery Networks (CDNs) that support IndexNow include Cloudflare and Akamai.
Through their proxy server, this will speed up the data transfer and bring it closer to the user.
What are the benefits of using IndexNow?
In addition to highlighting the evolution of indexing from pull to push, the main benefit of IndexNow for publishers is the elimination of the time between making updates and search engines discovering them.
By allowing webmasters to notify all participating URLs with a single API call, search engine content discovery is simplified, making the web more efficient.
This is good for site owners as there is less growing demand on the server.
Search engine spiders no longer need to perform exploratory crawls to determine if a page has been updated.
This enables servers to work more efficiently without redundant site loads, reducing energy usage.
Unlike XML sitemaps that only redirect SEO crawlers to pages within your site, IndexNow allows you to notify search engines of non-200 status code pages.
IndexNow FAQ
What happens after submitting the URL?
IndexNow communicates your changes to search engines, which then apply them to page rankings.
Using IndexNow does not guarantee that pages will be crawled or indexed, and it may take some time for changes to reflect.
How many URLs can I submit with IndexNow?
You can provide search engines with lists of up to 10,000 URLs with a single API call.
Will I rank better in search results if I index my content faster?
Yes, the sooner your pages are indexed, the sooner they can start competing for premium search engine positions.
What does it mean?
Advances in technology are unstoppable.
As such an important part of the modern world, changes to the internet, and search engines in particular, will naturally be a part of it.
IndexNow appears to be on the cusp of a revolution by transforming the process of indexing sites from crawling to a faster, more efficient process.
And the fact that companies like MSN, GitHub, and WordPress have all joined seems to suggest that the protocol will soon become the norm.
More resources:
Featured image: ra2 studio/Shutterstock
