Google Search Console allows you to see your website through Google’s eyes.
You get information about site performance and details about page experience, security issues, crawling or indexing.
The Exclusions section of the Google Search Console Index Coverage report provides information about the indexing status of your site’s pages.
Learn why certain pages of your site are showing up in the Exclusions report in Google Search Console, and how to fix it.
What is an index coverage report?
This Google Search Console Coverage Report Displays detailed information about the indexing status of your site’s pages.
Your web page can go into one of four buckets:
- mistake: pages that Google cannot index. You should review this report because Google thinks you might want to index these pages.
- warning is valid: Pages indexed by Google, but there are some issues you should fix.
- Effective: Pages indexed by Google.
- Excluded: Pages to exclude from indexing.
What are excluded pages?
Google does not index pages in the error and exclusion buckets.
The main differences between the two are:
- Google thinks pages in Error should be indexed, but not because of errors you should be looking at. For example, a non-indexable page submitted via an XML sitemap is an error.
- Google believes that pages in the Excluded bucket should indeed be excluded, which is your intent. For example, non-indexable pages that are not submitted to Google will appear in the Exclusions report.
Screenshot of Google Search Console, May 2022
However, Google doesn’t always get it right, and pages that should be indexed are sometimes excluded.
Fortunately, google search console Provides a reason for placing the page in a specific bucket.
That’s why it’s a good practice to double-check the pages in all four buckets.
Now let’s dive into the excluded buckets.
Possible reasons for excluded pages
There are 15 possible reasons why your page is in the exclusion group. Let’s take a closer look at each one.
Excluded by “noindex” tag
These are URLs with the “noindex” tag.
Google thinks you actually want to exclude these pages from the index because you didn’t list them in XML sitemap.
For example, these might be landing pages, user pages, or search results pages.
Suggested action:
- Check out these URLs sure you Want to exclude them from Google’s index.
- Check if the “noindex” tag still/actually exists on these URLs.
Crawled – not currently indexed
Google has crawled these pages, but has not yet indexed them.
as google documentthe URL in this bucket “may or may not be indexed in the future; there is no need to resubmit this URL for crawling.”
Many SEO professionals have noticed that if many normal and indexable pages are crawled (which are not currently indexed), a website can have some serious quality issues.
This could mean that Google has crawled these pages and doesn’t think they provide enough value to index.
Screenshot of Google Search Console, May 2022Suggested action:
- Review your website for quality and eat.
Discovered – not currently indexed
Pages under “Discovered” – not currently indexed “have been found by Google, but have not been crawled”, as the Google Docs states.
Google didn’t crawl the page to avoid overloading the server.A large number of pages under this bucket could mean that your website Crawl budget question.
Screenshot of Google Search Console, May 2022Suggested action:
- Check the health of the server.
Not found (404)
These are pages that return a status code of 404 (Not Found) when requested by Google.
These are not URLs submitted to Google (that is, in XML sitemap), but instead Google found those pages (i.e. via another site linking to an old page that was deleted a long time ago.
Screenshot of Google Search Console, May 2022Suggested action:
- Check out these pages and Decide whether to implement a 301 redirect to a job page.
Soft 404
Soft 404in most cases, is an error page that returns a status code of OK (200).
Alternatively, it can be a thin page containing almost nothing and using words like “sorry”, “error”, “not found”, etc.
Screenshot of Google Search Console, May 2022Suggested action:
- In the case of an error page, Make sure to return status code 404.
- For thin pages, Add unique content Help Google recognize this URL as a standalone page.
page with redirect
All redirecting pages on your site will go to the exclusion bucket, where you can view all redirecting pages Google has detected on your site.
Screenshot of Google Search Console, May 2022Suggested action:
- View redirected pages Make sure redirects are implemented intentionally.
- Some WordPress plugins automatically create redirects You may need to check these from time to time as you change URLs.
No duplication of user-selected specifications
Google considers these URLs to be duplicates of other URLs on your site and should not be indexed.
You did not set canonical labels for these URLs, Google chose canonical labels based on other signals.
Suggested action:
- Check these URLs to check which canonical URLs Google has chosen for these pages.
Repeat, Google chose a different specification than users
Screenshot of Google Search Console, May 2022In this case you declared a Canonical URL For this page, but even so, Google chose a different URL as the canonical. Therefore, the specs selected by Google are indexed, but the ones selected by the user are not.
Possible actions:
- Check URL to check canonical Google Choice.
- Analyze possible signals that drive Google’s choice Different specifications (ie external links).
Duplicate, submitted URL not selected as canonical
The difference between the above state and this state is that in the latter case, you submit a URL to Google for indexing without declaring its canonical address, and Google thinks a different URL would be a better canonical one.
Therefore, the canonical selected by Google is indexed, not the submitted URL.
Suggested action:
- Check URL to check canonical Google has chosen.
Alternate pages with appropriate canonical tags
These are just copies of pages that Google recognizes as canonical URLs.
These pages have canonical addresses that point to the correct canonical URL.
Suggested action:
- In most instances, No action required.
Blocked by bots.txt
these are robot.txt Blocked.
When analyzing this bucket, keep in mind that if Google finds references to these pages on other sites, Google can still index these pages (and display them in a “compromised” way).
Suggested action:
- Verify that these pages are not blocked Use the robots.txt tester.
- Add “noindex” tag and remove page from robots.txt if you want to remove them from the index.
Blocked by Page Removal Tool
This report lists pages that have been requested to be removed removal tool.
Keep in mind that this tool will only temporarily (90 days) remove pages from search results, it will not remove them from the index.
Suggested action:
- Verify that the page was submitted via the removal tool Should temporarily remove or have the “noindex” tag.
Blocked due to unauthorized request (401)
For these URLs, Googlebot was unable to access the pages due to an authorized request (401 status code).
You do not need to do anything unless these pages are available without authorization.
Google just tells you what it encounters.
Screenshot of Google Search Console, May 2022Suggested action:
- Verify that these pages really require authorization.
Blocked because access is forbidden (403)
this status code Usually the result of some server error.
A 403 is returned when incorrect credentials are provided and access to the page cannot be granted.
as Google Docs state:
“Googlebot never provides credentials, so your server returned this error incorrectly. This error should be fixed, or the page should be blocked by robots.txt or noindex.”
What can you learn from excluded pages?
A sudden huge spike in a specific bucket of excluded pages could indicate a serious website issue.
Here are three examples of spikes that could indicate serious problems with your site:
- Huge spikes in not found (404) pages may indicate unsuccessful migrant where the URL has changed, but redirection to the new address has not yet been implemented. This can also happen when, for example, an inexperienced person changes the slug of a blog post and therefore changes the URLs of all blogs.
- a huge peak Discovered – not currently indexed or crawled – Not currently indexed may indicate that your site has been hacked. Be sure to check out the sample pages to check if these are really yours, or if they were created by hackers (i.e. pages with Chinese characters).
- Huge spikes excluded by “noindex” labels may also indicate unsuccessful startup and migration. This usually happens when a new site goes into production with a staging site’s “noindex” tag.
review
With the exclusions section of the GSC Coverage report, you can learn a lot about your site and how Googlebot interacts with it.
whether you are new search engine optimization Or if you have several years of experience, make it a daily habit to check Google Search Console.
This can help you detect various technical SEO problems before they turn into real disasters.
More resources:
Featured image: Milan 1983/Shutterstock
