Google says that the best way to hide a website from search results is to use a password, but there are other options you can consider.
The latest Ask Googlebot video series on YouTube highlights this topic.
Google’s John Mueller answered a question asking how to prevent content from being indexed in search and whether to allow websites to do so.
“In short, yes, you can,” Mueller said.
There are three ways to hide a website from search results:
- Use password
- Stop crawling
- Block index
advertise
Keep reading below
Sites can choose not to be indexed at all, or they can use a password to index the content and hide it from Googlebot.
Blocking content from Googlebot does not violate the webmaster guidelines, as long as users are blocked from accessing at the same time.
For example, if a website is password protected when crawled by Googlebot, it must also password protect users.
Or, the website must have instructions to prevent Googlebot from crawling or indexing the website.
If your website provides different content to Googlebot than to users, you may be in trouble.
advertise
Keep reading below
This is the so-called “disguise” and violates Google’s guidelines.
With this distinction, the following is the correct way to hide content from search engines.
3 ways to hide content from search engines
1. Password protection
If you want to keep your website private, using a password to lock the website is usually the best way.
The password will ensure that neither search engines nor random web users can see your content.
This is a common practice for websites under development. Publishing a website in real time is an easy way to share work in progress with customers, while preventing Google from accessing websites that are not yet ready.
2. Block crawling
Another way to prevent Googlebot from accessing your site is to prevent crawling. This is done through the robots.txt file.
Using this method, people can visit your website via a direct link, but they won’t be selected by search engines that are “well performing”.
Mueller said this is not the best option, because search engines may still index website addresses without accessing the content.
This happens rarely, but you should be aware of this possibility.
3. Block Index
The third and final option is to prevent your website from being indexed.
advertise
Keep reading below
To do this, you add a noindex robots meta tag to the page.
The noindex tag tells search engines not to index the page until Rear They grab it.
The user cannot see the meta tag and can still access the page normally.
Mueller’s final thoughts
Mueller concluded the video saying that Google’s highest recommendation is to take the password route:
“In general, for private content, our recommendation is to use password protection. It’s easy to check if it’s working properly, and it can prevent anyone from accessing your content.
When the content is not private, blocking crawling or indexing is a good choice. Or if you only want to prevent certain parts of the website from appearing in searches. “
advertise
Keep reading below
See the full video below:
Featured image: Screenshot from: YouTube.com/GoogleSearchCentral, November 2021.
