What if I say that Meta Robots Tags are like the Robots.Txt file?
Not actually but for a short period, you will need to assume it because when you will move ahead in this blog, you will automatically get a clear insight into the Meta robots tags.
According to Moz:
Robots Meta directives (sometimes called “Meta Tags”) are pieces of code that provide crawlers instructions for how to crawl or index web page content.
In other words:
Meta robots tags are the piece of code that tells a Search Engine Bot that how to treat a Web Page, whether a Webpage should be indexed or not. The code is simply put in the head section of that Web Page.
Why do we need to use Meta Robots Tag?
There is no big use of Meta Robots Tag in SEO!
Because if you will not add a Robots Tag on your website so it’s not like that Google will not index your Web Pages, even Google will index them and rank as well on their priority but the thing is you will not have any control over the crawling and indexing!
That’s why we use the Meta Robots Tag beside them there are some other reasons as well to use Meta Robots Tag:
- The first and most common reason behind using the Meta Robots Tags is preventing them to be index and showed in Google SERPs.
- There are many other types of content those you want to show the Visitors, like:
- The page has something which you only want to offer the existing website users.
- The Web Page which has extremely thin content or no information for the users.
- If you are handling the eCommerce type big website so it can be a thank you page, etc.
Indexing & Crawling Controlling Parameters
The Main thing will start now!
Yes, friends, here we will know about all parameters those we can use for our Web Pages content but first, we will need to understand the proper hierarchy of Meta Robots Tag code?
In most common phases the big website you own the way more things you will need to handle and Meta robots are one of the things.
Most commonly there are two things always included in the Meta Robots Tag.
The Values & Attributes in Meta Robots Tag
Whenever you will deal with the Meta Robots Tag so you will need to deal with two different attributes:
- First is Name or User-Agent Attribute, and
- Second is the content or crawling/ indexing type attribute.
As shown below:
The Name or User Agent Attributes
<Meta Name = “Googlebot”
The first half of whole Meta Robots Tag is known as the “Name or User Agent Attributes”. Here you will need to indicate that for which Bot you are ruling.
The Content or Crawling/Indexing Attributes
The second part of the whole Meta Robots Tag is known as the “Content or Crawling/Indexing Attributes”. Here you will need to indicate that how to deal with that Web Pages content.
Here I have listed all the attributes below:
The default value of “index, follow”, no need to ever use this directive.
<meta name=”robots” content=”all” />
Instructs search engines not to index the page. That prevents it from showing in search results.
<meta name=”robots” content=”noindex” />
Stops robots from crawling all links on the page. Please note that those URLs still may be indexable, especially if they have backlinks pointing to them.
<meta name=”robots” content=”nofollow” />
The combination of noindex, nofollow. Avoid using this as other search engines (e.g., Bing) don’t support this.
<meta name=”robots” content=”none” />
Prevents Google from showing a cached copy of the page in the SERP.
<meta name=”robots” content=”noarchive” />
Prevents Google from offering a translation of the page in the SERP.
<meta name=”robots” content=”notranslate” />
Prevents Google from indexing images embedded on the page.
<meta name=”robots” content=”noimageindex” />
Tells Google not to show a page in search results after a specified date/time. A noindex directive with a timer. The date/time must be specified using the RFC 850 format.
<meta name=”robots” content=”unavailable_after: Sunday, 01-Sep-19 12:34:56 GMT” />
Opts out of all text and video snippets within the SERP. It also works as noarchive at the same time.
<meta name=”robots” content=”nosnippet” />
Specifies a maximum number of characters Google can show in their text snippets. Using 0 will opt out of the text snippets, ‑1 declares no limit on the text preview.
The following tag will set up the limit to 160 characters (similar to standard meta description length):
<meta name=”robots” content=”max-snippet:160″ />
Tells Google if and how big an image it can use for image snippets. This directive has three possible values:
none no image snippet will be shown
standard — a default image preview may be shown
large — the largest possible image preview may ‑be shown
<meta name=”robots” content=”max-image-preview:large” />
Sets up a maximum number of seconds for a video snippet. As with the text snippet, 0 will opt out completely, ‑1 places no limits.
The following tag would allow Google to show a maximum of 15 seconds:
<meta name=”robots” content=”max-video-preview:15″ />
So, these were the all attributes those can be used in the Meta Robots Tag beside them here I have attached the image for both majorly used Search Engine and I am damn sure that it will help you in the best manner:
So, Friends yes this was the last step and I hope that now you can use the Meta Robots Tag but don’t worry if you still have any issue in any concept, just do comment me below and I will help you in the best manner.