Today many SEO specialists often use Nofollow and Noindex tags in order to distribute the weight of pages and also to hide some unnecessary parts of content or html code. What is the difference between noindex and nofollow?
Understanding Nofollow – Noindex Tag Attributes
The first term is a tag which hides a part of text or the whole page from the search engines indexation, the second one is an attribute of <a> tag which helps to regulate the weight of a page by prohibiting to send it to the site being accessed.
Noindex is used to hide some unnecessary information from the search engines, e.g. site visitor counter codes, non-unique content, frequently updated information.
The main role of Nofollow is the preservation of the page weight. This attribute is needed if you do not wish to transfer a part of the page weight to an unconfirmed source, in case you link to such popular sites as Google or Subscribe. This tag will also be useful if a page contains too many outgoing links. Besides, you can also use Nofollow to hide the links in comments.
You can configure Nofollow and Noindex in Magento ® by yourself. GoMage experts recommend two possible ways:
- by placing a file robots.txt in the root directory of your site;
- by using meta name=”robots” tag.
In the first case, you need to update the file robots.txt with the names of pages you wish to hide using the following code:
User-agent: *
Disallow: /myfile.html
This method is basically used to prohibit indexation of the whole page instead of a specific part of the text or unnecessary code.
Correspondingly, if you do not wish to transfer a part of the page weight, you should use nofollow instead of noindex.
The second variant of indexing prohibition may look as follows:
<html>
<head>
<meta name=”robots” content=”noindex”/>
<title>Site page title</title>
</head>
The attribute ‘content’ can have the following values:
<meta name="robots" content="index, follow" />
<meta name="robots" content="noindex, follow" />
<meta name="robots" content="index, nofollow" />
<meta name="robots" content="noindex, nofollow" />
| INDEX, FOLLOW | Means that you “tell” search engine robots to index the certain site content and then come back and check for changes later. |
| NOINDEX, FOLLOW | Means that you “tell” search engine robots not to index the certain site content, but come back and check for changes later. |
| INDEX, NOFOLLOW | Means that you “tell” search engine robots to index the certain site content only once and don’t come back and check for changes later. |
| NOINDEX, NOFOLLOW | Means that you “tell” search engine robots not to index the certain site content and don’t come back and check for changes later. |
You can choose the content to hide in the Magento ® admin panel: System -> Configuration -> Design -> HTML Head -> Default Robots
In order to set noindex and nofollow on static pages, you can use layout update. In CMS – Pages tab choose the necessary page, then locate the field Layout Update XML and paste the following code:
<reference name="head">
<action method="setRobots"><value>noindex, nofollow</value><name>robots</name></action>
</reference>
Development Get a robust PWA
Supercharge your eCommerce efforts with superb UX and ‘superfast’ speed. Build an eCommerce PWA from scratch or save time and money using GoPWA Storefront.
This code can also be used in any template you choose. If you have any problems or questions we recommend you to contact our Magento ® development team. We will be glad to assist you.
Nofollow and Noindex in Magento ® 2
The procedure described above is absolutely acceptable for Magento ® 2. You should follow the same logic to configure noindex, nofollow tags and robots.txt.
You need to go to your Magento ® admin panel, then choose “Stores” >> “Configuration” section. On the left side under “General” option you should choose “Design”. Open the “Search Engine Robots” tab and configure the necessary settings.
By perforce, you can set your own custom instructions in the Edit Custom instruction of robots.txt file option. For instance, you need to disallow access to all folders while your online store is in development.
There are examples of custom Instructions:
If you want to allow full access
User-agent:*Disallow:
If you want to disallow access to all folders
User-agent:*Disallow: /
Default Instructions
Disallow: /lib/
Disallow: /*.php$
Disallow: /pkginfo/
Disallow: /report/
Disallow: /var/
Disallow: /catalog/
Disallow: /customer/
Disallow: /sendfriend/
Disallow: /review/
Disallow: /*SID=
There are not only technical reasons why you need to close your content from being indexed by search engine robots. Noindex tag can help you in some cases:
- you need to quote a part of the law or a passage from the book and such content can’t be unique;
- you need to place an interesting article that was published on another site. Why not? If such content may help to keep visitors in your online store longer, you may use it;
- technical texts or instructions may contain examples of code which can’t be unique, either.
In all these cases, you may prohibit indexation of a specific part of the text or of the whole page.
We hope that our tips will be useful for you. Our GoMage team is ready to assist you if necessary.
