Magento robots text SEO
Magento robots txt SEO It is groovy when search engines often invade your site and index your content. To discuss the scenario in a more meticulous manner let’s glimpse through a small example. When a page possess two versions one for viewing and the other for printing purpose, to evade threat of duplicate content penalty, the printing version can be omitted from creeping. Another instance could be when there are some sensitive data put up on the site and not intended to expose to the world, it is preferred that search engines do not index these pages. Robots metatag is one way to inform the search engines the files and folders to be avoided. But one should always hold in mind that metatag go unnoticed often, in this case robots.txt can be utilized. What is a robots.txt file? Robots.txt in magento is a significant aspect and let’s dig in more about the same. A robots.txt file is not perplexed but a text file which stands on your site which informs search robots the page t