Engine Templates

The robots.txt is a file containing information about which folders should be crawled and which should not by search engine bots.
The default robots.txt file provided by joomla assumes that all CSS and js files are in the media folder, which is not the case of a lot of Joomla extensions (modules, components, plugins) and in special for Virtuemart extensions.

To avoid to be penalized by google, you should modifiy the default robots.txt file.

The robots.txt file is on the joomla root. The default content of this file since Joomla 3.3 is:

User-agent: *
Disallow: /administrator/
Disallow: /bin/
Disallow: /cache/
Disallow: /cli/
Disallow: /components/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /layouts/
Disallow: /libraries/
Disallow: /logs/
Disallow: /modules/
Disallow: /plugins/
Disallow: /tmp/
All folders not mentionned in this file are Allow.

In the case of VirtueMart, add the following lines to the file robots.txt:

Allow: /components/com_virtuemart/assets/
Allow: /modules/mod_virtuemart_cart/assets/
If you want to be sure, what to change, please use the Webmaster Tools of the most common Search Engines.

If you have any question about this post, don’t hesitate to leave a comment below. And don’t forget to share with your friends if you find it useful! Thanks!

From: virtuemart.net