Add your sitemap to your robots.txt

      Home » Websites & software » Add your sitemap to your robots.txt

Add your sitemap to your robots.txt

This is great news. Robots indexing your website have to check the robots.txt file that you have (well you should have anyway).

Now you can use a new command to tell the robots where your sitemap is. This means that robots will be able to index the sitemap immediatly and therefore get a better list of your sites in their database.

All search crawlers recognize robots.txt, so it seemed like a good idea to use that mechanism to allow webmasters to share their Sitemaps. You agreed and encouraged us to allow robots.txt discovery of Sitemaps on our suggestion board. We took the idea to Google and Microsoft and are happy to announce today that you can now find your sitemaps in a uniform way across all participating engines. To do this, simply add the following line to your robots.txt file:

Sitemap: http://www.example.com/site...

Please provide the complete URL for your Sitemap on this line. We will pick it up wherever you put it in your robots.txt file. This directive is not specific to user-agent. If you have multiple Sitemaps, you can point to your Sitemap index file on this line. Details about the Sitemaps protocol including this addition are available on the protocol website -- http://www.sitemaps.org.

from http://www.ysearchblog.com/...
By netchicken: posted on 14-4-2007








Add your sitemap to your robots.txt | [Login ]
Powered by XMB
Privacy Policy