Spider Hunter

22 Mar

Sitemap Protocol for Yahoo

It is old news that Google uses the sitemap protocol which allows webmasters to tell Google’s spider, Googlebot, which web pages have been updated recently. The greatest advantage is that Google will use the sitemap to make sure they spider your web site in a more intelligent way that you control within the bounds of the sitemap protocol.

The sitemap protocol has grown beyond just Google these days. You can find out more information about the protocol itself at sitemaps.org. They give a good overview of the protocol, which is pretty simple, and in the FAQ they imply that other search engines are using the sitemaps protocol.

That fact turns out to be true. For some time now Yahoo has been using the sitemaps protocol as well. On their Site Explorer web site you can let them know about your sitemaps in much the same way that you would inform Google and they also take feeds, such as RSS feeds, in addition to the sitemap protocol.

MSN is also suppose to be using the sitemap protocol, but I haven’t found the page to submit any sitemaps to.

At this point the only thing that I can think of that would make this better is if there was a way to for auto discovery of sitemaps. Something similar to the auto discovery options for RSS feeds, a META tag that point to one or more sitemaps on your domain. 

With this type of auto discovery method any search engine could use the sitemap protocol and have an intelligent way to spider your site without having to login to all of the different services that use sitemaps. 

sitemaps, Google, Yahoo

Technorati Tags: , ,

Share and Enjoy: These icons link to social bookmarking sites where readers can share and discover new web pages.
  • Digg
  • del.icio.us
  • Netvouz
  • DZone
  • ThisNext
  • MisterWong
  • Wists

Leave a Reply

You must be logged in to post a comment.

© 2008 Spider Hunter | Entries (RSS) and Comments (RSS)