Spider Hunter

While I don’t think that these ideas will necessarily get you into either Google or Yahoo’s search indexes any faster or get you better positions looking at your RSS feeds through these interfaces can give you an idea of how your users might see your feed.

Last I heard My Yahoo! was the #1 RSS reader out there, now that Google has web page personalization I’d expect them to take a huge leap and get into the top 5 in short order. So what does this have to do with search engine spiders?

Both Google and Yahoo! are using some sort of caching technology. That means that they check your RSS feed once and deliver that same feed for some period of time, hopefully the TTL on your RSS feed.

In order of the data on each of the personalize home pages to be kept up to date and their customers to be kept happy that data has to be retrieved on a some what regular basis. This means that your RSS feeds will be kept fresh in a database at Google or Yahoo! and having any content in any database at either company is never a bad thing.

I have personally setup my RSS feeds on both services and the difference in the way they display them in interesting. I like Googles drag around interface better then Yahoo!’s layout designer. But Yahoo!’s interface gives me a better idea of when my last entry was on each of my sites.

So while both companies are going to tell you that these services are not going to change your search engine ranking, setting this up is simple and quick and it can’t hurt.

Besides if I was a huge company that maintained some of the largest databases in the world I’d have a hard time not linking them together to try and make some processes more efficient. Wouldn’t it be better to have one spider that collects the data and then multiple indexes that use the data in different ways? I have no proof that this is the way that things work, but if they didn’t it would be an awful waste of resources ….