*The guest tip was provided by Matthew Yarro, JuiceeLinks*
Not every website on the Internet is a blog. I know, It seems crazy in today’s super-social-media climate, but some websites simply exist to offer up whatever information they deem to be important–and do so with infrequent updates.
The problem with these “static” websites is that Google eventually realizes that the websites are not being frequently updated, and so it makes a point of not crawling these websites as often as it would blogs that are updated regularly. This can result in a huge time lag between the time content is added to a webpage and a Googlebot returns to the website to re-crawl it–or crawl it for the first time.
Keep Your Website Indexed and Up to Date with Google Reader
How can you increase the frequency of a crawl for relatively static content? By using the Google Reader.
The Google reader, which has been traditionally used to aggregate feeds from blogs and other sources, now allows you to create a feed for a static webpage:
Feeds make it easy to follow updates to all kinds of webpages, from blogs to news sites to Craigslist queries, but unfortunately not all pages on the web have feeds. Today we’re [Google] rolling out a change in Google Reader that lets you create a custom feed to track changes on pages that don’t have their own feed.
These custom feeds are most useful if you want to be alerted whenever a specific page has been updated… (Google.com)
Use Google Reader to Crawl My Website, Huh?
What does this all mean? Follow me for a moment. In order for the Google Reader to post changes to an otherwise static webpage, Google has to crawl that page and crawl it frequently. After all, there is no value in claiming that the Google Reader can watch a static page and alert you of any changes, if that alert comes seven days after the fact.
I believe that once you have targeted a webpage with the Google Reader, Google is crawling your content at least once every twenty-four hours. And that is precisely what the examples from the announcement on the Google Reader blog suggest.
If you check the examples, the minimum frequency between updates is twenty-four hours, and it could even be less. The Google Reader seems to aggregate the changes into a twenty-four hour period. So it is possible, that the reader is checking more than once in a twenty-four hour period.
And what’s more, if Google is going to crawl the page to see if the content has changed, then Google has your page indexed in one of its databases.
Get Your Google Reader in Gear
If you have static webpages or entirely new content that you need crawled, target your webpages with the Google Reader. Using the Google Reader allows Google to quickly find, crawl and index your content.