To specify that an HTML page should be removed from the search results after 3pm Eastern Standard Time on 25th August 2007, simply add the following tag to the first section of the page:<meta name="GOOGLEBOT" content="unavailable_after: 25-Aug-2007 15:00:00 EST">The date and time is specified in the RFC 850 format.This information is treated as a removal request: it will take about a day after the removal date passes for the page to disappear from the search results. We currently only support unavailable_after for Google web search results.
Posted by Matthew Elshaw at 4:55 AM GMT
Are you sure that it goes in the Robots.txt file? The blog suggests that it is added to the web page.
yeah, way to confuse us? I think I'll go with the meta tag ;)
Sorry my mistake! Your right the tag should be added to the webpage.