Professional Search Engine Optimization and SEO Marketing Firm

Home

Package Deals

Optimization

Submission

Pay-Per-Click

Paid Inclusion

Local Search

Conversion

Free Tools

Blog

Welcome to the ineedhits Search Engine Marketing blog, where we share the latest search engine and online marketing news, releases, industry trends and great DIY tips and advice.

We encourage you to get involved in our blog community - so share your opinions and experiences by leaving comments on our posts.

If you're looking for help with promoting your website - check out our range of affordable search engine marketing services.

Tuesday, April 11

Google's Spider of Death

In one case, Google's spider 'killed' an entire website.

Do you really want Google's spider of death to visit your website?

For most website owners, many of their search engine optimization efforts are aimed at getting Google's spider to visit their website on a regular basis. It's assumed that the more regularly the GoogleBot visits, the more up-to-date your search engine listings will be. And generally, regular visits by Google's spider are a sign that your website is viewed quite highly by the search engine.

In a case of "be careful what you wish for", a website owner experienced the full wraith of the GoogleBot recently. After engaging a web development firm to implement a content management system for their existing website, a month of painstaking development was swiftly undone when Google's spider essentially killed the newly re-developed web site.

While the disastrous impact GoogleBot's visit is alarming, it can't be held totally to blame. It is fair warning for anyone with a poorly programmed site.

The spider is designed to follow each hyperlink on a page, including those with "Delete Page" in the title. The net result; GoogleBot systematically went through the site and deleted everything. Not all was lost, as the prudent back-up of the previous site was soon re-published to fill the void in the spider's wake.

So if you don't wish for GoogleBot to make a disappearing trick of your web pages, especially if you have a content management system controlling your website, then it might be timely to check your page links and ensure you're not leaving your website open to the destructive power of Google's spider.

Post To Del.icio.us

Give your Google listing a boost.
7 day inclusion for only $19.95 - Money Back Guaranteed!

3 Comments

At 2:39 PM, Garnet said...: This is hilarious! A friend who worked at IBM in the good old days of room sized computers had a small placard from IBM on his desk, it only had one word on it: "Think"

This article doesn't give any solutions for how to avoid this problem, so let me give a couple of very simple things to keep in mind:

1) Keep commands with disasterous consequences behind a login, or otherwise unspiderable place.

2) Use form buttons instead of links for thinks like edit, save, etc. Form buttons are generally not spiderable.

Using straight HTML links for content management editing might look nice, but they are an open invitation for this kind of disaster not only from Google, but any kind of web-whacking downloader software.

- Garnet, developer of Free Hive Wiki
At 7:23 PM, Randy Davis said...: Either this is a poorly written article, or it is claiming that Google's spider can delete pages, or both. Sheer nonsense! No search engine spider can delete pages on your Web site -- content management or not. Could a poorly written Web site cause Google to de-list a Web site, or substantially lower its listing position? Certainly. But no search engine has the power of deletion! This article smacks of a search engine urban myth.
At 9:20 PM, brien said...: uh, randy, i think you need to actually *read* the article above.

If a poorly written website has a link titled "Delete Page" that, when followed, deletes a page, google bot, or any spider for that matter, can surely follow the link causing the page to be deleted