"Why is it that Yahoo! uses a different IP for every page request while spidering?A lot of sites, such as forums, have a "guests" or "online users" count listed on their page. When Yahoo! uses 146 different IP addresses to spider 146 pages, it artificially inflates the online users count"
"We're reducing our crawling machines by a significant amount with this update, so you might have observed a significant reduction in crawl load from us along with fewer machine IP addresses hitting you over the last few days.The new crawler continues to improve comprehensiveness and freshness of coverage while also enhancing crawl efficiency, which reduces spurious load on websites."
Posted by Matthew Elshaw at 8:30 AM GMT