At ZD Net, Tom Foremski asks a very interesting question:

I would love to know how much Internet bandwidth is used by the swarms of spiderbots? Because if bandwidth costs are going to go up, as the telco/cable last mile owners charge third parties for bandwidth, then the spiderbots might get banned.

Spiderbots eat up a huge amount of bandwidth, and if bandwidth gets more expensive, the spiderbots are going to suffer. For example, I get 5 per cent of my traffic from more than 18 spiderbots. They use up about one-third of my bandwidth.

That’s a key reason why Google, Yahoo and others, are pushing for net neutrality–equal access to bandwidth–at least the last mile pipe to the home, the most important pipe. If companies are going to have to pay extra to the telcos or cable companies for bandwidth to reach their users, they might not be so pleased to be paying for the bandwidth of the spiderbots.

In other words, it’s no surprise that Google, Yahoo and Microsoft — the search engine behemoths — are asking for special protection from the government. If they can avoid paying for the bandwith they use, they certainly will. And who gets stuck paying for the Internet usage of these large, wealthy companies? It doesn’t take a rocket scientist (or a network engineer) to figure out that the ones paying for it will be the rest of us.