Software:Gnutella crawler

A Gnutella crawler is a software program used to gather statistic information on the gnutella file sharing network, such as the number of users, the market share of different clients and the geographical distribution of the userbase.

Early crawlers used the Ping/Pong messages to discover hosts connected to the network. Although this method is still usable, it is too slow to capture enough data for a topological overview of the gnutella network as it requires initiating full gnutella connections; this involves several roundtrips to perform the header processing. An extension has been added to the gnutella protocol to allow crawlers to quickly access ultrapeers. Right now, there is no publicly accessible crawler online on the gnutella network, since the last one hosted by Lime Wire LLC has been taken down.

Gnutella2 (G2) also supports crawlers for the gain of statistical data such as the network size or the network composition (clients, versions, usernames and usercountries). Right now, there is only one crawler existing on the G2 network, called g2paranha. It is written and maintained by dcat and licensed under the GPL.

See also

• web crawler

References

• Daniel Stutzbach and Reza Rejaie, "Capturing Accurate Snapshots of the Gnutella Network", the Global Internet Symposium, March, 2005.

External links

• The Gnutella Crawler Protocol - An old, but still valid document on Gnutella network crawling.
• The GDF:Communicating Network Topology Information
• Ion Sampler - a statistical crawler that estimates the number of nodes in the Gnutella overlay.
• Cruiser - a distributed crawler that gives some of the best estimates of Gnutella's size.