Gnutella and the State of P2P

written for the pho list

To be honest, Gnutella is not doing very well.

There are about 2300 gnutella hosts on the network accepting incoming
connections [statistic from LimeWire.com] that are part of the “big
cloud” (i.e., public network – it’s possible to set up independant
trading “clouds” on gnutella, too.) One interesting consequence of
this is that all of the public nodes are known and easily
harvestable. Since most nodes hooked up to gnutella are on high-speed
links whose IP addresses don’t change very often, unless you’re behind
a trusty firewall you may find yourself open to attack.

Gnutella prides itself on being a true peered network, but there are
less than a dozen centralized “hostcatchers.” Lacking these, the
network could not function. While it’d be easy to set up a new
hostcatcher if others were taken down, the fact of the matter is that
most gnutella programs come preprogrammed to connect to a specific
hostcatcher. Just send off little letters to the ISPs hosting those
servers and you could defeat the majority of gnutella users and make
it a real pain for new users to join.

There are a wide variety of clients (a good thing), but most of them
suck. It’s not really fair to be that critical of the original program
or even the gnutella architecture, since all it was was a quick hack /
beta implementation on Justin’s part (who has been gagged by AOL). I
can’t imagine how frustrated he must be at people (perhaps like me)
railing at gnutella’s flaws and its lack of ease of use when what’s
out there is a prerelease developer beta that he’s unable to continue.

There’s a pretty good review of the clients out there at LimeWire, but I have to say that BearShare is by far and large my favorite. It renders much more information and is much easier to use (IMHO) than any of the other clients.

But even with BearShare’s pretty and helpful UI, gnutella is not very
usable. The main reason is bandwidth. Just being connected to the
network (not downloading or uploading any files) is pushing about
100kbps of traffic onto the network, with bursts up to 250kbps. Most
cable modems are capped at about 128kbps of upstream traffic. Even
high-quality ADSL can’t usually push more than 100kbps or so, and
that’s already going to pretty much nuke your download capability
(ADSL is odd like that!). So just being connected to gnutella requires
you to be on “extreme broadband,” such as a work connection or an SDSL
line. This is again before you are actually uploading or downloading
any files.

There are other problems, too. Only one out of every ~15 search
results will actually be downloadable, and a good percentage of those
that do succeed are pretty slow (possibly because they are already
burning all of their bandwidth in simply participating on the
network). On napster, around 2 out of 3 search results are actually
downloadable, leading to an environment much more amenable to actual
usage.

On a deeper level, Justin’s gag on continued work on this project
means that this space will probably be pathetically without a true
leader. Sure, various figures pop up from time to time to try and
shepherd the gnutella development/user communities in various
directions (and are periodically mistaken as “the creators of
Gnutella” – as happened in the case of Gene Kan), but this really is
Justin’s aborted baby. Best, IMHO, to let it die and create something
anew to replace it.

I can assure you that that entity will not be gPulp (formerly known as
gnutellaNG, for Gnutella: Next Generation). For those of you not in on
the story, a number of people who had put together gnutella clones and
who understood some of the limitations of the original (unfinished)
gnutella architecture wanted to band together to create a
next-generation protocol and architecture that would provide a great
deal more robustness, speed, scalability, and flexibility. They
started talking, setting up a mailing list and website (at
gnutellang.wego.com) and passed around a lot of really great academic
papers on scalable peered systems (some of which are still sitting on
the message boards). But this group, since it was talking and not
coding, rapidly got very political. One member, Sebastien Lambla, who
had taken charge of the gnutellang website, spontaneously declared the
project renamed to gPulp and announced himself as the “gPulp
Consortium President.” Pretty silly, considering there wasn’t even an
attempt at consensus among the other developers. Sebastien proceeded
to turn gPulp into something different than a next-generation
gnutella, announcing that it would be a “resource discovery protocol”
(without actually specifying the mechanics of file transfer or other
peered solutions).

Sebastien, as if he wasn’t already making enough enemies, then decided
to set up gpulp.com/org/net and establish those as the new center. As
part of the move-over process he announced (and the announcement is
still on the gnutellang page) that he was going to be deleting all of
the message board content on the gnutellang website. Keep in mind here
that the discussions and resource links mentioned in those posts are
*the only valuable thing that the gnutellang group has done to date*. He redirects people to gpulp.tv as the new center for next-generation development. Yes, go there. No, you won’t be able to connect (as of 2/3/01 @ 1:30pm). Same with gpulp.com/net/org. Some new center, huh?

So Sebastien has pretty much singlehandedly defeated the possibility
of there being a half-decent next-generation gnutella. Any attempts to
critique him or to suggest that perhaps another course of action be
taken have been protested as an “attempt to split the community,”
since I think he believes he owns the whole of it. Ouch. Oh well. It’s
a textbook example of what not to do with an Open Source project, if
you want to learn from it.

Ohaha doesn’t seem to be going anywhere, but iMesh is still alive and kicking: there are 62,620 users online right now. Not bad, and about 30 times as many are on gnutella. Too bad there weren’t hardly any files actually available on the network and that those that were “available” (with five stars on “availability”) I couldn’t download. Scour Exchange is (obviously) no longer available. CuteMX, which seems to be alternately yanked then reinstated every few months, is back with a new 2.5 version. Freenet is as obtuse and
inaccessible as ever. Excellent academic work is being done on Jungle Monkey, but it’s not clear if a truly usable and popular client (and indeed, if indeed any Windows port) comes out.

Other than Napster, what other alternatives are out there and useable?
It seems that most of the more recent P2P plays in stealth mode are
targeted at “semi-centralized” models – this makes sense for
efficiency’s sake and also for creating a business model. An open
architecture with free implementations makes it difficult to make a
rollicking lot of money (or at least nobody has quite figured out how
this would work!). But P2P has people thinking, including people in
academia. A lot of theses have yet to be written and will undoubtedly
focus on efficient, decentralized systems.

I have a vision for two classes of systems. The first model is for
rich media and public access. Web pages, chunks of databases,
encrypted file backups, songs, movies, home videos and the like will
all flow through this system. There may be some “delegated” nodes
whose responsibility it is to help the network dynamically anneal to a
near-optimal dataflow by providing “global insight.” They won’t be
necessary, but they will greatly improve the efficiency of the
network. Hostcatchers are crude, first-generation implementations of
such delegated nodes.

The second class of system will be built for preservation of free
speech. It will only carry heavily encrypted, steganographically
embedded text. Through multiple layers, content from this network may
bubble up to the public network, thus allowing for not only anonymous
publication (mixmaster systems already do this) but for invisible
publication (i.e., your local network administrator doesn’t know
you’re publishing texts, even if she suspects it and is looking very
carefully at your network traffic). Text is the primary medium for
free speech and it is substantially easier to conceal than rich media:
even if you drape a blanket over an elephant, it’s pretty clear that
there’s still something there! Source code to “forbidden” programs,
memoranda on government abuses, etc. will all be published on this
network.

A number of people are working on the former class of systems, but
disturbingly few on the latter, which really, for all the hype of
other systems, is the one system that will guarantee free speech to
people.

But there is one key piece of the puzzle that needs to be in place for either system to work. It is absolutely necessary, and I’ve got to thank my roommate, Dan Kaminsky, for pointing it out to me. The upload pipe needs to be in place. Excite @Home has capped all of their cable modems to 128kbps upstream (they used to be able to push 1mbps+!). Pacific Bell modified their ADSL algorithms to even more destructively remove download capacity as soon as there is any upload in progress – this allows them to “permit” servers and P2P traffic but make it impractical to actually run them. SDSL providers like Covad
(my provider) are on the verge of bankruptcy. Alternate Internet access mechanisms (1- and 2-way satellite, fixed wireless, and Ricochet) all have a large amount of up/down disparity. Except for at universities and at really well connected workplaces, it may be
impossible to practically contribute much to a P2P system with all but the absolute most expensive ($1000+/mo) connections. Translation? P2P won’t be able to make it to the masses, other than leeching off of universities’ connections if this trend keeps up.

Let’s hope the power companies move on this whole fiber to the home thing, and fast. An 100mbps uplink in half the homes of America could prove the guarantee of free speech and an open, creative Internet for future generations. Lacking this, we may find ourselves restricted and floundering (as so brilliantly expounded by Jaron Lanier).

(Amazing how some things go full circle, ain’t it?)

Author: dweekly

I like to start things. :)