Hired

I’m hired. I got a job at Legato Systems today. It’s a big company – a couple of thousand people. It looks like I’ll be one of their architects, helping them get a good vision for the future of backup and then helping them actually implement it. No, I won’t be able to tell you all everything that goes on there. 😉

It will be interesting being at a big company. Other than my work at MIT’s
Lincoln Labs, I’ve almost always worked in a startup-style environment, and
even at MIT, I was working as a “leaf” on the tree, free from having to deal
with any politicking or higher-level thinking. But in many ways, I find this
change attractive, because it will let me grow and understand how
larger corporations are structured. I plan to be at Legato for a while;
when I emerge, regardless of what I take on myself at that point, I’ll have a
firm understanding of larger-scale process.

One nice thing about working at a larger company will be that I will be able to work on side projects at home and also work on building Hacker Dojo version 2.0. (I’m currently living in 1.0.) I will continue my off-work exploits and be sure to post them here. 😉

One of the things that I’m honestly looking forward to most though is
culture hacking at Legato. By culture hacking, I mean changing the environment;
by changing people’s attitudes towards what can be done and what they can
expect out of their jobs. I’m looking forward to making Legato a really fun
and productive place to work. I’ve never been in an environment that felt so
ready for it: basically a bunch of smart people just waiting to have some
fun — asking for it explicitly, even. =)

I’m honestly just psyched to be working with a team of people on a regular
basis and to have some structure to my life to use as a platform for
world domination. =) In the interim, I’m learning Win32 as I contemplate
the possibility of getting a G4 when OS/X comes out so I can play with
such a pretty operating system that is actually Unix at its core.
[shudders in delight] I’m not a MacOS fan, but OS/X is starting to give
me tingles.

Secure Audio Path: A Bad Way To Go

written for the pho list

Well, what net news giveth it taketh away. Apparently Microsoft has gone ahead and incorporated “Secure Audio Path” software down to the kernel level. Basically they keep a stream encrypted until it’s passed on to an “authenticated” sound card driver. While there are still ways around this, it may make some things tougher and is a huge step against actually providing consumers the capability to listen to their music as they want to.

As one example, the page on Secure Audio Path explains that audio applications will no longer be able to apply equalization (e.g., bass boost) to an encrypted audio signal. Visualization programs are alotted an explicitly poor-quality version of the signal to which to perform visual effects synchronous with the music. Given the low existing quality of the Windows Media visualization plugins, you can bet this isn’t going to help much. (“Worse than telephone quality” as quoted from their page.) And you’ve got to love how they’ve stuffed all of this DRM software into the kernel – those of you engineers on the list know that as you increase the size of the software that runs in kernel mode, you dramatically increase your chances of locking up the kernel. The whole reason behind putting programs in user mode is to protect them from each other and from the OS – that way if an application crashes, it doesn’t take the system down with it, too. Stuffing all of this DRM software into the kernel sure isn’t going to help things. My experiences with Windows Media protection have been pretty marginal, often causing lockups & crashes. And that was at the user level! [sigh]

But it’s the cryptographic signing of the drivers that takes this thing to a whole different level of insidiousness. In order for an encrypted stream to play, Microsoft has to approve the driver for your sound card and sign it. Without a signed driver, DRM content won’t play. So every time your sound card company releases an update, you’ve got to wait to get it signed by Microsoft. The bad news is that they’ve already had a driver signing program for the last few years and that very few drivers are actually signed by Microsoft. With video DRM coming soon hereafter, what this really does is give Microsoft the power to determine what hardware it will allow to run Windows.

If your sound card company went and helped the Linux community make a driver, MS might just accidentally take another couple months to sign your driver — in the interim your competitor might just jump into the market with the special features you had on your card and, through a good MS relationship, get their drivers signed and their cards deployed. Your time to market, indeed nearly the whole of your success, depends upon Microsoft. Soon video card makers will be in the same boat as well as hard drive makers (remember the encryption ATA functions?). The same goes for CD readers, DVD readers, burners, scanners, printers, even USB speakers. Why allow any venue for those pirates? Since, technically, every component in the system is “suspect to interception,” and capable of assisting infringement, Microsoft can use this suspicion to force validation of all component drivers. Who is the arbiter of whether or not a driver is sufficient to be signed? Microsoft, of course.

Those of you who are strongly in favor of DRMs are playing the fools right into Microsoft’s hands. You are fighting free speech, fighting against consumer freedoms, and fighting against democracy. Worst yet, and you don’t even realize it yet, but you are playing right into the hands of Microsoft – and you’ll be so happy that all of your content is protected from those nasty hackers who would listen to your music for free. You’ll be happy until Microsoft has you firmly by the balls and starts to squeeze. Because once everyone is using a Microsoft platform and the hardware vendors and software vendors are all in lockstep with their plan…what do you think they’re going to do it for free? That MS has gone Open Source? No, they’re going to start charging you, the content producers, obscene amounts of money to publish onto their framework. And you really won’t have any choice at all in the matter. Because if you start publishing non-DRMed audio, your content won’t play on Windows boxes. Lanier was right, but he was wrong about timeframes – this is all happening way faster than he predicted.

Someday, with luck before it’s too late, you songwriters, you labels, you artists and musicians and composers, you will wake up and realize that the hackers were on your side all along. That you were singing about freedom and they were hacking freedom. And the hackers aren’t paid anything for their hacking and go, for the most part, unrecognized. And the musicians aren’t paid almost anything for their music and go, for the most part, unrecognized. What a perfect, beautiful couple! And yet the framework of the present situation has pushed us against each other with swords in balled fists and chanted for blood, for us to smite each other.

I don’t know any hacker who doesn’t think that musicians should get paid for their music. Some of my hacker friends compose music. I don’t know any musicians that aren’t excited by the subversive nature of the Internet and peered distribution mechanisms and, consequently, who don’t respect hackers. Why don’t we both work together, put down our swords, figure out how to put bread in each other’s mouths (yes, even hackers are having a harder time than usual with that these days) and subvert the structure that has caused this unnatural schizm between us?

Come, musicians, let me show you the beauty of Perl. You can show me how to compose a tune that will stick in people’s heads. And maybe together we’ll forge a hymn of clefs and compilers for a revolution that has only now begun to seed.

SDMI: Aris Wins, World Loses

written august 13, 1999

Didn’t the SDMI folks say that SDMI was a general wrapping specification
into which many technologies could fit? But they selected a singular
watermarking technology to handle security: Aris. Now it seems to me that of
all of the players involved (labels, artists, music VARs, portable device
manufacturers, and consumers) the only real winners from SDMI’s
pseudo-protection are the RIAA and Aris.

Once SDMI-only players are in place, the only way for an artist to get their
music on such a device is to use software licensed from Aris to put the
“SDMI Kosher” watermark on their music. Additionally, to distribute/sell
music from their website, artsts would need to purchase a complex per-user
watermarking server, with Aris-licensed technology. Of course, most artists
would never have enough savvy to do this: as such, they’ll be forced to sign
their music to a technology provider that had invested the money and time to
put such a system in place. Small, independant musician sites would likely
disappear under this paradigm.

Device manufacturers seeking to become SDMI-compliant will have to license
watermark-decoding technology from Aris and likely pay them a per-device
fee.

While agreed that it is at this time just wishful thinking, if a watermark
technology is needed for major content to move to the Net, an OpenSource
watermark technology would be the best choice. It would
encourage rapid and widespread adoption of a watermarked protection
mechanism and would ensure that even the smallest players would have a shot
at being able to set up music websites. The SDMI committee may believe that
restricting access to the technology will allow valid music publishers to
distribute their music while barring “pirates” from “stealing” their music.
Instead, it is giving Aris a monopoly and works strongly against small
publishers. As a folk artist in northern Oklahoma, how do you convince the
RIAA’s SDMI Watermark Office that you really are making music: that you have
a guitar, are recording original content, and deserve access to the
technology? What will allow the RIAA (or anybody) to decide whose music is
pirated and whose is valid? Does anybody have answers to these questions?

As a separate thought/consideration, the SDMI committee has made it clear
that computers will not be under the SDMI restrictions, only “portable music
devices.” And yet this distinction gets more blurry day by day! I argued
that the Rio was a computer, complete with storage, processing, a display,
input/output, and inter-computer communication capabilities — I created the
first patch to allow software to upload music from the Rio back to the
computer to prove this. But while the Rio positioned itself as a “computer
peripheral,” the new devices are incorporating more and more functionality
and are becoming PDA/Walkman hybrids. Is the Cassiopeia a computer? Surely!
And the Nomad? What if it added calendar support? How can the SDMI hope to
have any teeth at all if it doesn’t regulate any devices?

To add to all of this confusion, SDMI takes rights away from consumers.
SDMI prevents me from storing my music in multiple places, or from keeping
it in a central location that I can frequently access. If SDMI eventually
incorporates some of these “special cases,” it will likely be quite
complex: it has to be! Computer technology fundamentally allows for
sharing (see my article on this) and the technology to share tends to be easier
to write, and thus one step ahead, of technology to prevent sharing. As
hard as it is, it would be far wiser for labels to sell their music
in the clear like eMusic is doing.

This fake protection is not the savior. It will have its heyday, as
did software encryption some decade ago, but it will fade as people
realize the power of openness.


UPDATE: As it would have it, the above analysis is not perfectly correct. I’ve been notified by the co-chair of the SDMI committee and the CTO of Aris that clear (non-watermarked)
audio will indeed be free to be played on any SDMI-compliant device. The only music that gets blocked is music that is watermarked and labelled “Don’t Reproduce.”

See Also: Why SDMI Will Fail

Gnutella and the State of P2P

written for the pho list

To be honest, Gnutella is not doing very well.

There are about 2300 gnutella hosts on the network accepting incoming
connections [statistic from LimeWire.com] that are part of the “big
cloud” (i.e., public network – it’s possible to set up independant
trading “clouds” on gnutella, too.) One interesting consequence of
this is that all of the public nodes are known and easily
harvestable. Since most nodes hooked up to gnutella are on high-speed
links whose IP addresses don’t change very often, unless you’re behind
a trusty firewall you may find yourself open to attack.

Gnutella prides itself on being a true peered network, but there are
less than a dozen centralized “hostcatchers.” Lacking these, the
network could not function. While it’d be easy to set up a new
hostcatcher if others were taken down, the fact of the matter is that
most gnutella programs come preprogrammed to connect to a specific
hostcatcher. Just send off little letters to the ISPs hosting those
servers and you could defeat the majority of gnutella users and make
it a real pain for new users to join.

There are a wide variety of clients (a good thing), but most of them
suck. It’s not really fair to be that critical of the original program
or even the gnutella architecture, since all it was was a quick hack /
beta implementation on Justin’s part (who has been gagged by AOL). I
can’t imagine how frustrated he must be at people (perhaps like me)
railing at gnutella’s flaws and its lack of ease of use when what’s
out there is a prerelease developer beta that he’s unable to continue.

There’s a pretty good review of the clients out there at LimeWire, but I have to say that BearShare is by far and large my favorite. It renders much more information and is much easier to use (IMHO) than any of the other clients.

But even with BearShare’s pretty and helpful UI, gnutella is not very
usable. The main reason is bandwidth. Just being connected to the
network (not downloading or uploading any files) is pushing about
100kbps of traffic onto the network, with bursts up to 250kbps. Most
cable modems are capped at about 128kbps of upstream traffic. Even
high-quality ADSL can’t usually push more than 100kbps or so, and
that’s already going to pretty much nuke your download capability
(ADSL is odd like that!). So just being connected to gnutella requires
you to be on “extreme broadband,” such as a work connection or an SDSL
line. This is again before you are actually uploading or downloading
any files.

There are other problems, too. Only one out of every ~15 search
results will actually be downloadable, and a good percentage of those
that do succeed are pretty slow (possibly because they are already
burning all of their bandwidth in simply participating on the
network). On napster, around 2 out of 3 search results are actually
downloadable, leading to an environment much more amenable to actual
usage.

On a deeper level, Justin’s gag on continued work on this project
means that this space will probably be pathetically without a true
leader. Sure, various figures pop up from time to time to try and
shepherd the gnutella development/user communities in various
directions (and are periodically mistaken as “the creators of
Gnutella” – as happened in the case of Gene Kan), but this really is
Justin’s aborted baby. Best, IMHO, to let it die and create something
anew to replace it.

I can assure you that that entity will not be gPulp (formerly known as
gnutellaNG, for Gnutella: Next Generation). For those of you not in on
the story, a number of people who had put together gnutella clones and
who understood some of the limitations of the original (unfinished)
gnutella architecture wanted to band together to create a
next-generation protocol and architecture that would provide a great
deal more robustness, speed, scalability, and flexibility. They
started talking, setting up a mailing list and website (at
gnutellang.wego.com) and passed around a lot of really great academic
papers on scalable peered systems (some of which are still sitting on
the message boards). But this group, since it was talking and not
coding, rapidly got very political. One member, Sebastien Lambla, who
had taken charge of the gnutellang website, spontaneously declared the
project renamed to gPulp and announced himself as the “gPulp
Consortium President.” Pretty silly, considering there wasn’t even an
attempt at consensus among the other developers. Sebastien proceeded
to turn gPulp into something different than a next-generation
gnutella, announcing that it would be a “resource discovery protocol”
(without actually specifying the mechanics of file transfer or other
peered solutions).

Sebastien, as if he wasn’t already making enough enemies, then decided
to set up gpulp.com/org/net and establish those as the new center. As
part of the move-over process he announced (and the announcement is
still on the gnutellang page) that he was going to be deleting all of
the message board content on the gnutellang website. Keep in mind here
that the discussions and resource links mentioned in those posts are
*the only valuable thing that the gnutellang group has done to date*. He redirects people to gpulp.tv as the new center for next-generation development. Yes, go there. No, you won’t be able to connect (as of 2/3/01 @ 1:30pm). Same with gpulp.com/net/org. Some new center, huh?

So Sebastien has pretty much singlehandedly defeated the possibility
of there being a half-decent next-generation gnutella. Any attempts to
critique him or to suggest that perhaps another course of action be
taken have been protested as an “attempt to split the community,”
since I think he believes he owns the whole of it. Ouch. Oh well. It’s
a textbook example of what not to do with an Open Source project, if
you want to learn from it.

Ohaha doesn’t seem to be going anywhere, but iMesh is still alive and kicking: there are 62,620 users online right now. Not bad, and about 30 times as many are on gnutella. Too bad there weren’t hardly any files actually available on the network and that those that were “available” (with five stars on “availability”) I couldn’t download. Scour Exchange is (obviously) no longer available. CuteMX, which seems to be alternately yanked then reinstated every few months, is back with a new 2.5 version. Freenet is as obtuse and
inaccessible as ever. Excellent academic work is being done on Jungle Monkey, but it’s not clear if a truly usable and popular client (and indeed, if indeed any Windows port) comes out.

Other than Napster, what other alternatives are out there and useable?
It seems that most of the more recent P2P plays in stealth mode are
targeted at “semi-centralized” models – this makes sense for
efficiency’s sake and also for creating a business model. An open
architecture with free implementations makes it difficult to make a
rollicking lot of money (or at least nobody has quite figured out how
this would work!). But P2P has people thinking, including people in
academia. A lot of theses have yet to be written and will undoubtedly
focus on efficient, decentralized systems.

I have a vision for two classes of systems. The first model is for
rich media and public access. Web pages, chunks of databases,
encrypted file backups, songs, movies, home videos and the like will
all flow through this system. There may be some “delegated” nodes
whose responsibility it is to help the network dynamically anneal to a
near-optimal dataflow by providing “global insight.” They won’t be
necessary, but they will greatly improve the efficiency of the
network. Hostcatchers are crude, first-generation implementations of
such delegated nodes.

The second class of system will be built for preservation of free
speech. It will only carry heavily encrypted, steganographically
embedded text. Through multiple layers, content from this network may
bubble up to the public network, thus allowing for not only anonymous
publication (mixmaster systems already do this) but for invisible
publication (i.e., your local network administrator doesn’t know
you’re publishing texts, even if she suspects it and is looking very
carefully at your network traffic). Text is the primary medium for
free speech and it is substantially easier to conceal than rich media:
even if you drape a blanket over an elephant, it’s pretty clear that
there’s still something there! Source code to “forbidden” programs,
memoranda on government abuses, etc. will all be published on this
network.

A number of people are working on the former class of systems, but
disturbingly few on the latter, which really, for all the hype of
other systems, is the one system that will guarantee free speech to
people.

But there is one key piece of the puzzle that needs to be in place for either system to work. It is absolutely necessary, and I’ve got to thank my roommate, Dan Kaminsky, for pointing it out to me. The upload pipe needs to be in place. Excite @Home has capped all of their cable modems to 128kbps upstream (they used to be able to push 1mbps+!). Pacific Bell modified their ADSL algorithms to even more destructively remove download capacity as soon as there is any upload in progress – this allows them to “permit” servers and P2P traffic but make it impractical to actually run them. SDSL providers like Covad
(my provider) are on the verge of bankruptcy. Alternate Internet access mechanisms (1- and 2-way satellite, fixed wireless, and Ricochet) all have a large amount of up/down disparity. Except for at universities and at really well connected workplaces, it may be
impossible to practically contribute much to a P2P system with all but the absolute most expensive ($1000+/mo) connections. Translation? P2P won’t be able to make it to the masses, other than leeching off of universities’ connections if this trend keeps up.

Let’s hope the power companies move on this whole fiber to the home thing, and fast. An 100mbps uplink in half the homes of America could prove the guarantee of free speech and an open, creative Internet for future generations. Lacking this, we may find ourselves restricted and floundering (as so brilliantly expounded by Jaron Lanier).

(Amazing how some things go full circle, ain’t it?)