Mapping the Internet

With only about 50% of PBwiki’s traffic coming from North America and with preliminary benchmarks showing 3+ second page load times in Paris, I’ve been thinking a bit about how to make the PBwiki experience snappy for people around the world. We’ve experimented with using various CDNs, but I’ve actually yet to be blown away by any of them. Having our own nodes at the edge can provide a number of benefits, such as having a well-defined cache invalidation strategy, performing DNS closer to the edges of users’ networks, caching secure data, and performing SSL handshakes quickly for logins.

So a reasonable question to ask at this point is – where are the spots where we’d get the most bang for the buck adding a new server? Answering this question requires a basic understanding of Internet topology. Armed with VPS accounts in Singapore and The Netherlands and traceroute.org, I set out to get a basic feel for the current state of global networks.

My principal hoped-for finding turned out to be true – links are mostly additive. Meaning that if it takes 200ms to get from Singapore to California and 85ms to get from California to Virginia, it takes nearly 285ms to get from Singapore to Virginia. Generally the direct route times were about 10% faster than the sums of the links, but never a lot less than that. This was encouraging because it said that latency was fairly consistently due to the speed of light.

That said, there were some startling findings as to global connectivity – The Netherlands are about 4500 miles away from India, but packets from Amsterdam consistently routed through Palo Alto on a 16,000+ mile journey the wrong way around the planet.

I also found that most of South America seems to route through Miami – even traffic within South America! And that traffic for South Africa often routes through New York, even from London, crossing the Atlantic twice. SAT-3 doesn’t seem to be doing its job.

David’s Tips on International Expansion Ordering:

  1. 1st cluster: To be most Net-accessible, your first cluster should probably be hosted in the US on either the West or East coast, depending on target demographic.
  2. 2nd cluster: Your second cluster should probably be on the other US coast – this will mean you’re within ~40ms of nearly all of North America, are under 100ms from Europe, and are under 200ms from Asia & Oceania.
  3. 3rd, 4th, 5th, and 6th clusters: Once you get into the swing of having a few clusters, the remaining spots that make sense seem to be Europe (Amsterdam & London are eminently reasonable choices), Australia or New Zealand, Japan, and Singapore or Hong Kong. It looks like the European ISPs have been peering reasonably well and are all under 50ms from London or Amsterdam. AU and NZ are ~30ms apart (Sydney from Auckland), as are Singapore and Hong Kong.
  4. Extra clusters: As needed, you can deploy in Brazil (which won’t help other South American traffic), South Africa (which won’t help other African traffic), India, or Israel (for Middle East acceleration).

More later on how to expand into additional points of presence at a low cost.

On Free Speech

A post of mine from a recent email thread on why a co-op should continue to host a server with arguably distasteful but legal content:

“I do not agree with what you say, but I will defend to the death your right to say it.”

Those who believe in the power of free speech do not believe in the excellence of each speech, but in the excellence of the collective understanding that results. Each one of us has truths and falsities and we must together thresh the false ideas from the wise. Even wise men can be foolish and even fools may be privy to wisdom. So we must consider each speech on its own merits and not mute anyone or raise some other up on a special pedestal of infallibility. This perspective compels us to permit the submission of all ideas to the judgment of mankind. And if we are all separably and collectively wise in these things, devising best we can lies from knowledge, we shall thrive as a species, acting on all the best data available to us.

In this capacity have our servers been home to churches, governments, anarchists, junior republicans, UFO conspiracy theorists, strip clubs, libraries, political candidates, sex parties, kindergarden teachers, reading groups, carpentry shops, and multinational corporations. We welcome all(*).

– David

(*) Needless to say, the in-person user meetings could get awkward.

David’s Difficult Math Question

In honor of pi day (3/14)…

If pi has an infinite number of digits in random sequence, then pi must contain all sequences.

It is commonly said that “pi does not repeat” and while generally true (pi does not predictably repeat) I think this assertion is clearly not strictly correct – there are an infinite number of repeated sequences in pi (e.g. “123123” occurs in the first million digits) as subsets of the sets of numbers in pi.

Therefore, I postulate that there exists at least one decimal position N in pi that is followed by the exact sequence of digits 1 to N-1. Namely, pi is “3.14159…314159..”. In regular expression syntax this question would actually just be phrased as /^([0-9]+){2}/, omitting the period after 3, naturally.

It should be provable that not only such an N exists but there is an infinite set of such repeating points. I would guess that that first N would be very large, maybe larger than the number of digits of pi yet computed (in the trillions), but if there is one there are almost assuredly an infinite number of other repeating points. This set would start with an almost improbably large number and I would suppose the numbers would very quickly get ridiculously larger.

There would be different such sets for different transcendentals, so perhaps we can discuss the creation of a function “D” that defines the infinite set of these “repeating points” for any transcendental. D(π) , D(e), D(φ), D(√2), etc.

Mildly Controversial Thoughts for Further Exploration

The singularity is past. We’re already living in a collective consciousness with behaviors exceeding the capacity of any human or machine to understand, because that collective has intelligence exceeding our own by definition.

Death makes life valuable. In having a fixed lifetime, every moment living becomes precious, because every decision must be the absolute best. There’s no time to screw around with a short life. This is a good thing, so I embrace my mortality.

What you get done is more important than what you do. Leverage the work of others and become someone who causes useful things to get done. At the end of the day, nobody really cares how much work you put into something yourself. Results are it.  (This is the theme of the Four Hour Workweek and of most texts on executive management.)

People want to belong to a system. Systems can be designed and programmed.

Education gives the best returns. Few things can improve GDP as having a well-educated populace. Compare wealth generation for immigrant cultures with a strong educational focus to those that do not for a clear demonstration.

Humans think in stories. Everything is a story, from business to love to travel. Learn to tell a good story and listen to others’ stories.

The devil is in the determinism. You can and do affect the future and the fate of the world. You are writing your own destiny; make it a good one.

Consistency is more important than correctness. People can deal with predictable systems even if they are flawed – bugs have workarounds when understood. But non-deterministic issues, even if minor, will drive people batshit crazy. Separately, this is why people like drinking Coke and eating McDonald’s even if they know it is terrible – they appreciate the consistency.

Mission Statement

Who am I?
Who do I want to be?

I want to create, explore, dream
to add to the human experience
to love and inspire
and to give hope.

I want to experience the human condition
and have friends from all walks, in all places.

I want to feel pain, sorrow, loss, delight,
eat the finest and coarsest cuisines

I want to struggle and fail
to love unrequited
without becoming bitter.

I want to see God in every face.