Thought Palace

Next I need to talk about networking; having an identity and minting certificates isn’t very interesting until you can connect to someone else.

Point-to-Point Communications.

When one Cloudy peer wants to communicate with another one, it opens a TCP socket to its IP address —

[Hang on, there are two issues I suddenly glossed over in that last phrase. First, how did this peer find out the others’ IP address? These are just random computers, not servers, so they don’t have their own domain names or even stable addresses. This is indeed a problem with any unstructured peer-to-peer network, but the solution involves things I won’t get to until the next installment, in an unfortunately but necessary violation of layering.]

[Oh, and issue #2 is that most home computers are now behind Network Address Translators (usually some kind of WiFi base station or broadband router), which means they don’t have their own real IP addresses and can’t receive incoming connections. Fortunately, most NATs now support protocols that allow clients to open listening ports to the outside world, and doubly fortunately, Mac OS X 10.5 includes an API for making such connections. Cloudy opens such a port whenever it finds itself behind a NAT.]

— and [...]

Continuing from the previous Cloudy post …

At the root of Cloudy is the means for creating and establishing identity. A lot of peer-to-peer systems treat the peers mostly as interchangeable anonymous nodes, often deliberately so, but Cloudy is a social system.

Quick Crypto Recap.

The identity and security layers of Cloudy are tightly intertwined, because identity without security is useless. And security is accomplished entirely through cryptography, because the centralized alternatives like locking all of your servers up in a closet don’t apply. Cloudy doesn’t do anything new cryptographically (wisely so), but for the benefit of those who aren’t familiar with it, here’s a superficial overview of the off-the-shelf tools I’m using:

Cryptographic Hashes, or, Digests.

Like any hash algorithm, a cryptographic hash converts a block of data of arbitrary length into a short fixed-length output; the same input always produces the same output; and even the slightest change to the input should produce an entirely different output. Unlike a regular hash, two different inputs should never result in the same hash output. (That’s “never” in the practical sense: collisions are mathematically inevitable, but it should impractically long, ideally millions of years, to find one.) And it should be infeasible to identify anything [...]

Continuing from Unstealthing, Incrementally…

I have many ideas for applications, but most of them seem to rely on similar kinds of infrastructure, in particular a distributed, secure application-level messaging system. Unfortunately, this doesn’t really exist yet, at least not in any form that meets my needs.

What am I talking about here? More colloquially, it’s a mechanism for letting applications all over the network send messages to each other, without requiring a central server, and without allowing messages to be eavesdropped upon or faked.

Let’s take it one buzzword at a time…

Distributed.

I don’t know about you, but I’m getting fed up with centralization. It happens because it’s the path of least resistance: buy a domain name, rent a server, buy more servers and stick a load-balancer up front as your user base grows. It’s solving problems by throwing hardware at them. The end result can certainly work fine, but too often it’s fragile: both technically (site goes down, ten million users get pissed off) and politically (just one domain for China to censor, one company for France to file lawsuits against.)

In social software especially, there’s an additional type of cultural fragility, since the owners, implementors and users of a big social site [...]

I got about 14 minutes of fame back in January with a blog post, wherein I grumbled about (among other things) how I disliked Apple’s culture of secrecy, and announced that I’d left Apple to work on my own, unspecified, project. In the intervening three months, I haven’t said anything about what that project is, almost as though it were … secret.

The irony of this is not lost on me.

Admittedly, there are things about my app that I do want to keep under my hat until they’re ready to show off in their full glory. I want to spend my one minute of remaining fame wisely; ideally accompanied by a large friendly “BUY NOW” button on my website.

But the main reason I haven’t been talking is just that I’ve been lazy. Well, not lazy, but focused on coding rather than talking. I’m mindful of a quote by (I think) John Crowley, which goes something like:

—There are two kinds of poems: the ones you write, and the ones you talk about writing. They’re both important, but never get mixed up about which kind you have.

I feel like I’ve been talking about writing this type of app (if only to myself) for [...]

Cabel Sasser, of indie developer Panic, reports from Japan:

“Within minutes of riding on the first trains in Japan, I notice a significant change in advertising, from train to television. The trend? No more printed URLs. The replacement? Search boxes! With recommended search terms!” [*]

He goes on to note how common it is for people to type URLs or domain names into their browser’s search box instead of the address field. To American geeks this seems clueless, but Cabel points out that in Japan it makes more sense, since URLs are in a foreign alphabet, so search words are much more memorable.

First off, this instantly reminded me of two favorite jokes:

Homer Simpson, picking up the phone: “Operator! Get me the number for ‘911’!”

Scott Pilgrim, on finding out that the cute girl he saw at a party in Toronto works as a delivery courier for Amazon.ca: “Hey, Amazon.ca, that’s the online bookstore or whatever, right? … What’s the website for that?”

But seriously: This is another example of Zooko’s Triangle, which basically says “names cannot be global, securely unique, and memorable, all at the same time”. URLs are global and unique, but not memorable, especially not in Japan; search terms are global and [...]

I bow to my esteemed colleague Craig Hockenberry’s greater experience in iPhone development; but I must disagree with his take on the infeasibility of background applications. He gives two reasons why networked apps shouldn’t run in the background — one technical and one user-interface.

Battery life.

The heart of the problem are the radios. Both the EDGE and Wi-Fi transceivers have significant power requirements. Whenever that hardware is on, your battery life is going to suck. My 5 minute refresh kept the hardware on and used up a lot of precious power. *

My immediate response is that, yes, polling is inefficient. Everybody knows this; but it’s also easy to implement, which is why way too many protocols use it. Normally the problems with polling first manifest as scalability problems on the server (as Twitter quickly discovered), but in the case of mobile devices, polling kills battery life.

So it’s a good thing that none of the real instant-messaging services poll. AIM, Jabber, MSN, Yahoo and ICQ all open a socket at login and leave it open, sending data only when necessary. If you suppress buddy-list updates while the app isn’t active, then data only needs to be sent when you send or receive [...]

I had lost this historical document for a long time, but finally found it the other day on an old backup CD. It’s the original 1997 sketch I made of a chat user interface based on speech balloons.

Dear Lazyweb,

The project I’m working on will be using cryptographic certificates in a distributed web-of-trust model a little like that of PGP. It will also use certs as more than just proofs of identity. Given that I’ll be writing a lot of code using certs, I want to avoid the nastiness of X.509 whenever possible.

After thinking about this a while, it seems to me that RDF ought to be a good way to represent certs, since it describes arbitrary types of relationships between entities (e.g. FOAF), and allows them to be composed in complex ways. And there are a lot of tools available for parsing/storing/querying RDF.

Unfortunately, I know very little about RDF so far, or about the uses to which it’s being put. I’ve been looking, but I haven’t found any existing schema yet for using RDF for cryptographic certificates. Does anyone know of such a thing, or something related?

(The closest thing I know of is SDSI, a Simple Distributed Security Architecture, which was inspirational to me in showing how one can use general-purpose data structures like S-expressions to describe certs and form a web of trust. But SDSI and its successor SPKI seem to be dead, sadly, and nothing [...]

I finally made myself a Facebook account, mostly to see what it’s like. Overall, I’m pretty impressed: the UI is nicer than most such sites, particularly the still-antiquated LiveJournal and the disaster that is MySpace. The biggest issue there seems to be that the main profile page absolutely doesn’t scale up to handle the exploding number of apps/widgets people are stuffing into it, so you end up with mile-long profiles containing box after box of junk.

But the most interesting thing I noticed is how the service has no visible identifiers for user identities. Unlike most centralized services, there’s no unique username to pick. I assume that, internally, each account requires a unique email address, but that address plays very little role in the user experience, apart from its use in helping people find their existing contacts’ profiles. The service does assign a unique number to every profile, and this shows up in profiles’ URLs, but it never seems to appear in the page itself. So there’s no obvious way to say “this is my Facebook ID”, other than pasting in the completely non-mnemonic URL of your profile page. And conversely, the visible identifiers you see for other members are simply [...]

It’s a sure sign that wikis are going mainstream when one appears for a video-game console. “ZackAndWiki” has the requisite goofy name (like TikiWiki or WikkaWiki), but once you try it out, you’ll find it approaches its job very differently than you’re probably expecting.