Thought Palace

Last year I wrote a series of blog posts about a peer-to-peer system called Cloudy that I was developing. I was going up the stack, from messaging to identity, but didn’t finish documenting all the layers I’d built. I mostly stopped working on Cloudy after I went back to gainful employment, but I keep thinking about this stuff.

“Lakitu”?

I’ve since heard about another unrelated project nicknamed Cloudy; and the whole term “cloud” has gotten so debased in the past year that it now stands for outsourcing to giant hidden server farms, which is the antithesis of what I stand for. So I’ve decided to use the name Lakitu instead. Nintendo fans will recognize Lakitu as a bit character in the Mario games—he’s a goggled turtle who rides a little one-seater cloud. This makes him an appropriate mascot for P2P technologies, I think.

[I’m sure Nintendo has a trademark on the character, but they don’t appear to have copyrighted the word “Lakitu”. He’s not even known by that name in Japan, where he’s called “ジュゲム” or “Jugem”. I have been unable to find out what “Lakitu” means or why they decided to use it in the English translation. I could also note threateningly [...]

The GameKit framework in iPhone OS 3.0 is very interesting to a Bonjour / P2P head like yrs truly. It basically provides a very easy-to-use API for ad-hoc group formation and many-to-many messaging on a local network. Great for games, of course, but also for many other types of social apps. (I just saw a report on a dev forum that somebody had whipped up a basic chat app in about 15 minutes.)

GameKit uses BlueTooth networking; that lets it work where there’s no WiFi, but it also limits the range. BlueTooth covers just a few meters, whereas a WiFi network connected to an Ethernet subnet can easily cover a whole floor of a building.

My MYNetwork framework seems like a good way to bridge that gap. The TCP connection classes provide the Bonjour discovery and makes point-to-point connections, and the BLIP protocol lets you send data blobs over those connections.

It should be pretty straightforward to build some classes that are plug-compatible with the GameKit network classes but use MYNetwork. Then iPhone apps could easily support both protocols, and compatible Mac apps could be developed. Anyone want to try it?

[Note: I’m only referring to information that was publicly discussed at Apple’s press [...]

So, Web 2.0’s heyday is over, and somewhere out there, Web 3.0 is slouching toward us waiting to be born. What will it be?

There’s really no such single thing as “Web x“, of course. And all predictions are really just wishes. That being said, my wish is that Web 3.0 will be about distributed systems. To oversimplify:

Web 1.0 built up big brand-name websites with their own content—things written by them, or repurposed from the media companies that owned them, or stuff to buy.

Web 2.0 embraced “user-created content” and interaction between users. The content creation has become less centralized, outsourced to whomever wants to register an account and post stuff, but the sites managing, storing and serving the content are still centralized.

Web 3.0, I hope, will take the decentralization to the software, and the storage. Monolithic web apps run by huge server farms—Facebook, Blogger, Twitter, Flickr, etc.—will be at least in part supplanted by apps that users run locally (or at least ‘nearby’) and which share data among each other.

Why is this important?

Centralization creates concentrations of power, and that’s dangerous. The people who run the servers have total control over your (and everyone’s) data. They can snoop at it (however private [...]

I need a few brave people to test a pre-beta app for me. No, this is not Cloudy; it’s another app I’ve been working on in parallel. It’s called Your Move, and it’s the expanded version of my GeekGameBoard sample code. It lets you play board games against a human opponent; either at the same machine, over a local network, or by sending moves via email or iChat.

Obviously this would make a great iPhone app, and now that I have my iPhone developer certification I’ll start working on that. But for now it’s Mac-only.

To test Your Move you need to

Have Mac OS X 10.5.
Enjoy playing board games … in particular, Go and/or American checkers, as those are the main games it knows so far.
Have at least one friend who will play board games with you (I’ll do in a pinch, but I don’t have time to play against everyone. Plus, ironically, I suck at Go and checkers.)
Be willing to tolerate bugs, and committed to sending in bug reports, crash logs and such.

If you’re interested, please

Register an account on the Mooseyard projects website ; then
Email me at “jens” c/o this domain and tell me your username, so I can give you [...]

I’ve released the source code to my “BLIP” protocol implementation, as part of a project I call “MYNetwork”, for “Mooseyard Networking Libraries”. API documentation is available online, and you can check out or browse the source code from its Mercurial repository. For the really curious, there’s even a sketchy overview of the protocol’s wire format.

It’s working quite well for me in Cloudy; it’s been a while since I’ve found any outright bugs, although I know there’s more work to do on performance and features. It’s not production-ready code yet, but I think it’s ready for people to start using!

For now you can leave feedback in comments to this post, or by email to jens at this domain. If there’s enough interest, I could set up a mailing list.

Here’s the description of BLIP from the API docs…

What’s BLIP?

BLIP is a message-oriented network protocol that lets the two peers on either end of a TCP socket send request and response messages to each other. It’s a generic protocol, in that the requests and responses can contain any kind of data you like.

BLIP was inspired by BEEP (in fact BLIP stands for “BEEP-LIke Protocol”) but is deliberately simpler and somewhat more limited. That [...]

I’ve got my new BLIP protocol all implemented now. After my previous post on Monday:

On Tuesday I implemented message metadata.
On Wednesday I got SSL working (configuring the “server” side to verify the “client’s” cert was difficult.)
On Thursday I put Cloudy up on blocks, pried out Vortex and my Obj-C wrapper library, and replaced them with BLIP.
And on Friday (today) I debugged.

Cloudy’s back up and running, and all its features work. So, that makes one week of effort to implement the networking layer from scratch (I started sketching and coding on Saturday). Really makes me regret spending several times that on the previous library—writing an Obj-C API, fixing bugs, adding features. Still, I’m sure all that experience helped me implement BLIP so quickly.

(The code for BLIP is quite separable from Cloudy; but it does use some lower-level utilities and network classes I wrote, so it’s not quite standalone. And it needs documentation before it’ll make sense to anyone. How soon I release it depends on how much interest there is, so let me know if it’s something you’re interested in using.)

…and which side of that line am I on? Not in general; just in respect to my latest decision in Cloudy. It’s the old “make vs. buy” trade-off, or “write vs. reuse” in this case: do you go with an existing library, even if it’s problematic, or do you write your own implementation from scratch?

What am I talking about? The networking code in Cloudy. From the very beginning I wanted to use BEEP, a generic and flexible protocol for sending request/response messages over a socket. It has good support for parallelism, nice abstractions like multiple channels and feature negotiation, and supports SSL.

The BEEP implementation I’m using is Vortex. It has the benefits of existing (there aren’t a lot of BEEP implementations around), being written in native code, and being within my capabilities to get running on Mac OS X. Unfortunately it’s also got a very complex and unintuitive C API, spawns lots of threads and calls my code from them (so I have to deal with thread-safety), and isn’t quite finished yet. So over the months I’ve put a lot of work into writing an Objective-C API, figuring out how to get that working reliably, and diving into the Vortex [...]

I’ve started using NSOperation in a few places in Cloudy, which means I’m backsliding into using threads and locking and so forth. It definitely makes writing network code easier than Cocoa’s asynchronous API, but I really don’t want to get into a morass of threads.

What I’d really like to use are Actors. In a nutshell, an Actor is an object that has its own [cooperative] thread and message queue. Actors interact by message-passing instead of shared state. The idea is to eliminate the need for standard synchronization primitives like semaphors and locks, and get rid of the race conditions and deadlocks that plague multi-threaded programs.

The page I linked to above is from Revactor, a new Actor implementation for Ruby 1.9; Actors are also built into languages like Erlang and Io.

The only hard part about implementing Actors in Obj-C appears to be the underlying dependency on coroutines. Steve Dekorte [author of Io] has a C coroutine implementation ; when I discovered that last year, I then found an Objective-C wrapper for it, but that relies on a HigherOrderMessaging library which is incompatible with OS X 10.5 and hasn’t yet been updated.

So I’ve gone DIY.

I started with Steve Dekorte’s coroutine library, but [...]

Continuing from the previous Cloudy post …

The first time you connect to someone, how do you establish that digital identifier you’re communicating with is the human being you think it is? This is surprisingly difficult to do, because it’s prone to what cryptographers call the “man-in-the-middle attack”.

(Those of you already wearing tinfoil hats can skip past the general explanation, down to “What Cloudy Does”.)

1. A Quick Overview Of Verification Attacks.

First, consider the most obvious attack: simple spoofing.

Spoofing.

Let’s suppose there’s an instant-messaging UI, and while working at home you receive a message from someone with an unknown key, whose nickname is “AliceLiddell”, which happens to be the name of a co-worker.

“AliceLiddell”: yo, this is alice
You: hi alice, what’s up?
You add this identity to your friends-list.
Alice: i need the admin password to the web server to fix a template
You: oh ok, it’s wend4743kt
Alice: kthxbye

Fifteen minutes later your company’s website is pwned by the hacker who posed as Alice. All he had to do was create a new identity with her name as the nickname, and pretend to be her.

How do we get around this? You might think that asking questions before accepting someone’s claimed identity would help, and it does help [...]

Next I need to talk about networking; having an identity and minting certificates isn’t very interesting until you can connect to someone else.

Point-to-Point Communications.

When one Cloudy peer wants to communicate with another one, it opens a TCP socket to its IP address —

[Hang on, there are two issues I suddenly glossed over in that last phrase. First, how did this peer find out the others’ IP address? These are just random computers, not servers, so they don’t have their own domain names or even stable addresses. This is indeed a problem with any unstructured peer-to-peer network, but the solution involves things I won’t get to until the next installment, in an unfortunately but necessary violation of layering.]

[Oh, and issue #2 is that most home computers are now behind Network Address Translators (usually some kind of WiFi base station or broadband router), which means they don’t have their own real IP addresses and can’t receive incoming connections. Fortunately, most NATs now support protocols that allow clients to open listening ports to the outside world, and doubly fortunately, Mac OS X 10.5 includes an API for making such connections. Cloudy opens such a port whenever it finds itself behind a NAT.]

— and [...]

Continuing from the previous Cloudy post …

At the root of Cloudy is the means for creating and establishing identity. A lot of peer-to-peer systems treat the peers mostly as interchangeable anonymous nodes, often deliberately so, but Cloudy is a social system.

Quick Crypto Recap.

The identity and security layers of Cloudy are tightly intertwined, because identity without security is useless. And security is accomplished entirely through cryptography, because the centralized alternatives like locking all of your servers up in a closet don’t apply. Cloudy doesn’t do anything new cryptographically (wisely so), but for the benefit of those who aren’t familiar with it, here’s a superficial overview of the off-the-shelf tools I’m using:

Cryptographic Hashes, or, Digests.

Like any hash algorithm, a cryptographic hash converts a block of data of arbitrary length into a short fixed-length output; the same input always produces the same output; and even the slightest change to the input should produce an entirely different output. Unlike a regular hash, two different inputs should never result in the same hash output. (That’s “never” in the practical sense: collisions are mathematically inevitable, but it should impractically long, ideally millions of years, to find one.) And it should be infeasible to identify anything [...]

Continuing from Unstealthing, Incrementally…

I have many ideas for applications, but most of them seem to rely on similar kinds of infrastructure, in particular a distributed, secure application-level messaging system. Unfortunately, this doesn’t really exist yet, at least not in any form that meets my needs.

What am I talking about here? More colloquially, it’s a mechanism for letting applications all over the network send messages to each other, without requiring a central server, and without allowing messages to be eavesdropped upon or faked.

Let’s take it one buzzword at a time…

Distributed.

I don’t know about you, but I’m getting fed up with centralization. It happens because it’s the path of least resistance: buy a domain name, rent a server, buy more servers and stick a load-balancer up front as your user base grows. It’s solving problems by throwing hardware at them. The end result can certainly work fine, but too often it’s fragile: both technically (site goes down, ten million users get pissed off) and politically (just one domain for China to censor, one company for France to file lawsuits against.)

In social software especially, there’s an additional type of cultural fragility, since the owners, implementors and users of a big social site [...]

I got about 14 minutes of fame back in January with a blog post, wherein I grumbled about (among other things) how I disliked Apple’s culture of secrecy, and announced that I’d left Apple to work on my own, unspecified, project. In the intervening three months, I haven’t said anything about what that project is, almost as though it were … secret.

The irony of this is not lost on me.

Admittedly, there are things about my app that I do want to keep under my hat until they’re ready to show off in their full glory. I want to spend my one minute of remaining fame wisely; ideally accompanied by a large friendly “BUY NOW” button on my website.

But the main reason I haven’t been talking is just that I’ve been lazy. Well, not lazy, but focused on coding rather than talking. I’m mindful of a quote by (I think) John Crowley, which goes something like:

—There are two kinds of poems: the ones you write, and the ones you talk about writing. They’re both important, but never get mixed up about which kind you have.

I feel like I’ve been talking about writing this type of app (if only to myself) for [...]

The beach that fall was overrun by glass lizards; I had never seen them before. I knew there had been none at the beach when we had visited during the summers (I could remember four summers, and my parents said there had been more.)