BLIP: Come ‘n’ get it!
I’ve released the source code to my “BLIP” protocol implementation, as part of a project I call “MYNetwork”, for “Mooseyard Networking Libraries”. API documentation is available online, and you can check out or browse the source code from its Mercurial repository. For the really curious, there’s even a sketchy overview of the protocol’s wire format.
It’s working quite well for me in Cloudy; it’s been a while since I’ve found any outright bugs, although I know there’s more work to do on performance and features. It’s not production-ready code yet, but I think it’s ready for people to start using!
For now you can leave feedback in comments to this post, or by email to jens at this domain. If there’s enough interest, I could set up a mailing list.
Here’s the description of BLIP from the API docs…
What’s BLIP?
BLIP is a message-oriented network protocol that lets the two peers on either end of a TCP socket send request and response messages to each other. It’s a generic protocol, in that the requests and responses can contain any kind of data you like.
BLIP was inspired by BEEP (in fact BLIP stands for “BEEP-LIke Protocol”) but is deliberately simpler and somewhat more limited. That results in a smaller and cleaner implementation, especially since it takes advantage of Cocoa’s and CFNetwork’s existing support for network streams, SSL and Bonjour. (BLIP is currently a bit under 2,000 lines of code, and the rest of the MYNetwork classes it builds on add up to another 1,500. That’s at least an order of magnitude smaller than existing native-code BEEP libraries.)
BLIP Features:
- Each message is very much like a MIME body, as in email or HTTP: it consists of a blob of data of arbitrary length, plus a set of key/value pairs called properties. The properties are mostly ignored by BLIP itself, but clients can use them for metadata about the body, and for routing (i.e. something like BEEP’s “profiles”.)
- Either peer can send a request at any time; there’s no notion of “client” and “server” roles.
Multiple messages can be transmitted simultaneously in the same direction over the same connection, so a very long message does not block any other messages from being delivered. This means that message ordering is a bit looser than in BEEP or HTTP 1.1: the receiver will see the beginnings of messages in the same order in which the sender posted them, but they might not end in that same order. (For example, a long message will take longer to be delivered, so it may finish after messages that were begun after it.) - The BLIPDispatcher class lets you (optionally) use NSPredicate-based rules to route incoming requests to handler methods. The predicates can examine all of the message metadata, so they can act much like the routing functionality of web frameworks like Rails and TurboGears.
- The sender can indicate whether or not a message needs to be replied to; the response is tagged with the identity of the original message, to make it easy for the sender to recognize. This makes it straighforward to implement RPC-style (or REST-style) interactions. (Responses cannot be replied to again, however.)
- A message can be flagged as urgent. Urgent messages are pushed ahead in the outgoing queue and get a higher fraction of the available bandwidth.
- A message can be flagged as compressed. This runs its body through the gzip algorithm, ideally making it faster to transmit. (Common markup-based data formats like XML and JSON compress extremely well, at ratios up to 10::1.) The message is decompressed on the receiving end, invisibly to client code.
- The implementation supports SSL connections (with optional client-side certificates), and Bonjour service advertising.
May 26th, 2008 at 7:18 AM
Hi Jens, this is all looking very cool indeed. Out of interest, how difficult do you think it would be to implement the protocol on another platform where CFNetwork and friends aren’t available?
May 26th, 2008 at 9:01 AM
Mike — That depends on the platform! If it has a framework with comparable functionality, as Ruby/Python/Java/.NET do, then it should be pretty easy.
The details of the protocol are in the file “BLIP/BLIP Overview.txt”. It was hastily written, but I think everything needed is in there.
May 26th, 2008 at 12:38 PM
Are you planning on combining this with your Cocoa coroutine/actor work?
May 27th, 2008 at 2:36 PM
This looks pretty interesting. What kind of applications do you see this working well for? What did you have in mind when you created this protocol?
I am interested in doing some phone to phone networking on the iPhone for a simple turn based game. Do you think that BLIP would be well suited?
Thanks
Brian
May 27th, 2008 at 4:08 PM
Brian — This is what I had in mind. (Or you might want to start back at the beginning of that thread.) BLIP has now replaced BEEP in Cloudy.
BLIP is good for sending large numbers of lightweight request/response pairs, or for large interleaved requests (since it’s better at pipelining than most HTTP implementations.) It also provides both sides of the connection symmetrically, whereas with HTTP you need a separate client and server.
Phone-to-phone sounds like it would work OK with Bonjour; but from what I’m told, the cell carriers don’t allow incoming TCP connections over EDGE.
May 27th, 2008 at 5:08 PM
I don’t blame you for not using BEEP. I got a long ways into a BEEP implementation in ObjC/CFNetwork before we stopped development on the project in question; it was fascinating but insanely complex and I never got the hang of the SSL stuff.
I think the majority of peer-to-peer applications could do with the simpler heuristics of BLIP—so well done indeed!
May 30th, 2008 at 12:41 PM
What RFC’s do BLIP implements? Thanks.
May 30th, 2008 at 2:35 PM
twobyte — it doesn’t implements any RFCs. I just made it up a few weeks ago. Of course, if people find it useful and it gets used, it might be worth documenting it as an RFC at some point.
If you need a protocol like this that’s been standardized, look at BEEP. (But be warned that BLIP came about precisely because of my dissatisfaction with available BEEP implementations.)
May 30th, 2008 at 10:01 PM
Jens, I understand that you have Apple’s mentality, but was it possible that you could develop “BLIP” within BEEP framework? You know people write down RFCs not just for fun.
May 30th, 2008 at 11:19 PM
twobyte — No, it was not possible; not within my time constraints. BEEP is a much more complicated protocol and I did not want to have to implement it. You know people create new protocols not just for fun?
In my case, as described in recent blog posts, I had run into serious problems with the BEEP implementation I’d been using for five months. I considered how much time I’d already dumped into fixing and improving that library (and how awful its code was to work with) and decided that I had to rip it out and use something else. But there wasn’t anything else available. So I was left with writing my own. And I did not have the time or energy to attempt to implement the BEEP protocol, so I sat down and worked out a minimal set of features that I needed, and decided on the easiest way to represent them, in kind of a BEEP-like way, in a protocol.
If you really want to use BEEP, then go have a look at Vortex. Just be aware that the source code is very ugly C, it uses more threads than it needs to, it doesn’t implement the whole BEEP protocol (very limited MIME support) and by omitting MIME headers it happens to be fatally incompatible with any compliant BEEP implementation.
Or you could just write your own BEEP framework in Obj-C. But either way, stop bugging me about what I decided to do, for my own purposes, and then kindly shared with the world.
July 2nd, 2008 at 2:47 PM
Look at that, I just needed to read one more post. That was great instant satisfaction.