SIDEBAR
»
S
I
D
E
B
A
R
«
The Lost Lesson Of Instant Typing
Oct 14th, 2009 by jens

Farhad Manjoo writing in Slate about Google Wave:

The trouble is, everything you type into Wave is transmitted live, in real time—every keystroke was getting sent to Zach just as I hit it. This made me too self-conscious to get my thoughts across.
… Maybe I should just delete what I’d written and say, “Twitter works because it’s simple.” But I couldn’t do that, because Zach was watching me. He could see me struggling right now—he could see that I’d gotten myself stuck in a textual cul-de-sac and that I was desperately searching for a way out without looking foolish. Now I saw Zach beginning to type: “Don’t let the live-typing get you down!” The game was up; what was the point of making a point now? I ended my thought clumsily and then resolved never to attempt to say anything very deep on Wave.

The same thing happened seven years ago with the live-typing feature that I implemented in iChat 1.0 (which was only supported for Bonjour chats.) I thought it was an awesome idea, and I’d wanted to have it in a chat program since about 1997. But it turned out that, in actual use, people hated it, for exactly the reasons Manjoo describes: it makes you self-conscious. We took it out in the next release.

(Interestingly, I hate video chat for a very similar reason. Somehow, the fact that my picture is being shown in real time to the remote person makes me horrifically self-conscious, even though it wouldn’t bother me at all to talk face-to-face with that person. I don’t know whether it’s the little preview on my screen, or the fact that the person is spookily both present and not-present, but the few times I’ve tried video-chat have been really unpleasant.)

I’m usually on the side of more technology. I believe that our online communications tools are still horribly primitive and have only scratched the surface of what’s possible. But this was a case where more technology was bad.

The low-tech alternative that lots of people use in IM,
is to write in short fragments,
each a separate message,
so the other person can see them one by one
without waiting for you to finish the whole sentence.

The difference is that you’re in control over when to send a partial message, and the other person knows you’re in control, and so on. I still think it might be possible to do this in a higher-tech way, like using a hot-key to send a partial message on demand without having the funky line-breaks, but the current approach isn’t so bad as long as you’re not embarrassed about unintentional free verse.

I could have told the Wave people about what I’d learned, except I didn’t know Wave existed until April (shortly before the public announcement), and even then I was just some guy lost in the crowd at the demos….

Part of the problem, in both cases, is that live typing is one of those Cool Demo Features that looks really awesome when showing off the app. Features like that can be dangerous because they are legitimately very useful during the app’s gestation, when exciting demos are a key survival trait; but then they can’t be removed later on because they’re so well-known, even if they turn out to be useless. Sometimes these features aren’t actually harmful to the user experience, they just make the code more complex and harder to maintain. Instant typing is both, unfortunately. (The clever sync algorithms and rapid-fire network messages Wave uses would be needed even without live typing, but the fact that they have to run on every few keystrokes, not just every minute or so, pushes those things so much harder.)

Announcing Ottoman
Sep 20th, 2009 by jens

As promised

Ottoman is a lightweight, reliable key-value store with multi-version concurrency control.

A key-value store is a persistent on-disk dictionary that maps arbitrary keys (usually short text strings) to arbitrary values (binary data of any length). There’s already a large and venerable family of libraries that do this, often referred to by the name “db”. Notable members include dbm, Berkeley DB, cdb and Tokyo Cabinet. What makes Ottoman different?

  1. It tries to provide extremely reliable storage, immune to file corruption caused by operating system crashes or power failure while writing.
  2. It stores past versions of the database, as a useful side-effect of the way it works. Applications can use this to provide features like snapshots, timelines or persistent undo. (You can prune old versions, for space or privacy reasons, by writing a fresh copy of the file.)
  3. It has very low memory overhead. The file and its contents are memory-mapped, so even large values can be accessed without having to copy them into the heap. (Opening a database only mallocs a few hundred bytes!)
  4. It allows concurrent read/write access by multiple processes. Optimistic concurrency — as used by version-control systems like Subversion — keeps file locking to a minimum: readers are //never// blocked at all, and writers are only blocked briefly (clients can’t lock the database for arbitrary periods, only for as long as it takes the accumulated changes to be written to disk.)

Why is it named after a footstool? First, as an homage to CouchDB. Secondly, it turns out ottomans can be used for storage. Third, because I have a mild fascination with Asian and Islamic history.

_…Read more at the project home page!_

Long-Overdue Upgrade
Sep 6th, 2009 by jens

I finally bit the bullet and upgraded to the latest WordPress from the seriously prehistoric version I used to run (I think it was 2.0.4 or something like that.) With the current attacks against older versions, it finally seemed like a worthwhile thing to waste an afternoon on. :-P

Almost everything should work more or less the same. I’ve taken the opportunity to move to a subdomain—jens.mooseyard.com—but the old URLs redirect. The theme is different, because my hacked-up, customized old theme probably wouldn’t have survived the upgrade intact. I can’t say I like this one better, but it’ll do.
Read the rest of this entry »

I’m Building Me A B-Tree
Aug 14th, 2009 by jens

The other day I took it into my head to implement a B+tree. Why? Because they sound neat, and I’ve done hardly any serious programming with trees in my career. (Someone, I think Buzz Andersen, once noted that there are two kinds of programmers: those who do think in terms of trees, and those who do everything with hash tables. I’m in the latter camp.)

And also because I’m a big fan of CouchDB, and really admire its elegant storage model. It’s an on-disk B-tree—no surprises there—but the file is append-only, which both makes it impervious to crash-related corruption, provides nearly lockless concurrency, and makes it easy to access earlier revisions.

[In a nutshell: Updated data values or tree nodes are appended to the file instead of overwriting the earlier versions. Since updating a node changes its location, its parent node needs to be updated too to point to the new location. This recurses up the tree, meaning any change ends up with a new root node written at the very end of the file. In fact, when you open the file you find the root by looking at the very end. Since no data is ever changed, once you open the file you’re impervious to changes made by other writers since they don’t affect anything you’re looking at.]

I’d love to use something like that for various projects, but as CouchDB is implemented in the exotic functional language Erlang, I can’t really use its storage layer as-is. So: could I implement something like it in C++?

Thus far I have a working in-memory B+tree implementation. Inserts and deletes work, and I’m working on the iterator. Even this much was harder to get working than it should have been, or so it feels. But that always seems to be true—algorithms sound straightforward when you read about them, but putting them into practice exposes you to all the details and subtleties inherent in hand-waving like “now merge the node with a neighbor”.

Actually I haven’t implemented a straight B+tree, rather a ‘top-down’ variant described by Ohad Rodeh that’s better suited to this type of application because it changes fewer numbers of interior nodes during an update.

What’s next?

  • Support for string keys (so far it just handles ints)
  • Serializing nodes to/from disk
  • Keeping track of which nodes are touched during an operation, and appending those to the file
  • Writing a trailer to the file to mark a successful update (and to link back to the previous trailer, for historical purposes.)

Sounds straightforward, but of course the devil’s in the details.

iTunes 9 Deja Vu
Aug 11th, 2009 by jens

AppleInsider reports on the iTunes 9 rumors:

“The social networking integration that we reported iTunes 9 would have seems to be part of a bigger social networking push by Apple,” the report states. “We’ve been informed that Apple has plans to tie iTunes 9 into a “Social” application that they plan to release in the future.”

This sounds like the kind of app (though separate from iTunes) that Jessica Kahn and I kept trying in vain to get Apple to build, circa 2003-2005. Maybe they’ll get some use out of our abandoned prototypes.

The report goes on to say that the new application would allow users to share their listening habits with friends [and] send music to friends”

Mike Estee and I had actually prototyped this in iChat in 2003, but the feature never got approved since there were so many more important things to add, like 3-way video conferencing. (Plus the fact that Apple execs turned white as a sheet if you said the words “send music” near them.)

Anyway, personal bitterness aside, I think it’s really amusing that Apple keeps shoving the kitchen sink into iTunes, since that has to be the single nastiest, hardest-to-extend codebase they have — it’s their last remaining Carbon app, with a foundation that dates back to Casady & Greene’s SoundJam, circa 1998.

Career Update, Part ++n
Jul 3rd, 2009 by jens

I’ve been working at Google since last August. The Big G’s hiring process is rather weird—when you interview, it’s not for any specific team. It’s only after you get an offer that you decide which team to join, of the ones with open positions.

I decided on Google Sites, which I knew and liked from its days as JotSpot, a hosted wiki with some powerful features. It ended up not being the right place for me, for a couple of reasons:

  • Currently, Sites’ priorities are in website publishing, as a replacement for Google Page Creator (which is being phased out soon.) It’s quite good at it, but I’m less interested in that than in collaboration features.
  • Google’s server-side infrastructure is really, really, really huge and complex. There is an endless landscape of internal technologies and tools—the few that have been described in public (MapReduce, BigTable, Chubby, etc.) are just the tip of the iceberg. I have discovered that I am not very interested in this kind of stuff, and I quickly became frustrated by the deluge of technologies I needed to learn to get things done.
  • Running a large web service is like running a nuclear power plant or an electric power grid. It requires 24/7/365 monitoring, and at that scale, anything that can go wrong will go wrong, and frequently does. I do not have the right temperament for working with this, especially not when it comes to taking turns carrying a pager that wakes me up at 4AM because some service’s latency has gone above 300ms.
  • I’ve been vocal about my frustration with centralized systems; Google’s websites are kind of the ultimate in that (even though, ironically, they’re implemented as P2P-like networks internally, as I believe all large web operations are today.)

The good news is that Google encourages transfers between teams, and makes it easy to do so. The near-total transparancy inside the company makes it easy to find out everything that’s going on, and there’s a well-designed website for engineering transfers that helps you find teams that need people. I even went to an internal job fair.

I’ve now ended up working on Chrome, Google’s web browser. The team I’m on is responsible for implementing HTML 5 features, as well as designing and implementing other new features (for standardization) that will help web apps become as powerful as native apps. Much of what we do will go into the WebKit source tree, where it will also directly benefit Safari, Android, the Palm Pre, and other WebKit-based browsers.

In fact, everything I work on (more or less) is going to be open source. Both the WebKit and Chromium source trees are public. You can view, if you care to, the one patch I’ve contributed so far and the one that’s currently out for review. That’s kind of mind-blowing, in a good way, to me, steeped as I am in the secrecy of Apple.

I’m pretty excited by this. There are quite a lot of things I’m interested in working on—client-side storage, local apps, drag-and-drop, better font support, menus, even far-out stuff like peer-to-peer networking. Forward in all directions!

Murky Mailing List And Progress Report
May 8th, 2009 by jens

I’ve set up a Google Groups mailing-list for Murky, my Mercurial version-control app. It’ll be for both developers and users of the app.

Speaking of “developers”, I’ve been really pleased with the reaction to my initial source-dump to Bitbucket! So far, in nine days:

I’m especially happy about the patches I’ve received; I’ve gotten some in the past, but never this many or so quickly. Part of this is thanks to Bitbucket, which (like its inspiration, Github) makes it so easy to share code and manage contributions. I’ve been getting more contributions to MYNetwork as well, since hosting it there.

My initial motivation was like Tom Sawyer whitewashing the fence—I was hoping someone else would take over the project, since I’d run out of steam and hadn’t done any work on it in a while. But what happened instead is that the attention got me enthused to work on it again, so I’ve made a lot of progress on it this past week. That’s a good thing!, just not what I expected.

Now I think it’s about time to put out an actual easy-to-download binary release of Murky, as some of the main bugs have been squashed and I have more confidence that it works for other people besides me. I’ll probably do that this weekend, and mention it here as well as on the new mailing list (don’t forget to subscribe.)

Murky: A Mercurial Client App
Apr 29th, 2009 by jens

After repeated prodding, I’ve finally gotten off my butt and released Murky, a GUI client app for the Mercurial version-control system. I’ve been working on it for over a year, and using it a lot myself; I had always planned to open-source it “when it’s ready”, but never actually got around to doing the necessary cleanup and uploading.

I’m happy to say that Murky is finally blinking in the light of day, with its own Mercurial [natch] repository on Bitbucket.

Setting Expectations

Murky isn’t “done”, and this isn’t a 1.0, or even a beta, release. I’m not even providing a compiled app to download, just the source repository. No one but me has ever tested it. About nine months ago Murky became “good enough” for what I usually do, and I’m a happy customer, but I’ve lost the impetus to keep adding things to it. I hope some other people will grab the baton and improve it further. Then we can throw a party for a real 1.0 release, with several names on the About box.

Obligatory Screenshot

Features

Here’s what it can do so far:

  • Create a new repository from an existing folder
  • Clone a remote (or local) repository
  • View the revision history (log) of a repo as a table view, complete with * a visual graph showing the inheritance of revisions
  • View the source tree, and the contents of files, from any past revision
  • See which files were changed, added or removed in any revision
  • Compare revisions of files, either visually with FileMerge, or as straight ‘diff’ output
  • Add or remove files from the local working tree
  • Commit (or revert) changed files
  • Update to a specific revision (past or present)
  • Push to or pull from remote repositories
  • Automatically detects filesystem changes made by other apps (or by the ‘hg’ tool) and updates the display immediately
VoodooPad 4.1 and VP Reader for iPhone Released
Mar 7th, 2009 by jens

Flying Meat Software has released a VoodooPad Reader app for iPhone, as a complement to their great desktop wiki. So now you can sync your VP docs to your iPhone and access them on the go, with or without a network.

What’s especially cool, to me, is that VP Reader uses my MYNetwork library and BLIP protocol for its networking. Gus calls it “an awesome network library … It made moving pages from VoodooPad to the iPhone over the network completely painless. A++ would recommend again.” Thanks, Gus! I believe this is the first appearance of BLIP in a shipping product, since my own apps remain in limbo.

The VP Reader source code is available if you want to see how it was done.

Music, Alone
Feb 24th, 2009 by jens

The feelings created by music are so strong, for me, but so ineffable. The problem of perception is usually described using color — how can we know if the visual sensation I call “red” is anything like the one you call “red”? — but only gets worse as you ascend to higher order perceptions, where even names become harder to apply. What do you call the feeling incited by “Guernica”, and even if you find the same words I would, is it the same feeling? And yet vision is our strongest, highest-bandwidth, most describable sense. We struggle to describe sound without using the technical terms of musicians, or vague metaphors.

It doesn’t help that so much of the music I like is so inward-focused: the guitarist gazing (not at shoes) at effect pedals, the producer sliding waveforms around a timeline, the listener bracketed in headphones like my picture above.

Everyone wants their experience of music to be shared. To play an instrument or sing for others, to blast the song from car speakers. To identify with music meant to shock, and use it to shock others. To attend a concert and know that those around you are hearing and feeling the same thing you are, right then: sitting following the intricacies of Bach, or exploding in a mosh pit. Drugs of many kinds help to collapse this emotional waveform, unite a group into a Bose-ian condensate, whether it’s alcohol to bring out desire and anger, or Ecstasy to turn arpeggios and filter sweeps into spinal shivers and universal love.

Nowadays I experience music by myself, mostly on headphones, like right now on the bus to work. Little circuits inside them are actively removing the signs of the outside world, leaving a weird sibilant hush that I fill with the noise I choose. My eyes are on a screen where I sort my music into playlists, or read what other people say about the music they love, and sometimes reach out and listen to their music myself.

I take little piles of songs and chain them together, finding the little hooks at the ends where they fit, making daisy chains to share with people. Is this creation? I didn’t create the songs, but joining them can make something new, like a work of collage. I would love to use the songs as words given to me, and string them together to tell a story. In some of the mixes I’ve made I can hear some of that: abstract arcs of emotion flying across 80 minutes, or juxtapositions that change the sounds on either side into something new that reflects the other.

The worst part about creation is finishing, and throwing the end product out the door into the sunlight where it blows away in sheets. Diana tells me the abalone are dying out because they reproduce by scattering their seed into the ocean and hoping the eggs and sperm will meet; and as their numbers decrease, so does the likelihood of this success. In some ways the progression of my musical tastes has been like that. The Internet has been a tremendous boon, but the distance involved always makes me doubt, as the notes and bytes fly away out of sight, that they’ll ever find a mate, or if they do, whether I’ll even know.

»  Substance:WordPress   »  Style:Ahren Ahimsa