Comments on: hash musings

By: George Bailey

George Bailey — Wed, 20 Feb 2008 14:03:28 +0000

Considering the size of the library, you’d have more collisions than a Costa Rican highway.

By: fluffy

fluffy — Tue, 19 Feb 2008 21:10:38 +0000

Well, I think the whole relevance of the universal library as a thought experiment (in a CS context, anyway) is that it’s an application of the pigeonhole principle and basically says that the only way to index the content is to have the content itself.

By: Jens Alfke

Jens Alfke — Tue, 19 Feb 2008 21:03:22 +0000

Fluffy — I know. The QR-Code has lots of overhead for error-correction, which I’m sure would mess up the pretty pictures. But somehow this seems like a minor issue compared with the feasibility of the Universal Library in the first place.

By: fluffy

fluffy — Tue, 19 Feb 2008 17:10:18 +0000

6 megapixels != 6 million bits of useful information though. Also no two scans of a book (much less photographs) will have the same bit-for-bit information even if all alignment/registration/etc. is perfect. There’s always sensor noise and so on, and that noise also means that even if you heavily quantize things, then the fringe area between quanta will still vary.

By: Jens Alfke

Jens Alfke — Tue, 19 Feb 2008 16:22:14 +0000

Another thought — ISBNs reminded me of barcodes. Borges's librarians would appreciate having barcodes on their books so they can scan them when they're checked out or returned. But the barcode would need to be something like six million bits long, to number each book. A UPC-style barcode of that capacity would be about ten miles long and would stick off the edge so far they'd have to wrap it around until the book looked like one of those prizewinning balls of string. Not feasible. But if you used a two-dimensional barcode like a QR-Code™, it might be feasible. You can fit six megapixels on the back of a book, we do it all the time when we print out our snapshots. You'd need a really high-resolution barcode scanner, though. Then there's the fun of considering that the barcodes would contain every possible combination of those six million black and white pixels, meaning every possible halftoned image at that resolution. Which of course means that... They print the barcode on the front of the book, and make it the cover. The question is, how many of the vast numbers of books with a reasonable facsimile of the Mona Lisa as the cover barcode have contents that discuss the Mona Lisa, or Leonardo Da Vinci (or Marcel Duchamp, for that matter?)

By: Jens Alfke

Jens Alfke — Tue, 19 Feb 2008 16:07:50 +0000

No cigar. There’s nothing intrinsically binary about a hash function; the algorithms are just specified and implemented that way to make them fast on computers.

I was thinking of the fact that any useable hash function would completely break down on a data set the size of Borges’s library. According to the Wikipedia article, it contains nearly 2 x 10^1,834,097 books. But even the longest SHA function only has 2^512 (about 10^154) possible values. That means there are very roughly 10^1,833,943 collisions for every hash value. Oops. To get around that, we’ll need an SHA-10,000,000 algorithm.

But then again, you could argue that Dewey Decimal (or Library of Congress) numbers aren’t unique either; they just narrow down a particular topic, so there are probably dozens of books with any particular number. Which makes SHA a good analogy. The real fallacy would be to equate SHA with ISBN, I suppose.

By: fluffy

fluffy — Tue, 19 Feb 2008 16:05:04 +0000

Or is the fallacy that the DDC is intended for browseable categorization and a hash is only useful for a lookup if you already know the content?

By: Rosyna

Rosyna — Tue, 19 Feb 2008 15:58:25 +0000

Pfft, would have been more amusing if you chose two quotes that have the same hash.

By: jcburns

jcburns — Tue, 19 Feb 2008 14:10:47 +0000

…that hashing isn’t ‘decimal’ by definition, if it’s binary? Not very ‘dewey,’ either.