RichardKennaway comments on No One Can Exempt You From Rationality's Laws - Less Wrong

51 Post author: Eliezer_Yudkowsky 07 October 2007 05:24PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (51)

Sort By: Old

You are viewing a single comment's thread. Show more comments above.

Comment author: David_Gerard 19 January 2011 03:28:53PM 2 points [-]

Somewhat off-topic and answering an ancient comment, but a useful reminder of how important endeavours can actually be horribly short of resources and much more fragile than people think:

Rather than elections, Wikipedia would better serve its public IMO by scrapping elections and making it as easy as possible for groups to fork Wikipedia. Putting the content under a permissive, open-source license was a major step in that direction. The 2 major remaining steps IMO are a technical provision by which every competing encyclopedia's software may be notified of every change to every Wikipedia page as soon as the change is saved and the development of search engines able to lead the surfer through the bewildering array of world views and editorial approaches nurtured by the governance structure I just described.

We know this :-) Wikipedia as monopoly provider of the world's encyclopedia is an anti-pattern. But the network effects are very powerful.

This means we are a great big single point of failure. Our single data centre is one hurricane away from disappearing. Even making a good backup of English Wikipedia is a remarkably difficult endeavour because it's SO BIG. A billion and a half words. Can you mentally grasp how big that is? I sure can't.

And the distributed network you outline would be a wonderful thing. But, like most things that it would be nice to do with Wikipedia, it requires coding on MediaWiki. Lots of people have "Why don't you ..." technical ideas - nearly none of them follow them with the requisite code.

The budget for this year includes a pile of cash on technical resources: a second data centre and a lot more coders. We're also developing a pattern where young whizzkids work for WMF for a couple of years at charity pay and go off to make a bundle in industry - and that's fine by us.

Comment author: RichardKennaway 19 January 2011 05:31:40PM 3 points [-]

A billion and a half words. Can you mentally grasp how big that is?

1.5N GB, where N is the average bytes per English word. Multiply by, say, 5 for the HTML overhead and it would still all fit onto a 64GB memory stick uncompressed, though I'd want something faster for actually accessing it.

It would actually be larger, as you'd need all the images as well, and you'd want the ancillary things like wikisource and wiktionary (I don't know if those are independent projects or if they're included in your figure) but even so, it sounds like the whole thing would easily fit onto a typical hard disc.

Comment author: kybernetikos 19 January 2011 05:39:47PM *  1 point [-]

I have all of the english wikipedia available for offline searching on my phone. It's big, sure, but it doesn't fill the memory card by any means (and this is just the default one that came with the phone).

For offline access on a windows computer, WikiTaxi is a reasonable solution.

I'd recommend that everyone who can carry around offline versions of wikipedia. I consider it part of my disaster preparedness, not to mention the fun of learning new things by hitting the 'random article' button.