Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Comment author: Luke_A_Somers 20 July 2012 01:45:33PM 9 points [-]

The bible.

Comment author: nerzhin 20 July 2012 08:03:18PM 5 points [-]

Have you tried this? Does it work?

Comment author: nerzhin 16 February 2012 08:19:45PM 6 points [-]

I don't understand much of this, and I want to, so let me start by asking basic questions in a much simpler setting.

We are playing Conway's game of life with some given initial state. An disciple AI is given a 5 by 5 region of the board and allowed to manipulate its entries arbitrarily - information leaves that region according to the usual rules for the game.

The master AI decides on some algorithm for the disciple AI to execute. Then it runs the simulation with and without the disciple AI. The results can be compared directly - by, for example, counting the number of squares where the two futures differ. This can be a measure of the "impact" of the AI.

What complexities am I missing? Is it mainly that Conway's game of life is deterministic and we are designing an AI for a stochastic world?

Comment author: TheOtherDave 07 December 2011 09:59:50PM 9 points [-]

So... suppose hypothetically that it were analogously true of humans that our likelihood of voluntarily maximizing "Fun" is dependent on being in an environment in which our access to Fun appears primarily determined by chance and our own efforts, and in which we believe Fun may soon run out.

It seems to follow from that supposition that if an outside force (e.g., a superhuman FAI) wants to maximize the amount of Fun we have, while still respecting our agency, it has to create such an environment.

How does adding that hypothetical constraint affect the conclusions of the Fun Theory Sequence?
Or does it?

How does such an environment differ from the world we actually live in?
Or does it?

Not rhetorical questions.

Comment author: nerzhin 08 December 2011 06:27:35PM 1 point [-]

The first requirement:

Even as chicks, geese cannot be handled by a human, or encounter other geese who have been.

suggests that a FAI would not tell us that it exists. In other words, the singularity may already have happened.

Comment author: [deleted] 02 December 2011 04:59:17PM *  13 points [-]

SEO is clearly the most valuable work. Unfortunately, it's something "so mundane", that anybody could do it.

I actually think you have it backwards there. The reason people aren't engaging in this activity is because it is the opposite of mundane. It is confusing, difficult, and requires previous skills.

General Evidence: There are lots of postings for Search Engine Optimizers, and they all want applicants to already have experience doing SEO. If it was something that was so mundane that anyone could do it with a couple hours of training, what you'd see instead are "no experience necessary" job postings for SEO where the company is willing to take an hour or two to train a schlub that they can then pay minimum wage too.

(Speaking of minimum wage, if you guys are spending a significant amount of your time doing menial tasks like moving furniture, it might be time to get a schlub of your own. You can pay someone $8/hr to do menial tasks 20 hrs/ week, for a total of about $8000 / year.)

Personal Anecdotal Supporting Evidence: I clicked on your link, and the thought in my head wasn't "oh, this is too mundane", but rather was "wtf?? This looks super-complex and confusing. It must be the type of thing that "computer people" know how to do. Not something for me. I don't have the knowledge or skill-set"

In response to comment by [deleted] on Hack Away at the Edges
Comment author: nerzhin 02 December 2011 08:16:24PM 6 points [-]

You can pay someone $8/hr to do menial tasks 20 hrs/ week, for a total of about $8000 / year.

With payroll taxes and insurance, I would expect this to cost at least $12000 a year.

Comment author: JoshuaZ 07 September 2011 12:54:04AM 3 points [-]

If they are allowed to know the history of the bot they are playing against (that is, what it has done to them) then they have to know how many rounds they've played.

Comment author: nerzhin 07 September 2011 03:09:25PM 2 points [-]

Sure, but you could have a limit on how many rounds back they remember, or you could fill in the history with some rule.

Comment author: DavidPlumpton 06 September 2011 12:51:10AM 3 points [-]

Having a known number of rounds seems like a problem to me. Everybody wants to defect on the last round. It might be interesting to retry with a number of rounds chosen randomly from 100 to 110 or where after 100 each round has a 25% chance of being the final round. However a large number of matches might be needed to offset the advantages of playing in some matches with a higher than average number of rounds.

Comment author: nerzhin 07 September 2011 12:40:58AM 1 point [-]

Or just prohibit the bots from knowing which round they are playing.

Comment author: [deleted] 04 September 2011 10:22:20AM 8 points [-]

We don't optimize for well-being, we optimize for what we (think we) want, which are two very different things.

Natural selection does not cease operation. Say, for example, that someone invents a box that fully reproduces in every respect the subjective experience of eating and of having eaten by directly stimulating the brain. Dieters would love this device. Here's a device that implements in extreme form the very danger that you fear. In this case, the specific danger is that you will stop eating and die.

So the question is, will the device wipe out the human race? Almost certainly it will not wipe out the entire human race, simply because there are enough people around who would nevertheless choose to eat despite the availability of the device, possibly because they make a conscious decision to do so. These people will be the survivors, and they will reproduce, and their children will have both their values (transmitted culturally) and their genes, and so will probably be particularly resistant to the device.

That's an extreme case. In the actual case, there are doubtless many people who are not adapting well to technological change. They will tend to die out disproportionately, will tend to reproduce disproportionately less.

We have a model of this future in today's addictive drugs. Some people are more resistant to the lure of addictive drugs than others. Some people's lives are destroyed as they pursue the unnatural bliss of drugs, but many people manage to avoid their fate.

Many people have so far managed the trick of pursuing super stimuli without destroying their lives in the process.

In response to comment by [deleted] on Open Thread: September 2011
Comment author: nerzhin 05 September 2011 04:18:55PM 3 points [-]

It is not at all clear that the people resistant to addictive drugs are reproducing at a higher rate than those who aren't.

Comment author: lessdazed 02 September 2011 09:15:59PM 2 points [-]


Try out a hidden-name system in two weeks, weeks away from the regular quotes threads, as a trial run.

As an experimental run, begin a quotes thread with the following title and text:


Rationality Quotes Thread With Hidden Attributions and Voting


This thread has an experimental format for posting rationality quotes. Here is the format:

For those posting quotes:

Post the quote, but not the author, original language translated from, or other information.

As a reply to your comment, reply with the author and any other relevant information. This will be downvoted to -3 until it is hidden to users using the default setting of -3 downvotes. As another reply to your comment, comment with the single word "dynamo" and this comment will be voted up to +3, to offset downvotes from the other comment.

As a reply to the author's name, comment with the word "evaluate".

For those judging quotes:

Do not vote on the quote itself. Express your approval or disapproval by voting on the "evaluate" tab hidden within the tab containing the author's information and context. Author's names are downvoted to hide both them and the votes received by each quote. Ideally, every individual's first impression and interpretation of a quote will be unaffected by its author or knowledge of how others on LW have responded to it.

For this experimental thread only:

Please post quotes that appeared in an LW quote thread no more recently than one year ago. Feel free to choose quotes that would be interesting out of context.

Comment author: nerzhin 03 September 2011 05:15:47PM 4 points [-]

I think the sub-proposal is too complex and involves too many trivial inconveniences. I up-voted the original proposal.

Comment author: fiddlemath 31 August 2011 04:21:49AM 10 points [-]

Publishing in certain journals (and conferences) is high-reputation, because everyone in some field or subfield believes they are. And thus, they all try to submit their papers to that journal, and thus that journal gets to pick the best papers, and thus that journal keeps a high reputation. Publishing work elsewhere is considered a lesser achievement, and work elsewhere is read with less regard. So, the reputation of high-status journals is notably stable.

On the other hand, almost every researcher in CS flaunts copyright, posting their papers on their own websites. The practice is so pervasive that when I want to find a specific paper, I usually search for the author's website first, before I try accessing the journal -- even thouggh I have the expensive university access to those journals. (Well, in CS they're usually conference proceedings instead of journals, but that's beside the point.)

In fact, author home pages are a vastly better mechanism for finding research than journals or conference proceedings are, because they can carry far more information. Authors will often have more up-to-date versions of their research than the journal version does. Sometimes they have code or interactive demos. Often they put their later, related work in the same place, which is oftern clearer or more useful.

Other fields could make this shift too, subfield by subfield and journal audience by journal audience. For this to happen cleanly, a significant fraction of the top researchers in a field need to get together and agree to break copyright in the same way. In any given subfield, they probably already go to the same conferences, so there's plenty of opportunity to reach these agreements. One or two doing this alone would be prone to lawsuits from the publisher, but all at once would destroy their audience.

This may be a daunting risk for many academics, though, and it probably seems needlessly risky. And it is needlessly risky. It seems like it should be much easier to get that same roomful of academics to agree to move their shared regard from a closed-access journal to an open-access journal. Moreso, because there are already people building free journal-management software for exactly this purpose, this should be relatively easy to do.

Moving individual fields and subfields to totally-free, totally-open journals is, now, just a matter of memtic engineering. Convince enough people in any given field to switch, and make it easy for them to do so, and organize a little bit, and that switch can happen.

Academics, consider: what's the global value of opening the journals of your specialty? Are you, or anyone you can speak to, in a position to do this?

Everyone else: is anyone good at memetic engineering, and willing to take this as a challenge?

Comment author: nerzhin 31 August 2011 05:55:58PM 4 points [-]

almost every researcher in CS flaunts copyright, posting their papers on their own websites

Many journals explicitly allow you to distribute a "preprint" of your journal articles on your personal website. For example, the Elsevier policy states that authors retain:

the right to post a pre-print version of the journal article on Internet websites including electronic pre-print servers, and to retain indefinitely such version on such servers or sites for scholarly purposes

Comment author: nerzhin 30 August 2011 04:17:07PM 8 points [-]

There are open access journals. I recommend supporting them.

View more: Next