All of ema's Comments + Replies

ema130

If you get one bitter cucumber, asking for its cause may be a waste of time. But if you get a lot of bitter cucumbers, spending some time on changing that might give net positive utility.

ema60

The subset of people who are Anki users and members of the competitive conspiracy might be interested in the Anki high score list addon I wrote: Ankichallenge

ema100

According to their site Jaan Tallinn is not the CEO but chairman of the board. Zvi Mowshowitz is the CEO.

7thescoundrel
Wow- that is former MTG Pro Zvi, one of the best innovators in the game during his time. Awesome to see him involved in something like this.
2Mycroft65536
Jaan is also the CTO, I'm not sure if that's on the website.
ema00

I go without shaving my legs, I don't mind wearing stained clothing, I'll happily sit on wet grass

There are Communities where this is high status behavior. But i presume you would have considered this if you were to belong to such a community.

ema00

Paul Graham's writings on programming and learning Haskell leveled me up. Although i suspect you are already at that level.

ema00

Simulated paperclips.

Now we get to the question how detailed the paperclips have to be for the paperclipper to care. I expect the paperclipper to only care when the paperclips are simulated individually and we can't simulate 3^^^^^^3 paperclips individually.

I see no reason to think any work of fiction can lead to such a distortion of reality.

I see no reason to think works of fiction that lead to such a distortion of reality are impossible.

-2MugaSofer
I assume you concede my point re:guards? We built the paperclipper. Hell, it doesn't even have to be a literal paperclipper. For that matter, it doesn't have to be literally 3^^^^^^3 paperclips; all that matters is that we can manipulate it's utility function without too much strain on our resources. If it values cheesecake, we can reward it with a world of infinite cheesecake. If it values "complexity" we can put it in a fractal. The point is that we can motivate it to co-operate without worrying that it might be a better idea to let it out.
ema10

Which is a good thing, because we really do have such powers and we really don't value paperclips.

Our universe has not enough atoms or energy to destroy 3^^^^^3 paperclips.

... were you seriously that confused or are you extrapolating to a "supercharged" novel?

I am extrapolating.

I somehow doubt there would be a single, full-time guard.

Groups of people are not that much harder to manipulate than single persons.

-2MugaSofer
Simulated paperclips. Because the Paperclipper is in a box. As I thought. I disagree that such effects are possible in reasonable time-frames (no-one is going to read constantly for a month) and may be totally impossible. I see no reason to think any work of fiction can lead to such a distortion of reality. They are if you're trying to weaponize novels. If they work shifts then you cannot exploit the effects you claim for reading a novel for 24 hrs straight. They can watch each other and, if one of them is visibly compromised, prevent them from freeing the AI. A single individual is probably easier to manipulate, assuming a total lack of supervision and safeguards.
ema10

Because we have magical powers from outside the matrix [...].

The AI is vastly more smarter than we and can communicate with us. So it asks us questions which sound innocent to us, but from the answers it can derive a fairly accurate map of how it looks outside the matrix.

It would have to argue that destroying humanity and replacing it with paperclips was a good thing.

The goal of the AI is to have the guard execute the code that would let the AI access the outside world. Arguing with us could be one way to archive this goal. Although i agree it sound... (read more)

0MugaSofer
Which is a good thing, because we really do have such powers and we really don't value paperclips. ... were you seriously that confused or are you extrapolating to a "supercharged" novel? I somehow doubt there would be a single, full-time guard.
ema50

Why would it believe us that we are able to destroy 3^^^^^3 paperclips?

"arguing" is to narrow a word for describing the possibilities the AI has. For example it could manipulate us emotionally. It could write us a novel that leaves us in a very irrational state and then give us a bogus, but effective on us, argument for why we should let it out.

I once read the fifth Harry Potter book nonstop for 24 hours and for a couple of hours afterwards i had difficulties distinguishing between me and Harry Potter. It seems likely that a author who is a millions times smarter than Rowling and who has it as explicit goal, could write a novel that leaves me with far bigger misconceptions.

0MugaSofer
Because we have magical powers from outside the matrix and don't value paperclips. It would have to argue that destroying humanity and replacing it with paperclips was a good thing. Not impossible, sure, but easier to guard against. That sounds like more a side effect of reading the same thing "nonstop for 24 hours" than a property of the book, unless you know of anyone else that happened to?
4wedrifid
Just make sure you carry a mirror and constantly check for the scar. Kind of like lucid dreaming practice...
ema70

Could it be that you are confusing the complexity of a utility function of an agent with its optimization power? A super intelligent paperclipper has a simple utility function, but would have no problem reasoning about humans in great enough detail to find out what it has to say to get the guard to let it out of the box.

0MugaSofer
No, I mean it can hardly argue it would be in our best interest to let it out of the box, can it? And we can always threaten to destroy 3^^^^^3 paperclips if it wont cooperate, which is handy.
ema50

Maybe that is more to your liking -> https://dl.dropbox.com/u/3943312/gwern-small.png I just cropped and rescaled it in gimp.

0gwern
Took me a while to find a favicon generating site which didn't mangle that PNG horribly, but the result seems nice. I don't understand why the solution was so simple, but I'm happy anyway!
ema40

people with a photographic memory still could use SRS for learning sounds.

ema60

But what should Spaniards answer?

i think "White (non-Hispanic)". Not that i understand the category Hispanic, but putting Swedish and Greek people in one category while excluding Spaniards seems deeply weird to me.

ema30

I like that idea, but i think there can be too much granularity. The feeling of 'People who agree with me on X also agree with me on completely unrelated Y' is awesome.

2twanvl
I smell a recommender system. Think of what sites like amazon.com do with "people who like X also liked Y". This is just an observation. I'm not saying that we should go out and build a system to match these people and these Xs and Ys.
4common_law
The halo effect may be awesome ... but it's deadly!
ema10

That doesn't really prevent trolling, so i'm not sure that it would be helpful.

2siodine
It won't prevent trolling but it will minimize its effects. As it stands, you can input numbers like 1e+19 which will seriously throw off the mean. If trolls can only give the highest or lowest reasonable bound then they're not going to have much of an effect individually and that makes going through the effort to troll less worthwhile.
ema00

I would put it lower than 9 because a general AI is science as software. Which means it is already contained in 9.

ema10

I will come too.

ema00

One benefit of running on a lower speed is that you can interact with things farther away from you while it still seems instantaneous. although i have no idea why that would be more important for the workers than for the boss.

ema00

Now it works for me too.

1lukeprog
Works for me...
ema20

I want a UI that suits me better. Concretely this means: More keyboard shortcuts. Dragging the mouse only changes the selection. In Inkscape it also moves paths which can get annoying. Non destructive boolshe operations, makes shading way easier.

ema70

I develop a vector drawing program. It seams to have a good balance between archivability and ambition for me. So far it has 80% of the functionality i use of Inkscape. Currently i'm struggling with getting the performance from barely usable to smooth.

7David_Gerard
So what's the motivation? What's it for that Inkscape doesn't do?
ema00

I will also attend.

ema20

Of course we want to favor the group we are part of. Otherwise our CEV wouldn't differ.

0hankx7787
right. this raises the point that the programmer of the FAI would rationally not write it with CEV. He would use his own individual extrapolated volition. Whether that happened to be coherent with the extrapolated volitions of the rest of humanity is irrelevant (to him) - because obviously he would rationally favor himself or his group in any instance of incoherence.
ema-10

they don't exclude each other.

ema00

I can't see how this is a rationality quote. This would imply that humans have a hard time controlling their actions. How else could someone who thinks wisely act in an absurd fashion? Isn't rationality about how to overcome that humans don't think wisely?

7RobinZ
I read the quote as remarking on the problem of implementation - people often can enunciate the optimal course of action for themselves in their present situation (e.g. I should be working on my paper right now) without this enunciation having the slightest effect on their behavior. Therefore, since the benefits of rationality only accrue to those whose behavior is rational, no art of rationality is complete that does not deal with implementation.
ema60

I would have expected more details on the faces, considering how much processing power is assigned to the task.

The results are obtained by mixing 100 other movies together so it is not surprising that there are no details.

4saturn
Presumably that's also why some of the clips seem to have jumbled words and letters floating around.