thomblake comments on The mathematics of reduced impact: help needed - Less Wrong

10 Post author: Stuart_Armstrong 16 February 2012 02:23PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (94)
Tags: ai mathematics risk reduced_impact safety prediction algorithm

Comments (94)

You are viewing a single comment's thread. Show more comments above.

Comment author: Vaniver 16 February 2012 06:35:46PM *  4 points [-]

The most obvious path to coding reduced impact is to build a satisficer rather than a maximiser - but that proved unlikely to work.

As I commented there: I don't think you're using a useful definition for "satisficer," and I'm troubled by your use of the word "proved."

If I build a Clippy whose utility function is Num_Paperclips - Negentropy_Cost, then I expect it to increase the number of paperclips until the marginal benefit is lower than the marginal cost, and if I do F(Num_Paperclips)-G(Negentropy_Cost), where F is concave and G is convex, then it's even less likely to go foom because marginal benefit is penalized and marginal cost is overcounted. Is there a good reason to expect this won't work?

(Will comment on the rest of the article later.)

Comment author: thomblake 16 February 2012 07:12:16PM 4 points [-]

You probably want to do something to escape those underscores.

I'm troubled by your use of the word "proved."

That's a standard sense of the word 'proved', which is usually identifiable by its lack of a direct object. It just means that something turned out that way, or the evidence points that way.

Comment author: Vaniver 16 February 2012 07:25:22PM *  1 point [-]

Thanks, I noticed and fixed that.

It just means that something turned out that way, or the evidence points that way.

My complaint is twofold: first, I don't think the evidence points that way, and second, I would prefer them saying the evidence pointed that way to them using a stronger phrase.

Comment author: thomblake 16 February 2012 08:13:09PM 3 points [-]

I would prefer them saying the evidence pointed that way to them using a stronger phrase.

But that's not what that means - it's not very strong. If I say, "My search proved fruitful", then I'm not saying anything particularly strong - just that I found something. Saying "that proved unlikely to work" just means "based on <whatever work I've done>, I've observed that it's unlikely to work". <whatever work I've done> can be a search, some research, an experiment, or anything of that sort.

Note that this sense of "proved" does not even need to imply a particular conclusion - "The experiment proved inconclusive".

This is more similar to the use of "proof" in baking or alcohol than the use of "proof" in geometry or logic.

Powered by Reddit Powered by Reddit