All of the-citizen's Comments + Replies

True! I was actually trying to be funny in (4), tho apparently I need more work.

mistakes that are too often made by those with a philosophical background rather than the empirical sciences: the reasoning by analogy instead of the building and analyzing of predictive models

While there are quite a few exceptions, most actual philosophy is not done through metaphors and analogies. Some people may attempt to explain philosophy that way, while others with a casual interest in philosophy might not known the difference, but few actual philosophers I've met are silly enough not to know an analogy is an analogy. Philosophy and empirical sci... (read more)

Suffering and AIs

Disclaimer - For the sake of argument this post will treat utilitarianism as true, although I do not neccesarily think that

One future moral issue is that AIs may be created for the purpose of doing things that are unpleasant for humans to do. Let's say an AI is designed with the ability to have pain, fear, hope and pleasure of some kind. It might be reasonable to expect in such cases the unpleasant tasks might result in some form of suffering. Added to this problem is the fact that a finite lifespan and an approaching termination/shutdown ... (read more)

That seems like an interesting article, though I think it is focused on the issue of free-will and morality which is not my focus.

Suffering and AIs

Disclaimer - Under utilitarianism suffering is an intrinsically bad thing. While I am not a utilitarian, many people are and I will treat it as true for this post because it is the easiest approach for this issue. Also, apologies if others have already discussed this idea, which seems quite possible

One future moral issue is that AIs may be created for the purpose of doing things that are unpleasant for humans to do. Let's say an AI is designed with the ability to have pain, fear, hope and pleasure of some kind. It might be reasonable to ex... (read more)

Yeah I think you're right on that one. Still, I like and share his moral assumption that my-side-ism is harmful because it distorts and is often opposed to the truth in communication.

I retracted an earlier incorrect assertion and then edited to make this one instead. Not sure how that works exactly...

Indeed I do! :-)

Cheers thanks for the informative reply.

Yes some humans seem to have adopted this view where intelligence moves from being a tool and having instrumental value to being instrinsically/terminally valuable. I find often the justifcation for this to be pretty flimsy, though quite a few people seem to have this view. Let's hope a AGI doesn't lol.

Thanks for reply. That makes more sense to me now. I agree with a fair amount of what you say. I think you'd have a sense from our previous discussions why I favour physicalist approaches to the morals of a FAI, rather than idealist or dualist, regardless of whether physicalism is true or false. So I won't go there. I pretty much agree with the rest.

EDIT> Oh just on the deep ecology point, I believe that might be solvable by prioritising species based on genetic similarity to humans. So basically weighting humans highest and other species less so based ... (read more)

Thanks that's informative. Not entirely sure your own position is from your post, but I agree with what I take your implication to be - that a rationally discoverable set of ethics might not be as sensible notion as it sounds. But on the other hand human preference satisfaction seems a really bad goal - many human preferences in the world are awful - take a desire for power over others for example. Otherwise human society wouldn't have wars, torture, abuse etc etc. I haven't read up on CEV in detail, but from what I've seen it suffers from a confusion that... (read more)

The stability under self-modification is a core problem of AGI generally, isn't it? So isn't that an effort to solve AGI, not safety/friendliness (which would be fairly depressing given its stated goals)? Does MIRI have a way to define safety/friendliness that isn't derivative of moral philosophy?

Additionally, many human preferences are almost certainly not moral... surely a key part of the project would be to find some way to separate the two. Preference satisfaction seems like a potentially very unfriendly goal...

For the record, my current position is that if MIRI doesn't think it's central, then it's probably doing it wrong.

But perhaps moral philosophy is important for a FAI? Like for knowing right and wrong so we can teach/build it into the FAI? Understanding right and wrong in some form seems really central to FAI?

What do you feel is bad about moral philosophy? It looks like you dislike it because place it next to anthropormorphic thinking and technophobia.

I'll leave these two half-baked ideas here in case they're somehow useful:

DO UNTIL - Construct an AI to perform its utility function until an undesirable failsafe condition is met. (Somehow) make the utility function not take the failsafe into account when calculating utility (can it be made blind to the failsafe somehow? Force the utility function to exclude their existence? Make lack of knowledge about failsafes part of the utility function?) Failsafes could be every undesirable outcome we can think of, such as human death rate exceeds X, biomass reduct... (read more)

Wouldn't most AGI goals disregard property rights unless it was explicitly built in? And if it was built in, wouldn't an AGI just create a situation (eg. progressive blackmail or deception or something) where we wanted to sell it the universe for a dollar?

The risk of course is the AI predicting that it's nested in this sort of environment and finding a way to signal to observers. Even if it's blind to the other layers it might try it just in case. What you want is to develop a way for the simulated world environment to detect a harmful intellgience explosion and send a single bit binary communication "out of the box" to indicate that it has occurred. Then you can shut it down and keep trying multiple instances until you get a success at this level of safety. I guess you can then slowly expand the ... (read more)

Thanks for comment. I will reply as follows:

• Knowing how things could go wrong gives useful knowledge about scenarios/pathways to avoid
• Our knowledge of how to make things go right is not zero

My intention with the article is to draw attention to some broader non-technical difficulties in implementing FAI. One worrying theme in the reponses I've gotten is a conflation between knowledge of AGI risk and building a FAI. I think they are separate projects, and that success of the second relies on comprehensive prior knowledge of the first. Apparently MIRI's approach doesn't really acknowledge the two as separate.

Thanks for the reply.

If we change the story as you describe I guess the moral of the story would probably become "investigate thoroughly". Obviously Bayesians are never really certain - but deliberate manipulation of one's own map of probabilities is unwise unless there is an overwhelmingly good reason (your hypothetical would probably be one - but I believe in real life we rarely run into that species of situation).

The story itself is not the argument, but an illustration of it - it is "a calculation of the instrumentality of various option... (read more)

Your engagement here is insincere. You argue based on to cherry-picking and distorting my statements. You simply ignore the explanations given and say "you haven't given justification" and then you give off-hand vague answers for my own queries and then state "already answered". I'm done with this.

Not much I can think of that we can do about that, except provide a system with disincentives for harmful behaviour. What can easily correct is the possiblity of meaning well but making mistakes due to self-deception. This post attempts to examine one instance of that.

Cheers, that all seems to make sense. I wonder if the Basilisk with its rather obvious flaws actually provides a rather superb illustration of how memetic hazard works in practice, and so doing so provides a significant opportuntity of improving how we handle it.

Thanks for the feedback.

On top of that, it actually ends in the optimal outcome.

Just to clarify, no it doesn't. It's implied that the 20 deaths is worse than 5 for the consequentialist protagonist.

The analogy is problematic ... bizarre and unrealistic example.

Thanks for the feedback. It seems people have been split on this. Others have also found the analogy problematic. On the other hand an analogy doesn't usually attempt to provide proof, but illustrate the structure of an argument in an understandable way. I don't think it's bizarre if you thin... (read more)

What in particular was wrong with his handling of this incident? I'm not aware of all the details of his handling so its an honest question.

There aren't any inherently, unchangeably, mental concepts.

From what I can observe in your position it seems like you are treating consciousness in exactly this way. For example, could you explain how it could possibly be challenged by evidence? How could it change or be refined if we say "introspection therefore consciousness"?

The very fact of introspection indicates that consciousness, by a fairly minimal definition, is existent.

I don't see how this follows. As there are a whole host of definitions of consciousness, could you explicitl... (read more)

Cheers now that we've narrowed down our differences that's some really constructive feedback. I think I intended it primarily as a illustration and assume that most people in this context would probably already agree with that perspective, though this could be a bad assumption and it probably makes the argument seem pretty sloppy in any case. It'll definitely need refinement, so thanks.

EDIT> My reply attracted downvotes? Odd.

What part do you think was forced? So far quite a several others said they didn't mind that part so much, and that actually the second section bothered them. I'll probably make future alterations when I have spare time.

Jungle? Didn't we live on the savannah?

LOL it was just a turn of phrase.

And forming groups for survival, it seems just as plausible that we formed groups for availability of mates.

Genetically speaking mate-availability is a component to survival. My understanding of the forces that increased group size is that they are more complex than either of these (big groups win conflicts for terrritory, but food availability (via tool use) and travel speed are limiting factors I believe - big groups only work if you can access a lot of food and move on before... (read more)

Thanks for the useful suggestion. This appears to emerging as a consensus. I'll probably either tidy up the second section or cut it when I have time.

LOL. Well I agree with your first three sentences, but I'd also add that we systematically the costs of false beliefs because (1) at the point of deception we cannot reliably predict future instances in which the self-deceptive belief will become a premise in a decision (2) in instances where we make a instrumentally poor decision due to a self-deception, we often receive diminished or no feedback (we are unaware of the dead stones floating down the river).

LW appears to be mixed on the "truthiness should be part of instrumental rationality" issue.

It seems we disagree on the compartmentalising issue. I believe self-deception can't easily be compartmentalised in the way you describe because we can't accurately predict, in most cases, where our self-deception might become a premise in some future piece of reasoning. By its nature, we can't correct at the later date, because we are unaware that our belief is wrong. What's your reasoning regarding compartmentalizing? I'm interested in case I am overlook... (read more)

I guess my argument is that when people can't see an immediate utility for the truth, they can become lazy or rationalise that a self-deception is acceptable. This occurs because truth is seen as useful rather than essential or at least essential in all but the most extreme circumstances. I think this approach is present in the "truth isn't everything" interpretation of instrumental rationality. The systematised winning isn't intended to comprise this kind of interpretation, but I think the words it uses evokes too much that's tied into a problematic engagement with the truth. That's where I currently sit on the topic in any case.

Thanks for the group selection link. Unfortunately I'd have to say, to the best of my non-expert judgement, that the current trends in the field disagrees somewhat with Eliezer in this regard. The 60s group selection was definitely overstated and problematic, but quite a few biologists feel that this resulted in the idea being ruled out entirely in a kind of overreaction to the original mistakes. Even Dawkins, who's traditionally dismissed group selection, acknowledged it may play more of a role than he previously thought. So its been refined and is making... (read more)

Well I've done Map & Territory and have skimmed through random selections of other things. Pretty early days I know! So far I've not run into anything particularly objectionable for me or conflicting with any of the decent philosophy I've read. My main concern is this truth as incidental thing. I just posted on this topic: http://lesswrong.com/lw/l6z/the_truth_and_instrumental_rationality/

Cheers for comment. I think I perhaps should have made the river self-deception less deliberate, to create a greater link between it and the "winning" mentality. I guess I'm suggesting that there is a little inevitable self-deception incurred in the "systematised winning" and general "truth isn't everything" attitudes that I've run into so far in my LW experience. Several people have straight-up told me truth is only incidental in the common LWers approach to instrumental rationality, though I can see there are a range of views.

I think I'd be quite interested to know what % of CRAF people believe that rationality ought to include a component of "truthiness". Anything that could help on that?

I like the exporation of how emotions interact with rationality that seems to be going on over there.

For me over-analysis would be where further analysis is unlikely to yield practically improved knowledge of options to solve the problem at hand. I'd probably treat this as quite separate from bad analysis or the information supplied by instinct and emotion. In a sense then emotions wouldn't come to bear on the question of over-analysis generally. However, I'd heartily agree with the proposition that emotions are a good topic of exploration and study becaus... (read more)

Thanks for the interesting comments. I've not been on LW for wrong and so far I'm being selective about which sequences I'm reading. I'll see how that works out (or will I? lol).

I think my concern on the truthiness part of what you say is that there is an assumption that we can accurately predict the consequences of a non-truth belief decision. I think that's rarely the case. We are rarely given personal corrective evidence though, because its the nature of a self-deception that we're oblivious that we've screwed up. Applying a general rule of truthiness is a far more effective approach imo.

I think you've got a good point regarding having as many virtues as possible.

On the idea of perfection being dystopic, this reminds me of an argument I sometimes hear along the lines of "evil is good because without evil, good would just be normal", which I don't find very convincing. Still I guess a society and its people should always focus on betterment of themselves, and perfection is probably better thought of as a idealised goal than some place we arrive at.

This isn't productive. As you've insisted on a long and highly critical response again, I sadly feel I need to briefly reply.

What you need to disprove is the claim that physicalists employ the concept of consciousness

No. I merely show that the core claims of physicalism and this use of consciousness are incompatible. That's what I've done. Whether some particular physicalists choose to employ some concept is a different question and a massive red herring, as I already said.

One ought to provide evidence for extraordinary claims,

Whereas ordinary clai... (read more)

I don't 100% follow your comment, but I find the content of those links interesting. Care to expand on that thought at all?

I think I'm broadly supportive of your approach. The only problem I can see is that most people think its better to try to do stuff, as opposed to getting better at doing stuff. Rationality is a very generalised and very long-term approach and payoff. Still I'd not reject your approach at this point.

Another issue I find interesting is that several people have commented recently on LW that (instrumental) rationality isn't about knowing the truth but simply achieving goals most effectively. They claim this is the focus of most LWers too. As if "Truthiness" is only a tool that can be even be discarded when neccessary. I find that view curious.

Cheers for the mention. I still haven't worked out if Divergent is meant to be a dystopia or utopia (somewhere in between I think?). Its an interesting world.

Yes that's a fairly good point and I don't know any easy way around it either. Looking in the world of business, government, politics etc etc. would be a matter of fairly subjective ideas about moral goodness.

I suppose you could formulate an approach along the lines of experimental psychology, where you could deliberately design experiments with clearcut good/bad group outcomes. So get a bunch of people to be leaders in an experiment where their goal was to minimise their group members (including themselves) getting hit in the head with something unpleasan... (read more)

:-( I'm disappointed at this outcome. I think you're mentally avoiding the core issue here - but I guess my saying so is not going to achieve much. I'll answer some of your points quickly and make this my last post in this subthread.

What you need is evidence that monists don't or can't or shouldn't shouldn't believe in consciousness or subjectivity or introspection. Evidence that dualists do is not equivalent.

You're twisting my claim. Someone can't disprove a pure concept or schema - asking them to do so is a red herring. Instead one ought to prove ra... (read more)

Ok thanks for this comment.

studying philosophy for 35 years

Stealthy appeal to authority, but ok. I can see you're a good philosopher, I wouldn't seek to question your credibility as a philosopher, but I do wish to call into question this particular position, and I hope you'll come with on this :-)

Who told you that introspection implies separation of mind and body?

I wrote on this topic at uni, but you'll have to forgive me if I haven't got proper sources handy...

"The sharp distinction between subject and object corresponds to the distinction, i... (read more)

I am suprised that a significant group of people think that rationality is inclusive of useful false beliefs. Wouldn't we call LW an effectiveness forum, rather than a rationalist forum in that case?

That basically means that sometimes the person who seeks the truth doesn't win.

I think you're reading too much into that one quite rhetorical article, but I acknowledge he prioritises "winning" quite highly. I think he ought to revise that view. Trying to win with false beliefs risks not achieving your goals, but being oblivious to that fact. Lik... (read more)

Agreed, but I think they'd have some correlation, and I strongly suspect their absense would predict corruptability.