= 27d567bc2d48d9f40e9ccc20ea370b15)
r/HPMOR on heroic responsibility
r/HPMOR readers on heroic responsibility - not the OP, the comments. Holy snorkels this is good.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
r/HPMOR readers on heroic responsibility - not the OP, the comments. Holy snorkels this is good.
Comments (49)
It seems to me that if many people adopt heroic responsibility to their own values, then a handful of people with destructive values might screw up everyone else, because destroying is easier than helping people.
Well yes, but what exactly are you going to substitute for individual judgement of what actions to take? If you decide that following code X, and enforcing that others do so as well, even when it seems like breaking the code this one time would give better results, is the best course of action (perhaps for reasons of precommitment or average-case utility or what-have-you)... then oops, you seem to have made a decision in accordance with your best judgement, there.
That's the Übermensch morality of dictators and totalitarian regimes. The problem is that every dictator thinks of themselves as a benevolent dictator, but it turns out that they are often mistaken.
Even when there are multiple aspiring dictators, all with essentially benevolent values, conflict on who should rule can be very much destructive.
Of course this presupposes consequentialism. In deontologism morality is typically viewed as a social contract where each party has well defined responsabilities.
I don't think it's an accident that violent totalitarian ideologies tend to be consequentialist: these innocent heretics/Jews/bourgeois stand between us and our utopia. Kill'em all!
citations please! I doubt that most dictators think they are benevolent and are consequentialists.
I think that most dictators who make it into history books think about benevolence differently than most people.
Thankyou! I get tired of the whole "everybody thinks they are good" nonsense I hear all the time. I call it mind-projection. Some people just don't care.
There is an obvious parallel between HJPEV and AGI: he can do (and does) stuff no (other) human can even conceive of doing.
How do you know if your values and goals are constructive or destructive? It all comes down to the same hard question of FAI (well, FHHI, friendly human hero intelligence, in this case): are your values CEV-aligned? So, the first thing Harry should do is stop running around saving people and derive the CEV :)
One can, of course, argue that, as a human hero, HJPEV has the right CEV built-in and should just run around implementing it. However, given that even his friends and allies disagree (Dumbledore is a deathist, McGonagall is a disciplinarian, Hermione thinks they are too young, Weasley twins just want to have fun), and he gets the most help from the anti-hero Quirrell, this point of view is hard to defend.
The situation is much worse outside of fiction, where buggy or limited wetware constantly leads would-be heroes (Lenin, Castro, Lincoln or even Hitler) to commit or cause suffering or destruction of the same group of people they aspired to help.
So the first question a would-be hero should ask herself is whether she is prepared to live with the consequences of her actions if they backfire. (And if the answer is yes, she is clearly a villain.)
CEV is a construct for AI purposes that actual human beings can't eval - I don't think I've ever seen a human discussion that was helped by invoking it. It's not like Solomonoff Induction where sometimes you really can be helped by thinking formally about Occam's Razor. In practice, human beings arguing about ethics are either already approximating their part of the 'good' as best they can, or they're confused about something much simpler than CEV, like consequentialism. If you should never use the word 'truth' when you can just talk about the object level, and never say 'because it's not optimal!' when that just means 'I don't think you should do that', then there's basically never a good time to talk about CEV - it always deflates out of the sentence unless you're talking directly about FAI.
I suppose my point is that, if you adopt "heroic responsibility", you ought to put in the correspondingly heroic amount of effort into figuring out what a hero ought to do. And given that your Harry plans to take over the world and then radically change it, he ought to do an awful lot of figuring out first. Probably of the same order of magnitude an FAI would.
That's curious, because Solomonoff Induction is something not even an enormously powerful (but computable) AI can evaluate.
Yes, but my point is that thinking about SI or MML in the abstract helps because people sometimes gain insight from asking "How complex is that computer program?" I haven't seen appeal-to-CEV produce much insight in practice, and any insight it could produce can probably be better produced by appealing to the relevant component principle of CEV instead. (Nor yet is this a critique of CEV, because it's meant as an AI design, not as a moral intuition pump.)
Can you provde an example where Solomonoff Induction can be used to gain insight that Occam's razor doesn't help to gain?
How else can you impartially wield Occam's Razor than with a formal model, and what convincing formalization is there other than Kolmogorov Complexity (and assorted variants), which SI in a way extends?
Setting aside the theoretical objections to Solomonoff induction (a priori assumption of computability of the hypotheses, disregard of logical depth, dependance on the details of the computational model, normalization issues), even if you accept it as a proper formalization of Occam's Razor, in order to apply it in a formal argument, you would have to perform an uncomputable calculation.
Since you can't do that, what's left of it?
Besides noting that there are computable versions of Kolmogorov Complexity (such as MML), in your parent comment you contrasted the use of SI with using Occam's Razor itself.
That's what I was asking about, and it doesn't seem like you answered it:
How do you use Occam's Razor, what formalizations do you perceive as "proper", or if you're just intuiting the heuristic, guesstimating the complexity, what is the formal principle that your intuition derives from / approximates and how does it differ from e.g. Kolmogorov Complexity?
If by MML you mean Minimum message length, then I don't think that's correct. This paper compares Minimum message length with Kolmogorov Complexity but it doesn't seem to make that claim.
My point is that Kolmogorov complexity, Solomonoff induction, etc., are matematical constructions with a formal semantics. Talking about "informal" Kolmogorov complexity is pseudo-mathematics, which is usually an attempt to make your arguments sound more compelling than they are by dressing them in mathematical language.
If there is a disagreement about which hypothesis is simpler, trying to introduce concepts such as ill-defined program lengths that can't be computed, can only obscure the terms of the debate, rather than clarifying them.
Willam of Ockham originally used his principle to argue for the existence of God (God is the only necessary entity, therefore the simplest explanation).
That's a truly epic fail, since Occam's razor is the strongest argument against the existence of God.
It's worth noting that the current formulation "entities must not be multiplied beyond necessity" is much more recent than Ockham's original formulation "For nothing ought to be posited without a reason given, unless it is self-evident (literally, known through itself) or known by experience or proved by the authority of Sacred Scripture."
I suppose that he included the reference to the Sacred Scripture specifically because he realized that without it, God would be the first thing to fly out of the window.
I sometimes wish I knew which philosophers of the time were sincere in their religious disclaimers.
Consider it done.
It's exactly like Solomonoff Induction where most of the time you really can't be helped by thinking formally about Occam's Razor. In practice, human beings arguing about probabilities are either already approximating their part of the 'simple' as best they can, or they're confused about something much simpler (haha) than Solomonoff Induction, like Bayesianism.
Nitpicking note : I don't think the Weasley twins just want to have fun. They are in Griffyndor, in the Order of the Phoenix, they fight the Death Eaters to the end, ... they want to have fun, but they also want others to have fun.
In the canon, sure. Their HPMOR characters are not nearly as nuanced.
Well, it's not as if they've been given nearly as much opportunity to characterize themselves by anything else.
Oh, I did not mean this in a negative way.
A lot of the comments take a very consequentialist point of view, and they explain themselves fairly well, which is good.
Perhaps it is because I've seen many really bad reddit comments before (even in subreddits relating to fields usually sympathetic to rationalist ideals) and what I'm seeing here is of a different standard, but I find myself hoping to some extent that some of the people commenting here were idiots before reading HPMOR, and that somehow they became more insightful and eloquent as a result of being exposed to the fic and related content.
I think I might recommend the fic to some more people...
I think many of them are secretly (not secretly) LW readers :P
Thinking back, I'm almost certain that the way I was introduced to Less Wrong was through HPMOR. The long gaps between updates encouraged me to start reading the sequences, even.
It's probably the most effective recruiting tool for rationality in the world as of now.
At least for myself , I first heard of Eliezer via the HPMOR TV Tropes page. There's a good chance I would have read the sequences sooner or later even if I hadn't (my brother found them independently and recommended them), but it definitely helped.
And I wouldn't say I was an idiot before, but twenty minutes of conversation with myself from a couple years ago might change my mind. And of course it's hard to tell how much of the difference is LW's influence and how much is just a matter of being older and wiser.
I don't know that Harry or Eliezer make this distinction, but I'd say that it's important to recognize a difference between judgments about yourself, and judgments about others.
When Harry talks about heroic responsibility, he's speaking from the first person perspective (at least). He has his values, and he wants them achieved regardless of the good/bad behavior of others.
The usual responsibility we talk about is second/third person - your fault/his fault. It's about negotiation with and judgment of others. This is where someone judges someone as "fulfilling a role" - it's a thumbs down or a thumbs up on another person, according to the rules of interpersonal behavior you approve of.
We can treat ourselves as objects, and apply that thumbs up or thumbs down to ourselves as well. That is useful in understanding the reactions of other people to you, in negotiating and otherwise interacting with others, and asserting your values in terms of that negotiating and interacting, but it's a category error to confuse that interpersonal thumbs up/thumbs down applied to yourself as the proper way to choose your actions. I'm pretty far down the path of just considering it a neurosis.
Choose your actions to achieve your values, don't choose them to achieve good boy status.
However, it seems to me that what Harry has described as heroic responsibility is a conflation of the 1st and 3rd person perspective. He still judges his own good boy status in the 3rd person, but he does it from the perspective of behaving properly from a 1st person perspective. He is a good boy if he chooses the best actions to achieve his values, and a bad boy besides. Actually, he seems a little worse than this, in that any failure makes him a bad boy, which is an impossible standard.
(EDIT: jimrandomh makes a similar point)
He is maximizing his utility part-function: people should not be harmed. If I know of a situation such that: Someone will be harmed if I do nothing; They will be harmed equally if I tell the authorities; If I intercede with my allies, they will be harmed less.
The first two actions are equal. The third one is better. The ability to determine the actual consequences of future actions is magical. Fortunately for Harry, he has access to magic. I expect Harry to develop a procedure where he makes several plans, checks for a note indicating which plan he should use and any changes he should make, executes the plan, and then time-turns a note back to himself indicating which plan worked and any retroactive changes to it, then assists himself if needed.
Something tells me that the note would be more likely to say something like "DO NOT MESS WITH TIME".
Really? Is that what happened just before he got the time-turner?
As I recall, he was trying to demonstrate a way to solve any problem in C time, where C is the time required to falsify a proposed solution. That's different than realizing that you have a higher chance of destroying Azkaban in an hour if you help yourself eight times.
It sounds like you're assuming that all Morality-speak is a ploy to get others to follow our will, and a way of taking stock of others' comparable attempts to influence us. I'd like to suggest that there are other ways to construe Morality-speak like 'hero' and 'virtue:'
One could simply treat one's values and Morality-speak as interchangeable. Why use one language for the second- and third-person, and a totally different language for the first-person? It's confusing, makes generalizations of your desires difficult, and greatly weakens your rhetorical power (since you seem to be subscribing to a double standard, if only terminologically). If your values are such that you don't actually give a privileged status to your own welfare as opposed to others, then this linguistic shift conceals an important symmetry.
Morality-speak might be an idealization of one's optimized values, one's values once they've been brought into optimal reflective equilibrium. What you currently care about might not be what you think you ought care about, even if you ultimately define 'oughtness' in terms of the aforementioned preferences. Otherwise it would be incoherent to lament how much one presently likes something, to wish for a reform to one's secondary concerns that would bring them into greater internal harmony.
Two issues: First, he's probably exaggerating at least a little. Second, it's clear that he's adopting the standard in question for utilitarian reasons; he happens to be especially motivated by falling short of some lofty ideal. For most people an at least somewhat less exacting standard would probably be desirable; but since standards are just heuristics for winning, the question of how demanding to be is one for empirical psychology.
Ironic. I consider anything else ethical solipsism. Why would it matter when it's your fault, but not when it's someone else's?
Also, I don't think Harry made an argument. He just defined heroic responsibility.
I would say that he was making at least the argument that "this level of responsibility is something you should adopt if you want to be a hero", and probably the more general argument that "people should adopt this attitude toward responsibility.
To what extent does it contains spoilers? I haven't read HP:MOR yet.
You won't gain knowledge about a twist in the plot by reading the article; the comments contain descriptions of events in the story, but I didn't see anything that I would call a spoiler.
Decius is right that there aren't really spoilers, but I would argue that your time would be better spent reading HP:MOR than the discussion.