= 27d567bc2d48d9f40e9ccc20ea370b15)
shminux comments on r/HPMOR on heroic responsibility - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (49)
There is an obvious parallel between HJPEV and AGI: he can do (and does) stuff no (other) human can even conceive of doing.
How do you know if your values and goals are constructive or destructive? It all comes down to the same hard question of FAI (well, FHHI, friendly human hero intelligence, in this case): are your values CEV-aligned? So, the first thing Harry should do is stop running around saving people and derive the CEV :)
One can, of course, argue that, as a human hero, HJPEV has the right CEV built-in and should just run around implementing it. However, given that even his friends and allies disagree (Dumbledore is a deathist, McGonagall is a disciplinarian, Hermione thinks they are too young, Weasley twins just want to have fun), and he gets the most help from the anti-hero Quirrell, this point of view is hard to defend.
The situation is much worse outside of fiction, where buggy or limited wetware constantly leads would-be heroes (Lenin, Castro, Lincoln or even Hitler) to commit or cause suffering or destruction of the same group of people they aspired to help.
So the first question a would-be hero should ask herself is whether she is prepared to live with the consequences of her actions if they backfire. (And if the answer is yes, she is clearly a villain.)
CEV is a construct for AI purposes that actual human beings can't eval - I don't think I've ever seen a human discussion that was helped by invoking it. It's not like Solomonoff Induction where sometimes you really can be helped by thinking formally about Occam's Razor. In practice, human beings arguing about ethics are either already approximating their part of the 'good' as best they can, or they're confused about something much simpler than CEV, like consequentialism. If you should never use the word 'truth' when you can just talk about the object level, and never say 'because it's not optimal!' when that just means 'I don't think you should do that', then there's basically never a good time to talk about CEV - it always deflates out of the sentence unless you're talking directly about FAI.
I suppose my point is that, if you adopt "heroic responsibility", you ought to put in the correspondingly heroic amount of effort into figuring out what a hero ought to do. And given that your Harry plans to take over the world and then radically change it, he ought to do an awful lot of figuring out first. Probably of the same order of magnitude an FAI would.
That's curious, because Solomonoff Induction is something not even an enormously powerful (but computable) AI can evaluate.
Yes, but my point is that thinking about SI or MML in the abstract helps because people sometimes gain insight from asking "How complex is that computer program?" I haven't seen appeal-to-CEV produce much insight in practice, and any insight it could produce can probably be better produced by appealing to the relevant component principle of CEV instead. (Nor yet is this a critique of CEV, because it's meant as an AI design, not as a moral intuition pump.)
Can you provde an example where Solomonoff Induction can be used to gain insight that Occam's razor doesn't help to gain?
How else can you impartially wield Occam's Razor than with a formal model, and what convincing formalization is there other than Kolmogorov Complexity (and assorted variants), which SI in a way extends?
Setting aside the theoretical objections to Solomonoff induction (a priori assumption of computability of the hypotheses, disregard of logical depth, dependance on the details of the computational model, normalization issues), even if you accept it as a proper formalization of Occam's Razor, in order to apply it in a formal argument, you would have to perform an uncomputable calculation.
Since you can't do that, what's left of it?
Besides noting that there are computable versions of Kolmogorov Complexity (such as MML), in your parent comment you contrasted the use of SI with using Occam's Razor itself.
That's what I was asking about, and it doesn't seem like you answered it:
How do you use Occam's Razor, what formalizations do you perceive as "proper", or if you're just intuiting the heuristic, guesstimating the complexity, what is the formal principle that your intuition derives from / approximates and how does it differ from e.g. Kolmogorov Complexity?
If by MML you mean Minimum message length, then I don't think that's correct. This paper compares Minimum message length with Kolmogorov Complexity but it doesn't seem to make that claim.
My point is that Kolmogorov complexity, Solomonoff induction, etc., are matematical constructions with a formal semantics. Talking about "informal" Kolmogorov complexity is pseudo-mathematics, which is usually an attempt to make your arguments sound more compelling than they are by dressing them in mathematical language.
If there is a disagreement about which hypothesis is simpler, trying to introduce concepts such as ill-defined program lengths that can't be computed, can only obscure the terms of the debate, rather than clarifying them.
From the paper you cited:
"(...) MML usually (but not necessarily) restricts the reference machine to a non-universal form in the interest of computational feasibility. (...) As a result, MML can be, and has routinely been, applied with some confidence to many problems of machine learning (...)"
There will be such disagreement about many different hypotheses, and even when there's not our common intuition will usely have approximated the informational content density of the hypotheses, their complexity.
How do you suggest to resolve such disagreements, or reach common ground without resorting to an intuition ultimately resting on complexity measures?
How do you use Occam's Razor without an appeal a formal notion that grounds your intuition? What does your intuition rest on, if not information theory?
Willam of Ockham originally used his principle to argue for the existence of God (God is the only necessary entity, therefore the simplest explanation).
That's a truly epic fail, since Occam's razor is the strongest argument against the existence of God.
It's worth noting that the current formulation "entities must not be multiplied beyond necessity" is much more recent than Ockham's original formulation "For nothing ought to be posited without a reason given, unless it is self-evident (literally, known through itself) or known by experience or proved by the authority of Sacred Scripture."
I suppose that he included the reference to the Sacred Scripture specifically because he realized that without it, God would be the first thing to fly out of the window.
I sometimes wish I knew which philosophers of the time were sincere in their religious disclaimers.
Consider it done.
It's exactly like Solomonoff Induction where most of the time you really can't be helped by thinking formally about Occam's Razor. In practice, human beings arguing about probabilities are either already approximating their part of the 'simple' as best they can, or they're confused about something much simpler (haha) than Solomonoff Induction, like Bayesianism.
Nitpicking note : I don't think the Weasley twins just want to have fun. They are in Griffyndor, in the Order of the Phoenix, they fight the Death Eaters to the end, ... they want to have fun, but they also want others to have fun.
In the canon, sure. Their HPMOR characters are not nearly as nuanced.
Well, it's not as if they've been given nearly as much opportunity to characterize themselves by anything else.
Oh, I did not mean this in a negative way.