Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Comment author: Kaj_Sotala 17 September 2017 04:02:12PM 4 points [-]

LW1.0's problem with karma is that karma isn't measuring anything useful (certainly not quality).

That's the exact opposite of my experience. Higher-voted comments are consistently more insightful and interesting than low-voted ones.

Quality is not decided by majority vote.

Obviously not decided by it, but aggregating lots of individual estimates of quality sure can help discover the quality.

Comment author: Vladimir_Nesov 17 September 2017 04:15:34PM 1 point [-]

Higher-voted comments are consistently more insightful and interesting than low-voted ones.

This was also my experience (on LW) several years ago, but not recently. On Reddit, I don't see much difference between highly- and moderately-upvoted comments, only poorly-upvoted comments (in a popular thread) are consistently bad.

Comment author: DragonGod 16 September 2017 05:54:26PM *  0 points [-]

that hasn't been proven/isnt known to be unsolvable)

An optimistic attitude towards problems that are potentially solvable is instrumentally useful—and dare I argue—instrumentally rational. The drawbacks of encouraging an optimistic attitude towards open problems are far outweighed by the potential benefits.

Comment author: Vladimir_Nesov 16 September 2017 06:09:25PM *  0 points [-]

(The quote markup in your comment designates a quote from your earlier comment, not my comment.)

You are not engaging the distinction I've drawn. Saying "It's useful" isn't the final analysis, there are potential improvements that avoid the horror of intentionally holding and professing false beliefs (to the point of disapproving of other people pointing out their falsehood; this happened in your reply to Ilya).

The problem of improving over the stance of an "optimistic attitude" might be solvable.

Comment author: DragonGod 16 September 2017 05:03:31PM *  2 points [-]

You should tell Google and academia, they will be most interested in your ideas. Don't you think people already thought very hard about this? This is such a typical LW attitude.

This reply contributes nothing to the discussion of the problem at hand, and is quite uncharitable, I hope such replies were discouraged, and if downvoting was enabled, I would have downvoted it.

If thinking that they can solve the problem at hand (and making attempts at it) is a "typical LW attitude", then it is an attitude I want to see more of and believe should be encouraged (thus, I'll be upvoting /u/John_Maxwell_IV 's post). A priori assuming that one cannot solve a problem (that hasn't been proven/isnt known to be unsolvable) and thus refraining from even attempting the problem, isn't an attitude that I want to see become the norm in Lesswrong. It's not an attitude that I think is useful, productive, optimal or efficient.

It is my opinion, that we want to encourage people to attempt problems of interest to the community (the potential benefits are vast (e.g the problem is solved, and/or significant improvements are made on the problem, and future endeavours would have a better starting point), and the potential demerits are of lesser impact (time (ours and whoever attempts it) is wasted on an unpromising solution).

Coming back to the topic that was being discussed, I think methods of costly signalling are promising (for example, when you upvote a post you transfer X karma to the user, and you lose k*X (k < 1)).

Comment author: Vladimir_Nesov 16 September 2017 05:17:15PM 0 points [-]

A priori assuming that one cannot solve a problem

("A priori" suggests lack of knowledge to temper an initial impression, which doesn't apply here.)

There are problems one can't by default solve, and a statement, standing on its own, that it's feasible to solve them is known to be wrong. A "useful attitude" of believing something wrong is a popular stance, but is it good? How does its usefulness work, specifically, if it does, and can we get the benefits without the ugliness?

Comment author: Manfred 15 September 2017 07:58:46PM *  1 point [-]

Moderation is basically the only way, I think. You could try to use fancy pagerank-anchored-by-trusted-users ratings, or make votes costly to the user in some way, but I think moderation is the necessary fallback.

Goodhart's law is real, but people still try to use metrics. Quality may speak for itself, but it can be too costly to listen to the quality of every single thing anyone says.

Comment author: Vladimir_Nesov 15 September 2017 10:17:31PM 0 points [-]

Quality may speak for itself, but it can be too costly to listen to the quality of every single thing anyone says.

Which is why there should be a way to vote on users, not content, the quantity of unevaluated content shouldn't divide the signal. This would matter if the primary mission succeeds and there is actual conversation worth protecting.

Comment author: dglukhov 12 September 2017 07:10:16PM 0 points [-]

Is there a copy of Eliezer's book in russian? I'm having a hard time finding translations for this text.

Comment author: Vladimir_Nesov 12 September 2017 08:52:27PM *  1 point [-]

There is a partial translation of the book and other things at lesswrong.ru.

Comment author: Vladimir_Nesov 08 February 2017 10:00:04AM *  2 points [-]

I'll cite the thought experiment for the reference:

Betting on the Past: In my pocket (says Bob) I have a slip of paper on which is written a proposition P. You must choose between two bets. Bet 1 is a bet on P at 10:1 for a stake of one dollar. Bet 2 is a bet on P at 1:10 for a stake of ten dollars. So your pay-offs are as follows: Bet 1, P is true: 10; Bet 1, P is false: -1; Bet 2, P is true: 1; Bet 2, P is false: -10. Before you choose whether to take Bet 1 or Bet 2 I should tell you what P is. It is the proposition that the past state of the world was such as to cause you now to take Bet 2. [Ahmed 2014, p. 120]

Some comments on your post:

Alice is betting on a past state of the world. She can’t causally influence the past, and she’s uncertain whether the proposition is true or not.

More precisely, Alice is betting on implications of the past state of the world, on what it means about the future, or perhaps on what it causes the future to be. Specifically Alice's action, an implication of the past state of the world. If we say that Alice can causally influence her own action, it's fair to say that Alice can causally influence the truth of the proposition, even if she can't causally influence the state of the past. So she can't influence the state of the past, but can influence implications of the state of the past, such as her own action. Similarly, a decision algorithm can't influence its own code, but can influence the result it computes. (So I'm not even sure what CDT is supposed to do here, since it's not clear that the bet is really on the past state of the world and not on truth of a proposition about the future state of the world.)

Perhaps if the bet was about the state of the world yesterday, LDT would still take Bet 2. Clearly, LDT’s algorithm already existed yesterday, and it can influence this algorithm’s output; so if it chooses Bet 2, it can change yesterday’s world and make the proposition true.

It's better to avoid the idea of "change" in this context. Change always compares alternatives, but for UDT, there is no default state of the world before-decision-is-made, there are only alternative states of the world following the alternative decisions. So a decision doesn't change things from the way they were before it's made to the way after it's made, at most you can compare how things are after one possible decision to how things are after the other possible decision.

Given that, I don't see what role "LDT’s algorithm already existed yesterday" plays here, and I think it's misleading to state that "it can change yesterday’s world and make the proposition true". Instead it can make the proposition true without changing yesterday’s world, by ensuring that yesterday’s world was always such that the proposition is true. There is no change, yesterday’s world was never different and the proposition was never false. What changed (in our observation of the decision making process) is the state of knowledge about yesterday’s world, from uncertainty about the truth of the proposition to knowledge that it's true.

If we choose a more distant point in the past as a reference for Alice’s bet – maybe as far back as the birth of our universe – she’ll eventually be unable to exert any possible influence via logical counterfactuals.

Following from the preceding point, it doesn't matter when the past state of the world is, since we are not trying to influence it, we are instead trying to influence its consequences, which are in the future. There is something unusual about influencing consequences of a construction without influencing the construction itself, but it helps to recall that it's exactly what any program does, when it influences its actions without influencing its code. It's what a human emulation in a computer does, by making decisions without changing the initial image of their brain from which the running emulation was loaded. And it's also what a regular human running inside physics without any emulation does.

Comment author: Johannes_Treutlein 03 February 2017 10:53:46AM 0 points [-]

CDT, TDT, and UDT would not give away the money because there is no causal (or acausal) influence on the number of universes.

I'm not so sure about UDT's response. From what I've heard, depending on the exact formal implementation of the problem, UDT might also pay the money? If your thought experiment works via a correlation between the type of universe you live in and the decision theory you employ, then it might be a similar problem to the Coin Flip Creation. I introduced the latter decision problem in an attempt to make a less ambiguous version of the Smoking Lesion. In a comment in response to my post, cousin_it writes:

Here's why I think egoistic UDT would one-box. From the problem setup it's provable that one-boxing implies finding money in box A. That's exactly the information that UDT requires for decision making ("logical counterfactual"). It doesn't need to deduce unconditionally that there's money in box A or that it will one-box.

One possible confounder in your thought experiment is the agent’s altruism. The agent doesn’t care about which world he lives in, but only about which worlds exist. If you reason from an “updateless”, outside perspective (like Anthropic Decision Theory), it then becomes irrelevant what you choose. This is because if you act in a way that’s only logically compatible with world A, you know you just wouldn’t have existed in the other world. A way around this would be if you’re not completely updateless, but if you instead have already updated on the fact that you do exist. In this case you’d have more power with your decision. “One-boxing” might also make sense if you're just a copy-egoist and prefer to live in world A.

Comment author: Vladimir_Nesov 07 February 2017 03:18:18AM *  1 point [-]

A way around this would be if you’re not completely updateless, but if you instead have already updated on the fact that you do exist.

It's not a given that you can easily observe your existence. From updateless point of view, all possible worlds, or theories of worlds, or maybe finite fragments of reasoning about them, in principle "exist" to some degree, in the sense of being data potentially relevant for estimating the value of everything, which is something to be done for the strategies under agent's consideration. So in case of worlds, or instances of the agent in worlds, the useful sense of "existence" is relevance for estimating the value of everything (or of change in value depending on agent's strategy, which is the sense in which worlds that couldn't contain or think about the agent, don't exist). Since in this case we are talking about possible worlds, they do or don't exist in the sense of having no measure (probability) in the updateless prior (to the extent that it makes sense to talk about the decision algorithm using a prior). In this sense, observing one's existence means observing an argument about the a priori probability of the world you inhabit. In a world that has relatively tiny a priori probability, you should be able to observe your own (or rather the world's) non-existence, in the same sense.

This also follows the principle of reducing concepts like existence or probability (where they make sense) to components of the decision algorithm, and abandoning them in sufficiently unusual thought experiments (where they may fail to make sense, but where it's still possible to talk about decisions). See also this post of Vadim's and the idea of cognitive reductions (looking for the role a concept plays in your thinking, not just for what it could match in the world).

Comment author: Johannes_Treutlein 26 January 2017 03:14:56PM *  0 points [-]

Thanks for your comment! I find your line of reasoning in the ASP problem and the Coin Flip Creation plausible. So your point is that, in both cases, by choosing a decision algorithm, one also gets to choose where this algorithm is being instantiated? I would say that in the CFC, choosing the right action is sufficient, while in the ASP you also have to choose the whole UDP program so as to be instantiated in a beneficial way (similar to the distinction of how TDT iterates over acts and UDT iterates over policies).

Would you agree that the Coin Flip Creation is similar to e.g. the Smoking Lesion? I could also imagine that by not smoking, UDT would become more likely to be instantiated in a world where the UDT agent doesn't have the gene (or that the gene would eliminate (some of) the UDT agents from the worlds where they have cancer). Otherwise there couldn't be a study showing a correlation between UDT agents' genes and their smoking habits. If the participants of the study used a different decision theory or, unlike us, didn't have knowledge of the results of the study, UDT would probably smoke. But in this case I would argue that EDT would do so as well, since conditioning on all of this information puts it out of the reference class of the people in the study.

One could probably generalize this kind of "likelihood of being instantiated" reasoning. My guess would be that an UDT version that takes it into account might behave according to conditional probabilities like EDT. Take e.g. the example from this post by Nate Soares. If there isn't a principled difference to the Coin Flip Case that I've overlooked, then UDT might reason that if it takes "green", it will become very likely that it will be instantiated only in a world where gamma rays hit the UDT agent (since apparently, UDT agents that choose green are "eliminated" from worlds without gamma rays – or at least that's what I have to assume if I don't know any additional facts). Therefore our specified version of UDT takes the red box. The main argument I'm trying to make is that if you solve the problem like this, then UDT would (at least here, and possibly in all cases) become equivalent to updateless EDT. Which as far as I know would be a relief, since (u)EDT seems easier to formalize?

Comment author: Vladimir_Nesov 07 February 2017 02:51:24AM *  1 point [-]

So your point is that, in both cases, by choosing a decision algorithm, one also gets to choose where this algorithm is being instantiated?

To clarify, it's the algorithm itself that chooses how it behaves. So I'm not talking about how algorithm's instantiation depends on the way programmer chooses to write it, instead I'm talking about how algorithm's instantiation depends on the choices that the algorithm itself makes, where we are talking about a particular algorithm that's already written. Less mysteriously, the idea of algorithm's decisions influencing things describes a step in the algorithm, it's how the algorithm operates, by figuring out something we could call "how algorithm's decisions influence outcomes". The algorithm then takes that thing and does further computations that depend on it.

Comment author: Vladimir_Nesov 24 January 2017 09:00:48PM *  2 points [-]

This is similar to the ASP problem, an unusual anthropic use case. The issue with UDT is that it's underspecified for such cases, but I think some of its concepts are still clearer than the classical probability/causality language.

UDT can be reframed in the following way. There is an abstract agent that's not part of any real world of interest, which is just a process that runs according to its program and can't be disrupted with an anvil dropped on its head. It covers all possibilities, so it includes more than one history. Worlds can "incarnate" parts of this process, either directly, by straightforward interpretation of its program with particular observations fed to it, or indirectly, by reasoning about it. As a result, certain events in certain worlds are controlled by the abstract process through such incarnations. (This imagery doesn't apply to PD though, where the controlled thing is not an event in a world; this restriction puts it closer to what TDT does, whereas proof-based UDT is more general.)

The normal way of describing UDT's algorithm (in this restricted form) is that there are three phases. In the first phase, usually screened off by the problem statement, the agent identifies the events in the worlds of interest that it controls. Then, in the second phase, it examines the consequences of the possible action strategies, and selects a strategy. In the third phase, it enacts the strategy, selecting a concrete action depending on observations.

The problem with this in anthropic problems, such as ASP and your Coin Flip Creation problem, is that strategy-selection and action-selection can affect which events are influenced by incarnations of the agent. Some of the computations that could be performed on any of the phases make it impossible to incarnate the agent in some of the situations where it would otherwise get to be incarnated, so the results of the first phase can depend on how the agent is thinking on the subsequent phases. For example, if the agent is just simulated to completion, then it loses access to the action if it takes too long to complete. This also applies to abstract reasoning about the agent, where it can diagonalize that reasoning to make it impossible.

So an agent should sometimes decide how to think, in a way that doesn't discourage too many situations in the worlds where it's thinking that. This creates additional problems (different agents that have to think differently, unlike the unified UDT), but that's outside the scope of this post. For ASP, the trick is to notice how simple its thinking has to be to retain control over Predictor's prediction, and to make the decision within that constraint.

For Coin Flip Creation, an agent that decides counter to its gene doesn't get to inhabit the world with that gene, since there is no difference between the decision making setups in the two worlds other than the agents who are making the decision. The agent will be "eliminated" by Omega from the world whose gene is different from the agent's decision (i.e. not allowed to reach the decision making setup, via an arrangement of the initial conditions), and instead a different agent will be put in control in that world. So one-boxing makes the two-box gene world inaccessible to the agent, and conversely. Since I assume randomizing is impossible or punished in some way, the choice is really between which world the agent will inhabit, in which case the one-box world seems a bit better (the other world will be inhabited by an agent with a different decision theory, possibly a crazier one, less capable of putting money to good use). If the agent is "altruistic" and doesn't expect much difference in how its counterpart will manage its funds, the choice doesn't matter. On the other hand, if the agent were told its gene, then it should just go with it (act according to the gene), since that will give it access to both worlds (in this case, it doesn't matter at all what's in the boxes).

Comment author: Lumifer 07 December 2016 06:49:02PM *  0 points [-]

So, undergrad textbooks. Let me quote Andrew Gelman (a professor of statistics at Columbia):

Dear Major Academic Publisher,

You just sent me, unsolicited, an introductory statistics textbook that is 800 pages and weighs about 5 pounds. It’s the 3rd edition of a book by someone I’ve never heard of. That’s fine—a newcomer can write a good book. The real problem is that the book is crap. It’s just the usual conventional intro stat stuff. The book even has a table of the normal distribution on the inside cover! How retro is that?

The book is bad in so many many ways, I don’t really feel like going into it. There’s nothing interesting here at all, the examples are uniformly fake, and I really can’t imagine this is a good way to teach this material to anybody. None of it makes sense, and a lot of the advice is out-and-out bad (for example, a table saying that a p-value between 0.05 and 0.10 is “moderate evidence” and that a p-value between 0.10 and 0.15 is “slight evidence”). This is not at all the worst thing I saw; I’m just mentioning it here to give a sense of the book’s horrible mixture of ignorance and sloppiness.

I could go on and on. But, again, I don’t want to do so.

I can’t blame the author, who, I’m sure, has no idea what he is doing in any case. It would be as if someone hired me to write a book about, ummm, I dunno, football. Or maybe rugby would be an even better analogy, since I don’t even know the rules to that one.

Comment author: Vladimir_Nesov 08 December 2016 10:19:55PM *  1 point [-]

Yes, it's worthwhile to spend quite a lot of time on choosing which textbooks to study seriously, at least a fraction of the time needed to study them, and to continually reevaluate the choice as you go on.

View more: Next