All of Dacyn's Comments + Replies

Basically both of these arguments will seem obvious if you fall into camp #2 here, and nonsensical if you fall into camp #1.

Memento is easily one of the best movies about “rationality as practiced by the individual” ever made. [...] When the “map” is a panoply of literal paper notes and photographs, and the “territory” is further removed from one’s lived experience than usual… it behooves one to take rationality, bias, motivated cognition, unquestioned assumptions, and information pretty damn seriously!

Wasn't the main character's attempt at "rationality as practiced by the individual" kind of quixotic though? I didn't get the impression that the moral of the story was "you s... (read more)

Let’s say my p(intelligent ancestor) is 0.1. Imagine I have a friend, Richard, who disagrees.

No wait, the order of these two things matters. Is P(intelligent ancestor|just my background information) = 0.1 or is P(intelligent ancestor|my background information + the fact that Richard disagrees) = 0.1? I agree that if the latter holds, conservation of expected evidence comes into play and gives the conclusion you assert. But the former doesn't imply the latter.

I don't agree that I am making unwarranted assumptions; I think what you call "assumptions" are merely observations about the meanings of words. I agree that it is hard to program an AI to determine who the "he"s refer to, but I think as a matter of fact the meanings of those words don't allow for any other possible interpretation. It's just hard to explain to an AI what the meanings of words are. Anyway I'm not sure if it is productive to argue this any further as we seem to be repeating ourselves.

No, because John could be speaking about himself administering the medication.

If it's about John administering the medication then you'd have to say "... he refused to let him".

It’s also possible to refuse to do something you’ve already acknowledged you should do, so the 3rd he could still be John regardless of who is being told what.

But the sentence did not claim John merely acknowledged that he should administer the medication, it claimed John was the originator of that statement. Is John supposed to be refusing his own requests?

John told Mark that he should administer the medication immediately because he was in critical condition, but he refused.

Wait, who is in critical condition? Which one refused? Who’s supposed to be administering the meds? And administer to whom? Impossible to answer without additional context.

I don't think the sentence is actually as ambiguous as you're saying. The first and third "he"s both have to refer to Mark, because you can only refuse to do something after being told you should do it. Only the second "he" could be either John or Mark.

Early discussion of AI risk often focused on debating the viability of various elaborate safety schemes humanity might someday devise—designing AI systems to be more like “tools” than “agents,” for example, or as purely question-answering oracles locked within some kryptonite-style box. These debates feel a bit quaint now, as AI companies race to release agentic models they barely understand directly onto the internet.

Why do you call current AI models "agentic"? It seems to me they are more like tool AI or oracle AI...

I am still seeing "succomb".

In the long scale a trillion is 10^18, not 10^24.

I say "zero" when reciting phone numbers. Harder to miss that way.

I think you want to define $A?B$ to be true if $A$ is true when we restrict to some neighbourhood $N\in N_w$ such that $A\cap N$ is nonempty. Otherwise your later example doesn't make sense.

I noticed all the political ones were phrased to support the left-wing position.

This doesn't completely explain the trick, though. In the step where you write f=(1-I)^{-1} 0, if you interpret I as an operator then you get f=0 as the result. To get f=Ce^x you need to have f=(1-I)^{-1} C in that step instead. You can get this by replacing \int f by If+C at the beginning.

If you find yourself thinking about the differences between geometric expected utility and expected utility in terms of utility functions, remind yourself that, for any utility function, one can choose* either* averaging method.

No, you can only use the geometric expected utility for nonnegative utility functions.

It's obvious to us that the prompts are lying; how do you know it isn't also obvious to the AI? (To the degree it even makes sense to talk about the AI having "revealed preferences")

Calvinists believe in predestination, not Protestants in general.

Wouldn't that mean every sub-faction recursively gets a veto? Or do the sub-faction vetos only allow the sub-faction to veto the faction veto, rather than the original legislation? The former seems unwieldy, while the latter seems to contradict the original purpose of DVF...

(But then: aren’t there zillions of Boltzmann brains with these memories of coherence, who are making this sort of move too?)

According to standard cosmology, there are also zillions of actually coherent copies of you, and the ratio is heavily tilted towards the actually coherent copies under any reasonable way of measuring. So I don't think this is a good objection.

“Only food that can be easily digested will provide calories”

That statement would seem to also be obviously wrong. Plenty of things are ‘easily digested’ in any reasonable meaning of that phrase, while providing ~0 calories.

I think you've interpreted this backwards; the claim isn't that "easily digested" implies "provides calories", but rather that "provides calories" implies "easily digested".

In constructivist logic, proof by contradiction must construct an example of the mathematical object which contradicts the negated theorem.

This isn't true. In constructivist logic, if you are trying to disprove a statement of the form "for all x, P(x)", you do not actually have to find an x such that P(x) is false -- it is enough to assume that P(x) holds for various values of x and then derive a contradiction. By contrast, if you are trying to prove a statement of the form "there exists x such that P(x) holds", then you do actually need to construct an example of x such that P(x) holds (in constructivist logic at least).

Just a technical point, but it is not true that most of the probability mass of a hypothesis has to come from "the shortest claw". You can have lots of longer claws which together have more probability mass than a shorter one. This is relevant to situations like quantum mechanics, where the claw first needs to extract you from an individual universe of the multiverse, and that costs a lot of bits (more than just describing your full sensory data would cost), but from an epistemological point of view there are many possible such universes that you might be a part of.

As I understood it, the whole point is that the buyer is proposing C as an alternative to A and B. Otherwise, there is no advantage to him downplaying how much he prefers A to B / pretending to prefer B to A.

Hmm, the fact that C and D are even on the table makes it seem less collaborative to me, even if you are only explicitly comparing A and B. But I guess it is kind of subjective.

It seems weird to me to call a buyer and seller's values aligned just because they both prefer outcome A to outcome B, when the buyer prefers C > A > B > D and the seller prefers D > A > B > C, which are almost exactly misaligned. (Here A = sell at current price, B = don't sell, C = sell at lower price, D = sell at higher price.)

Isn't the fact that the buyer wants a lower price proof that the seller and buyer's values aren't aligned?

You're right that "Experiencing is intrinsically valuable to humans". But why does this mean humans are irrational? It just means that experience is a terminal value. But any set of terminal values is consistent with rationality.

Of course, from a pedagogical point of view it may be hard to explain why the "empty function" is actually a function.

When you multiply two prime numbers, the product will have at least two distinct prime factors: the two prime numbers being multiplied.

Technically, it is not true that the prime numbers being multiplied need to be distinct. For example, 2*2=4 is the product of two prime numbers, but it is not the product of two distinct prime numbers.

As a result, it is impossible to determine the sum of the largest and second largest prime numbers, since neither of these can be definitively identified.

This seems wrong: "neither can be definitively identified" makes ... (read more)

OK, that makes sense.

OK, that's fair, I should have written down the precise formula rather than an approximation. My point though is that your statement

the expected value of X happening can be high when it happens a little (because you probably get the good effects and not the bad effects Y)

is wrong because a low probability of large bad effects can swamp a high probability of small good effects in expected value calculations.

Yeah, but the expected value would still be $K(\frac{N-1}{N}A-\frac{B}{N})$.

I don't see why you say Sequential Proportional Approval Voting gives little incentive for strategic voting. If I am confident a candidate I support is going to be elected in the first round, it's in my interest not to vote for them so that my votes for other candidates I support will count for more. Of course, if a lot of people think like this then a popular candidate could actually lose, so there is a bit of a brinksmanship dynamic going on here. I don't think that is a good thing.

The definition of a derivative seems wrong. For example, suppose that $f\left(x\right)=0$ for rational $x$ but $f\left(x\right)=1$ for irrational $x$. Then $f$ is not differentiable anywhere, but according to your definition it would have a derivative of 0 everywhere (since $\Delta x$ could be an infinitesimal consisting of a sequence of only rational numbers).

But if they are linearly independent, then they evolve independently, which means that any one of them, alone, could have been the whole thing—so why would we need to postulate the other worlds? And anyway, aren’t the worlds supposed to be interacting?

Can't this be answered by an appeal to the fact that the initial state of the universe is supposed to be low-entropy? The wavefunction corresponding to one of the worlds, run back in time to the start of the universe, would have higher entropy than the wavefunction corresponding to all of them together, so it's not as good a candidate for the starting wavefunction of the universe.

No, the whole premise of the face-reading scenario is that the agent can tell that his face is being read, and that's why he pays the money. If the agent can't tell whether his face is being read, then his correct action (under FDT) is to pay the money if and only if (probability of being read) times (utility of returning to civilization) is greater than (utility of the money). Now, if this condition holds but in fact the driver can't read faces, then FDT does pay the $50, but this is just because it got unlucky, and we shouldn't hold that against it.

In your new dilemma, FDT does not say to pay the $50. It only says to pay when the driver's decision of whether or not to take you to the city depends on what you are planning to do when you get to the city. Which isn't true in your setup, since you assume the driver can't read faces.

a random letter contains about 7.8 (bits of information)

This is wrong, a random letter contains log(26)/log(2) = 4.7 bits of information.

I have tinnitus every time I think about the question of whether I have tinnitus. So do I have tinnitus all the time, or only the times when I notice?

I was confused at first what you meant by "1 is true" because when you copied the post from your blog you didn't copy the numbering of the claims. You should probably fix that.

The number 99 isn’t unique—this works with any payoff between 30 and 100.

Actually, it only works with payoffs below 99.3 -- this is the payoff you get by setting the dial to 30 every round while everyone else sets their dials to 100, so any Nash equilibrium must beat that. This was mentioned in jessicata's original post.

Incidentally, this feature prevents the example from being a subgame perfect Nash equilibrium -- once someone defects by setting the dial to 30, there's no incentive to "punish" them for it, and any attempt to create such an incentive via a "punish non-punishers" rule would run into the trouble that punishment is only effective up to the 99.3 limit.

It's part of the "frontpage comment guidelines" that show up every time you make a comment. They don't appear on GreaterWrong though, which is why I guess you can't see them...

I explained the problem with the votes-per-dollar formula in my first post. 45% of the vote / $1 >> 55% of the vote / $2, so it is not worth it for a candidate to spend money even if they can buy 10% of the vote for $1 (which is absurdly unrealistically high).

When I said maybe a formula would help, I meant a formula to explain what you mean by "coefficient" or "effective exchange rate". The formula "votes / dollars spent" doesn't have a coefficient in it.

If one candidate gets 200 votes and spends 200 dollars, and candidate 2 gets 201 votes and spen

I don't think the data dependency is a serious problem, all we need is a very loose estimate. I don't know what you mean by a "spending barrier" or by "effective exchange rate", and I still don't know what coefficient you are talking about. Maybe it would help if you wrote down some formulas to explain what you mean.

I don't understand what you mean; multiplying the numerator by a coefficient wouldn't change the analysis. I think if you wanted to have a formula that was somewhat sensitive to campaign spending but didn't rule out campaign spending completely as a strategy, Votes/(10X+Y) might work, where Y is the amount spent of campaign spending, and X is an estimate of average campaign spending. (The factor of 10 is because campaign spending just isn't that large a factor to how many votes you get in absolute terms; it's easy to get maybe 45% of the vote with no campaign spending at all, just by having (D) or (R) in front of your name.)

The result of this will be that no one will spend more than the $1 minimum. It's just not worth it. So your proposal is basically equivalent to illegalizing campaign spending.

I wonder whether this one is true (and can be easily proved): For a normal form game G and actions ai for a player i, removing a set of actions a−i from the game yields a game G− in which the Nash equilibria are worse on average for i (or alternatively the pareto-best/pareto-worst Nash equilibrium is worse for G− than for G).

It's false: consider the normal form game

(0,0) (2,1)

(1,1) (3,0)

For the first player the first option is dominated by the second, but once the second player knows the first player is going to choose the second option, he's moti... (read more)

Not eating meat is not a Pascal's mugging because there is a solid theoretical argument for why the expected value is positive even if the payoff distribution is somewhat unbalanced: if a large number of people decide not to eat meat, then this will necessarily have the effect of shifting production, for supply to meet demand. Since you have no way of knowing where you are in that large ensemble, the expected value of you not eating meat is equal to the size of the effect divided by the number of people in the ensemble, which is presumably what we would ex... (read more)

A proof you don’t understand does not obligate you to believe anything; it is Bayesian evidence like anything else. If an alien sends a 1GB Coq file Riemann.v, running it on your computer does not obligate you to believe that the Riemann hypothesis is true. If you’re ever in that situation, do not let anyone tell you that Coq is so awesome that you don’t roll to disbelieve. 1GB of plaintext is too much, you’ll get exhausted before you understand anything. Do not ask the LLM to summarize the proof.

I'm not sure what you are trying to say here. Even with 1... (read more)