All of A.H.'s Comments + Replies

HCH is not defined in this post, nor in the link, about it.

For those reading who do not know what HCH means (like me!), HCH is a recursive acronym which stands for 'Humans Consulting HCH', an idea I think originating with Paul Christiano related to iterated amplification. It involves humans being able to recursively consult copies/simulations of themselves to solve a problem. It is discussed and explained in more detail in these two posts:

Humans Consulting HCH

Strong HCH

I Googled things like 'Aubrey de Grey on Demis Hassabis' for 5 minutes and couldn't find anything matching this description. The closest I could find was this interview with de Grey where he says:

I actually know a lot of people who are at the cutting edge of AI research. I actually know Demis Hassabis, the guy who runs DeepMind, from when he was an undergraduate at Cambridge several years after me. We’ve kept in touch and try to connect every so often.

He says they know each other and keep in touch but its not really a character reference. 

(I'm not cla... (read more)

Here is an archived version of the page : 

http://web.archive.org/web/20050403015136/http://www.cenqua.com/commentator/

Well, you can't have some states as "avoid at all costs" and others as "achieve at all costs", because having them in the same lottery leads to nonsense, no matter what averaging you use. And allowing only one of the two seems arbitrary. So it seems cleanest to disallow both.

Fine. But the purpose of exploring different averaging methods is to see whether it expands the richness of the kind of behaviour we want to describe. The point is that using arithmetic averaging is a choice which limits the kind of behaviour we can get. Maybe we want to describe behav... (read more)

(apologies for taking a couple of days to respond, work has been busy)

I think your robot example nicely demonstrates the difference between our intuitions. As cubefox pointed out in another comment, what representation you want to use depends on what you take as basic.

There are certain types of preferences/behaviours which cannot be expressed using arithmetic averaging. These are the ones which violate VNM, and I think violating VNM axioms isn't totally crazy. I think its worth exploring these VNM-violating preferences and seeing what they look like when m... (read more)

Thanks for pointing this out, I missed a word. I have added it now.

Without wishing to be facetious: how much (if any) of the post did you read?  If you disagree with me, that's fine, but I feel like I'm answering questions which I already addressed in the post!

Are you arguing that we ought to (1) assign some "goodness" values to outcomes, and then (2) maximize the geometric expectation of "goodness" resulting from our actions?

I'm not arguing that we ought to maximize the geometric expectation of "goodness" resulting from our actions. I'm exploring what it might look like if we did. In the conclusion, (and indeed, man... (read more)

The word 'utility' can be used in two different ways: normative and descriptive. 

You are describing 'utility' in the descriptive sense. I am using it in the normative sense. These are explained in the first paragraph of the Wikipedia page for 'utility'.

As I explained in the opening paragraph, I'm using the word 'utility' to mean the goodness/desirability/value of an outcome. This is normative: if an outcome is 'good' then there is the implication that you ought to pursue it.

This looks like an exact duplicate of something you posted a month ago.

Is this intentional?

Thanks for the comment. Naively, I agree that this sounds like a good idea, but I need to know more about it.

Do you know if anyone has explicitly written down the value learning solution to the corrigibility problem and treated it a bit more rigorously ?

Thanks for this comment-it explains your view very clearly and I understand what you are getting at now.

I think its a fair criticism. I've added footnotes within the post, linking people to your comment.

I still think it's a problem that this argument rests on the idea that investors are irrationally not renting land they own, but you don't provide any evidence for that.

I disagree. Firstly, even if, they were renting out their land, this would still be bad, for reasons described in the article (landlords extract land rent without doing anything productive etc.)

The section of the post which argues about empty homes rests on the fact that there are empty homes and a land tax would reduce them. I then provide evidence that there are, indeed, a significant num... (read more)

The second link is to a Scottish political campaign that doesn't claim to know way the houses are empty (at least on this page) and doesn't contain the 700,000 number in the link text (the linked political campaign claims 46,000 in Scotland and doesn't seem to say anything about the UK).

The phrase '700,000 empty homes throughout' the UK has different links for each word: one for England, Northern Ireland, Scotland, and Wales. If you follow the link on 700,000, you will be taken to this page which gives a figure of 676,304 empty homes in England. Add this t... (read more)

Interesting, thanks for sharing! I hadn't heard of this.

From Wikipedia:

An El Nino during the winter of 1998 produced above-average rainfall, which enabled extensive growth of underbrush and vegetation in the state's forests. In early April, however, the rains came to an abrupt halt, and the ensuing drought lasted until July.^[2] These months of continuing dry conditions saw the drought index rise to 700 (out of 800), indicating wildfire potential similar to that usually found in western states.

I would assume that the drought was also exacerbated by El Nino,... (read more)

Eyeballing it, doesn't it imply that while 2024 will be hotter than 2023, the difference between 2024 and 2023 will be smaller than the difference between 2023 and 2022? Because the slope of the various lines is decreasing and in no case increasing?

Yeah, that sounds right I think.

Or is the y-axis measuring YoY impact rather than impact-relative-to-some-fixed-beginning? If so then I'm confused why the global warming section looks the way it does.

I agree, I don't think that YoY interpretation makes sense. I realise now that it's not made completely clear but... (read more)

Yeah, thanks for highlighting this. I started writing about it but realised I was out of my depth (even further out of my depth than for the rest of the post!) so I scrapped it. 

Thanks for clarifying with Robert Rohde!

I reached roughly the conclusion you did. When water vapour is injected into the troposphere (the lowest level of the atmosphere) it is quickly rained out, as you point out. However, the power of the Hunga-Tonga explosion meant that the water vapour was injected much higher, into the stratosphere (what the diagram calls the 'upper atmosp... (read more)

plenty of people are very good at math but never produce any technical writing on scientific journals

Fair enough! Its just that, unless they produce technical results, or pass graduate exams or do something else tangible its quite hard to distinguish people who are very good at math from people who are not.

his story seems to strongly imply that his past self wouldn't have been able to pass those math classes

Obviously its hard to tell from that interview, but he seems to suggest that the reason he didn't pass his classes was because he spent time partying, ... (read more)

Sorry to be a party pooper, but I find the story of Jason Padgett (the guy who 'banged his head and become a math genius') completely unconvincing. From the video that you cite, here is the 'evidence' that he is 'math genius':

• He tells us, with no context, 'the inner boundary of pi is f(x)=x sin(pi/x)'. Ok!
• He makes 'math inspired' drawings (some of which admittedly are pretty cool but they're not exactly original) and sells them on his website
• He claims that a physicist (who is not named or interviewed) saw him drawing in the mall, and, on the basis of this,

I don't find it convincing that what you experienced has any relation to sudden savant syndrome. It sounds like you had a waking dream where you believed you can play the piano.

You did not actually play the piano and produce music though, right?

I have had dreams where I have believed I could do all kinds of things (play the guitar, lift heavy weights, fly etc.), but they didn't overflow in any way to real life. (I've even had dreams where I've thought to myself 'I know that I am dreaming, but this is definitely going to work when I wake up')

If I ask you to... (read more)

Yes, I too am more concerned from a 'maybe this framing isn't super useful as it fails to capture important distinctions between corrigible and non-corrigible' point of view rather than a 'we might outlaw some good actions' point of view.

Thanks for the links, they look interesting!

For $h_a$, this is the policy that is optimal when $q=0$ which has $u\left(h_a\right)=a\left(h_a\right)$. Then $\frac{du}{dE\left(K\right)}<0$. 

Please could you explain how you get $\frac{du}{dE\left(K\right)}<0$ when $u=a=E\left(A\right(1-K\left)\right)$?

Possibly a dumb question but I don't have a good intuition for what it means to differentiate an expected value with respect to an expected value.

I can see that this is the case when $A$ is positive (as expected for a utility function) and uncorrelated with $K$, but is is true in general? Even when $A$ is strongly correlated (o... (read more)

I find that the walkinlabs.com domain does not give any results. I think the correct url is www.walkinlab.com (no 's' in the url). Is this the one you used?

Good point! Noticeably, some of your examples are 'one-way': one party updated while the other did not. In the case of Google/Twitter and the museum, you updated but they didn't, so this sounds like standard Bayesian updating, not specifically Aumann-like (though maybe this distinction doesn't matter, as the latter is a special case of the former).  

When I wrote the answer, I guess I was thinking about Aumann updating where both parties end up changing their probabilities (ie. Alice starts with a high probability of some proposition P and Bob starts w... (read more)

Oh damn, you're right. That was a stupid mistake. 

Yes, so the 3-8 billion fish per day does overstate the number of farmed fish killed. The real number of farmed fish killed per day is somewhere between 0.1 billion and 0.5 billion, which is a lot less than the wild fish killed per day.

EDIT: AS POINTED OUT BY LOCALDEITY THIS COMMENT IS WRONG - I CONFUSED ANNUAL AND DAILY FISH DEATHS. HOWEVER, IT IS THE CASE THAT THIS POST OVERSTATES THE NUMBER OF FISH KILLED FROM FISH FARMS. SEE COMMENTS BELOW FOR CLARIFICATION.

I was going to call you out for a bit of a bait-and-switch in the paragraph starting 'Lewis Bollard notes...'

Lewis Bollard notes “The fishing industry alone kills 3-8 billion animals every day, most by slow suffocation, crushing, or live disemboweling.” So roughly the same number of fish are killed in horrifying, inhumane ways eve

I see, thanks for taking the time to explain!

sometimes the batter near the bowler starts to run before the bowler has actually thrown.

Yes!

In the rules of cricket, that gives the bowler the chance to get them out instead of throwing the ball like they normally would.

There is a line drawn on the floor known as the 'crease', about a metre pasts the stumps. If the batter has run past this line while the bowler still has the ball, the bowler can tap the stumps with the ball and get the batter out.

It's in the spirit of cricket for the bowler to say "hey, if you do that I'm gonna try to get you out". It's n

Thanks, that makes sense! And to be clear, would an 'explanation' be a program which could generate the data 3,1,4,1,5,9? And a good explanation would be one which took up fewer bits of information than just the list 3,1,4,1,5,9? 

This seems very interesting but I'm having trouble understanding something. Can you specify what is meant by:

An explanation is good if it is smaller than just hard-coding the answer.

What does 'just hard-coding the answer' mean and look like?

The purpose of participating in a game is to maximize performance, think laterally, exploit mistakes, and do everything you can, within the explicit rules, to win. Doing that is what makes games fun to play. Watching other people do that, at a level that you could never hope to reach is what makes spectator sports fun to watch.

I don't know if you read the rest of the piece, but the point I was trying to make is that sometimes this isn't true! Sometimes if each team does everything within the rules to win then the game becomes less fun to watch and play (yo... (read more)

I think the two-player-game-but-player2-gets-to-modify-the-rules is not a fair analogy here. Like I said it's the cricket-loving public that decides, not player 2.

Broadly, I agree with Richard Ngo's characterisation. You are right that the 'cricket loving public' plays some part in determining what counts as 'within the spirit' but it is the decision of the players themselves that often is most important.

How is this different from games with a referee? A foul is what the referee says it is; the spirit of cricket is what the cricket-lovers say it is. In bot

Interesting, thanks for sharing! Its cool to see how different games manage the conflict between coming up with innovative tactics (which for me is all part of the fun of sports) and exploiting the rules in a way that makes the game boring.

Also thanks for the link to David Sirlin. I haven't heard of him and the website looks interesting!

Not a money pump unless there's some path back to "trust me enough that I can extort you again", but that's unlikely related to ethical framework.

I don't understand this. Why would paying out to an extortionist once make you disbelieve them when they threatened you a second time?

The "give me money otherwise I'll kill you" money pump is arguably not a money pump

I'm not sure how you mean this. I think that it is a money pump when combined with the assumption that you want to stay alive. You pay money to end up in the same position you started in (presuming you want to stay alive). When back in the position you started, someone can then threaten you again in the same way and get more money from you. It just has fewer steps than the standard money pump. Sure, you could reject the 'I want to stay alive' assumption but then you end up d... (read more)

I am confused about something. You write that a preference ordering $L⪯M$ is geometrically rational if$G_{U\sim P}{E}_{O\sim L}U\left(O\right)\le G_{U\sim P}{E}_{O\sim M}U\left(O\right)$.

This is compared to VNM rationality which favours $L⪯M$ if and only if $E_{O\sim L}U\left(O\right)\le E_{O\sim M}U\left(O\right)$. 

Why, in the the definition of geometric rationality, do we have both the geometric average and the arithmetic average? Why not just say "an ordering is geometrically rational if it favours $L⪯M$ if and only if $G_{O\sim L}U\left(O\right)\le G_{O\sim M}U\left(O\right)$ " ?

As I understand it, this is what Kelly betting does. It doesn't favou... (read more)

I think this is a good idea, thanks for implementing!

Very minor but the link lesswrong.com/moderation#rejected-comments just goes to the same page as lesswrong.com/moderation#rejected-posts (the written address is correct but the hyperlink goes to the wrong page)

The link to Harsanyi's paper doesn't work for me. Here is a link that does, if anyone is looking for one: 

https://hceconomics.uchicago.edu/sites/default/files/pdf/events/Harsanyi_1955_JPE_v63_n4.pdf 

The infinite non-uniform discrete case is not much more difficult. If $X$ is a finite or countably infinite set, $f:X\to R^{\ge 0}$ assigns a nonnegative value to each $x\in X$, and $P$ is a probability distribution on $Y$, then $E_{x\sim P}f\left(x\right)={\sum }_{x\in X}P\left(x\right)f\left(x\right)$

Very minor, but shouldn't this read "$P$ is a probability distribution on $X$" not $Y$?

Thanks for writing this. I think that the arguments in parts III and IV are particularly compelling and well-written.

Thanks for writing this. I wanted to write something about how Deutsch performs a bit a of motte-and-bailey argument (motte:'there are some problems in physics which are hard to solve using the dynamical laws approach'. bailey:'these problems can be solved using constructor theory specifically, rather than other approaches'). Your comment does a good job of making this case. In the end I didn't include it, as the piece was already too long. I just wrote the sentence 

Pointing out problems in the dynamical laws approach to physics and trying to find solutions is useful, even if constructor theory turns out not to be the best solution to them. 

and left it at that.

I didn’t use ‘modal’ because that is used to refer to logical possibility/impossibility, whereas I am interested in referring to physical possibility/impossibility. Depending on your philosophical views, those two things may or may not be the same.

The form of a counterfactual law ("your perpetual motion machine won't work even if you make that screw longer or do anything else different") seems to be "A, no matter which parameter you change".

I don’t think this is right. As I am using it, ‘counterfactual’ refers to a statement about whether something is possible or impossible. Statements of the form "A, no matter which parameter you change" are not always like this. For example if A=’this ball has a mass of 10kg’. This is not a statement about what is possible or impossible. You could frame it as ‘it ... (read more)

Glad that confusion is removed!

I think that it is the best word to use. When used as an adjective Collins defines 'counterfactual' as 'expressing what has not happened but could, would, or might under differing conditions '. I think that this fits the way I was talking about it (eg. when referring to 'counterfactual laws'). In the first post, I talk about whether the lamp 'could would, or might' have been in a different state. In this post, we talk about whether a perpetual motion machine  'could would, or might' work if it was made using a different ... (read more)

Hi, thanks for the question. I am using the term 'counterfactual' (admittedly somewhat loosely) to describe facts that refer to whether things are possible or impossible, regardless of whether they actually happen. 

In the first post, I claimed that it is only meaningful to say that the lamp transmits information if it is possible for the lamp to be a in a different state. Conversely, if the lamp was broken, then it is impossible for the lamp to be in a different state, and information does not get transmitted. If you just describe the system in terms ... (read more)

I think this was well-written and clear, so good job there! I also happen to disagree with the contents.

Thanks for your comment!

First off, I'm highly suspicious of any definition of a "prevailing conception" of physics that excludes the second law of thermodynamics! It seems like in actual practice, sometimes people make predictions by simulation, (the "PC") sometimes they make predictions by generalizing about the character of physical law (the quantum gravity example), and sometimes they do something in between those things and make abstractions/generali

I think I disagree with your characterisation of the split between 'objective' Shannon information and information as meaning, which requires interpretation.

As you point at the end of your comment, Shannon information requires you to know the probability distribution from which your data is drawn. And probabilities are reflections of your own state of knowledge, which is subjective. (Or at least subjectively objective, if you are using 'objective' in that sense, then I guess I agree.) For example, if Alice sends Bob a string '11111', we might be tempted to... (read more)

I agree with your example and think that it touches on something important. However, in this post, I did not claim that the counterfactual condition was the only condition required for information transfer. You are correct to say that the lamp signal would not constitute information to someone who was unaware of the plan. But this is because, in that situation, there are other conditions that have not been met. Since the other person seeing the lamp signal would not react differently to the different signals, there is no causal link between the signal and ... (read more)

Hi, thanks for your question. I have a big piece covering all of this in more detail which I plan to post in a couple of days once I've finished writing it. In the meantime, please accept this 'teaser' of a few problems in the prevailing conception (PC):

1. Dealing with hybrid systems. If we are operating in a regime where there are two contradictory sets of dynamical laws, we do not know what kind of evolution the system will follow. An example of such a system is one where both gravity (as governed by general relativity) and quantum mechanics are relevant. I