LESSWRONG
LW

All of fin's Comments + Replies

when will LLMs become human-level bloggers?
Answer by fin32

There are some social reasons for writing and reading blogs.

One reason is that “a blog post is a very long and complex search query to find fascinating people and make them route interesting stuff to your inbox”. I expect to continue to value finding new people who share my interests after AI starts writing better blog posts than me, which could be very soon. I'm less sure about whether this continues to be a good reason to write them, since I imagine blog posts will become a less credible signal of what I'm like.

Another property that makes me want to read... (read more)

Reply
Why is Toby Ord's likelihood of human extinction due to AI so low?
Answer by fin50

As Buck points out, Toby's estimate of P(AI doom) is closer to the 'mainstream' than MIRI's, and close enough that "so low" doesn't seem like a good description.

I can't really speak on behalf of others at FHI, of course, by I don't think there is some 'FHI consensus' that is markedly higher or lower than Toby's estimate.

Also, I just want to point out that Toby's 1/10 figure is not for human extinction, it is for existential catastrophe caused by AI, which includes scenarios which don't involve extinction (forms of 'lock-in'). Therefore his estimate for ext... (read more)

Reply
Ethics in Many Worlds
fin20

Yes, I'm almost certain it's too 'galaxy brained'! But does the case rely on entities outside our light cone? Aren't there many 'worlds' within our light cone? (I literally have no idea, you may be right, and someone who knows should intervene)

I'm more confident that this needn't relate to the literature on infinite ethics, since I don't think any of this relies on inifinities.

Reply
1Randomized, Controlled
I use 'light cone' to point at 'something which cannot ever conceivably casually affect anything in our present or future'. I don't know if the light-cone concept generalizes to the Everett QM branches; if not, the substitute 'anything which is in principle unknowable'. 
Ethics in Many Worlds
fin20

Thanks, this is useful.

Reply
I'm still mystified by the Born rule
fin*30

There are some interesting and tangentially related comments in the discussion of this post (incidentally, the first time I've been 'ratioed' on LW).

Reply
Inner Alignment in Salt-Starved Rats
fin10

Thanks, really appreciate it!

Reply
Embedded Interactive Predictions on LessWrong
fin50

Was wondering the same thing — would it be possible to set others' answers as hidden by default on a post until the reader makes a prediction?

Reply
Inner Alignment in Salt-Starved Rats
fin40

I interviewed Kent Berridge a while ago about this experiment and others. If folks are interested, I wrote something about it here, mostly trying to explain his work on addiction. You can listen to the audio on the same page.

Reply
2Steven Byrnes
Update: Both the podcast and the article were interesting, enjoyable, and helpful :-)
2Steven Byrnes
No way! Awesome, looking forward to listening to that! :-)
Ethics in Many Worlds
fin10

Got it, thanks very much for explaining.

Reply
Ethics in Many Worlds
fin10

Thanks, that's a nice framing.

Reply
Ethics in Many Worlds
fin30

Thanks for the response. I'm bumping up against my lack of technical knowledge here, but a few thoughts about the idea of a 'measure of existence' — I like how UDASSA tries to explain how the Born probabilities drop out of a kind of sampling rule, and why, intuitively, I should give more 'weight' to minds instantiated by brains rather than a mug of coffee. But this idea of 'weight' is ambiguous to me. Why should sampling weight (you're more likely to find yourself as a real vs Boltzmann brain, or 'thick' vs 'arbitrary' computation) imply ethical weight (th... (read more)

Reply
1interstice
I think the weights for prediction and moral value should be the same or at least related. Consider, if we're trying to act selfishly, then we should make choices that lead to the best futures according to the sampling weight(conditioned on our experience so far), since the sampling weight is basically defined as our prior on future sense experiences. But then it seems strange to weigh other peoples' experiences differently than our own. I think of the measure as being a generalization of what it means to 'count' experiences, not a property of the experiences themselves. So this is more like how, in utilitarianism, the value of an experience has to be multiplied by the number of people having it to get the total moral value. Here we're just multiplying by the measure instead. People like to claim that, but fundamentally you need to add some sort of axiom that describes how the wave function cashes out in terms of observations. The best you can get is an argument like "any other way of weighting the branches would be silly/mathematically inelegant". Maybe, but you're still gonna have to put it in if you want to actually predict anything. If you want to think of it in terms of writing a computer program, it simply won't return predictions without adding the Born rule(what I'm calling the 'Hilbert measure' here)
2Viliam
I think the actual reason is more like: there is nothing you can do to improve the average experience of Boltzman brains.
Ethics in Many Worlds
fin10

More Notes

Something very like the view I'm suggesting can be found in Albert & Loewer (1988) and their so-called 'many minds' interpretation. This is interesting to read about, but the whole idea strikes me as extremely hand-wavey and silly. Here's David Wallace with a dunk: “If it is just a fundamental law that consciousness is associated with some given basis, clearly there is no hope of a functional explanation of how consciousness emerges from basic physics.”

I should also mention that I tried explaining this idea to another philosopher of physics, ... (read more)

Reply
What are some beautiful, rationalist artworks?
fin30

Thanks, that's far more relevant!

Reply
What are some beautiful, rationalist artworks?
fin70

From Wikipedia: An Experiment on a Bird in the Air Pump is a 1768 oil-on-canvas painting by Joseph Wright of Derby, one of a number of candlelit scenes that Wright painted during the 1760s. The painting departed from convention of the time by depicting a scientific subject in the reverential manner formerly reserved for scenes of historical or religious significance. Wright was intimately involved in depicting the Industrial Revolution and the scientific advances of the Enlightenment. While his paintings were recognized as exceptional by his contemporaries... (read more)

Reply
MikkW100

Thanks for sharing the picture. I feel like the excerpt you posted from Wikipedia isn't the most important part. While it touches on the artistic significance of the piece, it leaves open many questions about the relevance of the piece as a symbol of rationality. I found the following excerpts from the same article to be more enlightening in this regard:

The painting depicts a natural philosopher, a forerunner of the modern scientist, recreating one of Robert Boyle's air pump experiments, in which a bird is deprived of air, before a varied group of onlook

... (read more)
Reply
What are some beautiful, rationalist artworks?
Answer by fin490
An Experiment on a Bird in the Air Pump
Reply
6johnswentworth
I love the ambience of dark fascination in this one. Tension between horror and curiosity.
7fin
From Wikipedia: An Experiment on a Bird in the Air Pump is a 1768 oil-on-canvas painting by Joseph Wright of Derby, one of a number of candlelit scenes that Wright painted during the 1760s. The painting departed from convention of the time by depicting a scientific subject in the reverential manner formerly reserved for scenes of historical or religious significance. Wright was intimately involved in depicting the Industrial Revolution and the scientific advances of the Enlightenment. While his paintings were recognized as exceptional by his contemporaries, his provincial status and choice of subjects meant the style was never widely imitated. The picture has been owned by the National Gallery in London since 1863 and is regarded as a masterpiece of British art. 
What are some beautiful, rationalist artworks?
fin20

'Earthrise'. Taken from lunar orbit by astronaut William Anders on December 24, 1968, during the Apollo 8 mission. Nature photographer Galen Rowell declared it "the most influential environmental photograph ever taken".

Reply
2fin
'Earthrise'. Taken from lunar orbit by astronaut William Anders on December 24, 1968, during the Apollo 8 mission. Nature photographer Galen Rowell declared it "the most influential environmental photograph ever taken".
Review and Summary of 'Moral Uncertainty'
fin20

Understood. I'm not so sure there is such a big difference between uses of 'rational' and 'moral' in terms of implying the existence of norms 'outside of ourselves'. In any case, it sounds to me now like you're saying that everyday moral language assumes cognitivism + realism. Maybe so, but I'm not so sure what this has to do with moral uncertainty specifically.

Reply
Review and Summary of 'Moral Uncertainty'
fin10

Got it, thanks. I think the phrase 'non-physical essences' makes moral realism sound way spookier than necessary. I don't think involve 'essences' in a similar way to how one decision could be objectively more rational than another without there being any rationality 'essences'. But what you're saying sounds basically right. Makes me wonder — it's super unclear what to do if you're also just uncertain between cognitivism and non-cognitivism. Would you need some extra layer of uncertainty and a corresponding decision procedure? I'm really not sure.

Reply
3Charlie Steiner
Yeah, that's a good point. With rationality, though, we can usually agree that half of the heavy lifting is done inside the definition of the word "rational." If I say something controversial, like that preferring chocolate ice cream to vanilla is rational, then you might suppose that we're using the word "rational" in different ways, not that we disagree about some unique and impersonal standard of rationality. Not to say that you can't do the same thing with morality. But when I mention a morality-essence, I mean to imply the other treatment, where there's something outside of ourselves and our definitions that does most of the heavy lifting, so that when we disagree it's probably not that we define morality differently, it's that we disagree about the state of this external factor.
Review and Summary of 'Moral Uncertainty'
fin10

Hmm. Moral uncertainty definitely doesn't assume moral realism. You could just have some credence in the possibility that there are no moral facts.

If instead by 'essentialism' you mean moral cognitivism (the view that moral beliefs can take truth values) then you're right that moral uncertainty makes most sense under cognitivism. But non-cognitivist versions (where your moral beliefs are just expressions of preference, approval, or desire) also seem workable. I'm not sure what any of this has to do with 'non-physical essences' though. I think I know what y... (read more)

Reply
2Charlie Steiner
Yeah, I agree that it's compatible with non-realism / non-cognitivism. But it doesn't make it convenient for you :) For example, your first thought for how to include non-realism was probably to include a possibility in which no moral theory is Right. But this doesn't get you out of trouble, because it's still equivalent to saying that realist moral theories are all that we should base decisions on. Instead, compatibility has to come from interpreting the whole moral uncertainty framework in a different light. This brings us back to "non-physical essences" - things that there are facts about that are independent of physical reality. The most obvious interpretation of moral uncertainty is as uncertainty over the state of this Rightness essence, even given full facts about the physical world. To re-interpret moral uncertainty in a naturalistic way (in terms of, e.g. a human's model of how their feelings might change in the future) seems interesting, but also seems to require swimming against the current of the framing.