I think the whole concept of labeling goods as "fungible" or "non-fungible" is a category error. Everything trades off against something.
Either you value your fingers more than what [some specific amount of money] will buy you or you don't. If you value your fingers more, then keeping them is the right call for you.
Lots of things have a value that we might call "infinite" according to this argument. Everything from a human life to reading a book spoiler counts as "something you cannot buy back if you regret it later."
Even if we choose to label some things as "non-fungible", we must often weigh them against each other nevertheless. I claim, not that the choice never hurts, but that there is no need to feel guilty about it.
Does this also mean there is no such thing as "inherent good"?
Yes.
If so, then one cannot say, "X is good", they would have to say "I think that X is good", for "good" would be a fact of their mind, not the environment.
One can say all sorts of things. People use the phrase "X is good" to mean lots of things: "I'm cheering for X", "I value X", "X has consequences most people endorse", etc. I don't recommend we abandon the phrase, for many phrases are similarly ambiguous but still useful. I recommend keeping this ambiguity in mind, however, and di...
What happens then when a non-thinking thing feels happy? Is that happiness valued? To whom? Or do you think this is impossible?
When a baby feels happy, it feels happy. Nothing else happens.
There are differences among wanting, liking, and endorsing something.
A happy blob may like feeling happy, and might even feel a desire to experience more of it, but it cannot endorse things if it doesn't have agency. Human fulfillment and wellbeing typically involves some element of all three.
An unthinking being cannot value even its own happiness, beca...
I think you have correctly noticed an empirical fact about emotions (they tend to be preferred or dispreferred by animals who experience them) but are drawing several incorrect conclusions therefrom.
First and foremost, my model of the universe leaves no room for it valuing anything. "Values" happen to be a thing possessed by thinking entities; the universe cares not one whit more for our happiness or sadness than the rules of the game of chess care whether the game is won by white or black. Values happen inside minds, they are not fundamental to the ...
I'm assuming the Cosmic Flipper is offering, not a doubling of the universe's current value, but a doubling of its current expected value (including whatever you think the future is worth) plus a little more. If it's just doubling current niceness or something, then yeah, that's not nearly enough.
Did he really? If true, that's actually much dumber than I thought, but I couldn't find anything saying that when I looked.
I wouldn't characterize that as a "commitment to utilitarianism", though; you can be a perfect utilitarian and have value that is linear in matter and energy (and presumably number of people?), or be a perfect utilitarian and have some other value function.
The possible redundancy of conscious patterns was one of the things I was thinking about when I wrote:
...Secondly, and more importantly, I question whether it is possible ev
I don't actually mean the thing you're calling the motte at all, and I'm not sure I agree with the bailey either. The thought experiment as I understand it was never quite a St. Petersburg Paradox because both the payout ("double universe value") and the method of choosing how to play (single initial payment vs repeated choice betting everything each time) are different. It also can't literally be applied to the real world at all, part of the point is that I don't even know what it would look like for this scenario to be possible in the real world, there a...
Thanks for your thoughts, Cam! The confusion as I see it comes from sneaking in assumptions with the phrase "what they are trained to do". What are they trained to do, really? Do you, personally, understand this?
Consider Claude's Constitution. Look at the "principles in full" - all 60-odd of them. Pick a few at random. Do you wholeheartedly endorse them? Are they really truly representative of your values, or of total human wellbeing? What is missing? Would you want to be ruled by a mind that squeezed these words as hard as physically possible, to th...
Love this post. I've also used the five-minute technique at work, especially when facilitating meetings. In fact, there's a whole technique called think-pair-share that goes something like:
There's an optional step involving groups of four, but I'd rarely bother with that one unless it's a really huge meeting (and at that point I'm actively trying to shrink it because huge committees are shit decision-makers).
This was a good post, and shifted my view slightly on accelerating vs halting AI capabilities progress.
I was confused by your "overhang" argument all the way until footnote 9, but I think I have the gist. You're saying that even if absolute progress in capabilities increases as a result of earlier investment, progress relative to safety will be slower.
A key assumption seems to be that we are not expecting doom immediately; i.e. the next major jump in capabilities is deemed nearly impossible to kill us all with misaligned AI. I'm not sure I buy this assumpt...
I found this a very useful post. I would also emphasize how important it is to be specific, whether one's project involves a grand x-risk moonshot or a narrow incremental improvement.
Greetings from The Kingdom of Lurkers Below. Longtime reader here with an intro and an offer. I'm a former Reliability Engineer with expertise in data analysis, facilitation, incident investigation, technical writing, and more. I'm currently studying deep learning and cataloguing EA projects and AI safety efforts, as well as facilitating both formal and informal study groups for AI Safety Fundamentals.
I have, and am willing to offer to EA or AI Safety focused individuals and organizations, the following generalist skills:
Chaos in complex systems is guaranteed but also bounded. I cannot know what the weather will be like in New York City one month from now. I can, however, predict that it probably won't be "tornado" and near-certainly won't be "five hundred simultaneous tornadoes level the city". We know it's possible to build buildings that can withstand ~all possible weather for a very long time. I imagine that a thing you're calling a puppet-master could build systems that operate within predictable bounds robustly and reliably enough to more or less guarantee broad cont... (read more)