= 27d567bc2d48d9f40e9ccc20ea370b15)
cousin_it comments on Formalizing Value Extrapolation - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (48)
The plan doesn't require a truncated AIXI-like solver with lots of hardware. It's a goal specification you can code directly into a self-improving AI that starts out with weak hardware. "Follow the utility function that program X would output if given enough time" doesn't require the AI to run program X, only to reason about the likely outputs of program X.
It doesn't in principle require this, but might in practice, in which case the AI might eat the universe if that's the amount of computational resources necessary to compute the results of running program X. That is a potential downside of this plan.
Well on the dark, sardonic upside, it might find it convenient to eat the people in the process of using their minds to compute a CEV-function. Infinite varieties of infinite hell-eternities for everyone!