Stupid questions thread, October 2015

philh

This thread is for asking any questions that might seem obvious, tangential, silly or what-have-you. Don't be shy, everyone has holes in their knowledge, though the fewer and the smaller we can make them, the better.

Please be respectful of other people's admitting ignorance and don't mock them for it, as they're doing a noble thing.

To any future monthly posters of SQ threads, please remember to add the "stupid_questions" tag.

Please be respectful of other people's admitting ignorance and don't mock them for it, as they're doing a noble thing.

To any future monthly posters of SQ threads, please remember to add the "stupid_questions" tag.

Someone downvoted the question above. What the hell? (My guess: it's VoiceOfRa doing his downvote-the-enemy thing again.)

To the actual question: first of all, I think it's entirely possible that we have additional layers (sys1 means "fast heuristic", sys2 means "slow deliberate reasoning"; we surely have a big bag of heuristics, and I bet there are cases where we have extra-fast heuristics, fastish heuristics, and slow deliberate reasoning); and it seems like one could envisage an AI with (1) nothing like sys1 at all because its "proper" reasoning is cheap enough to be used all the time, (2) a human-like bag of heuristics that get used when circumstances allow, producing much the same distinction as we have, (3) smoothly varying how-much-approximation knobs that adjust according to how valuable quicker answers are, interpolating continuously between "system 1" and "system 2", and probably (4) all sorts of other things I haven't thought of.

The sort of provably-safe AI that MIRI would like to see would presumably either be in category 1, or be designed so that in some sense sufficiently consequential decisions always get made "properly". The latter seems like it would be hard to reason about. (Er, or it might be in category 4 in which case by definition I have nothing to say about it.)

Is retributive downvoting on other forums, or is it just a LW thing? Do we have more retributive downvoting than other sites? Can anyone think of some relationship between rationality and vindication? I feel like if anything we should be above that and have far less...

-1SilentCal11y

I'd expect MIRI to build the "proper reasoning" layer, and then the AI would use its "proper reasoning" to design fast heuristic layers. One example of a very-hard-to-avoid layering would be if the AI were distributed over large areas (global or perhaps interplanetary), there would probably have to be decisions made locally, but other decisions made globally. You could say the local processes are subagents and only the global process is the real AI, but it wouldn't change the fact that the AI has designed trustworthy faster less-accurate decision processes.

2NancyLebovitz11y

I was wondering about VoR and that downvote. Now that I think about it, we do have a range of speeds, including the occasional sudden revelation (as when an addict realizes that they really can and must stop).

7

Stupid questions thread, October 2015

7

7

7

Stupid questions thread, October 2015

7

7