A counterpoint may be that human civilization is just the observable of underlying optimization processes (individual and group selection on several levels). Obviously this observable has had variance whenever a new advantage got widely adopted and civilization morphed into the memeplex of whoever had the advantage. So from AI we shouldn't expect this process to suddenly stop. However, the entities with the next advantage may well be agentic AIs.
Probably not saying something new here, but what is the probability of you living at a certain point in the evolution of the universe? Because of grabby aliens and the likelihood that grabby aliens don't have human-like minds you're not likely to live late in the history of the universe but rather early when the life-density of the universe is largest but the various life forms haven't propagated their grabby variants yet.
Have we just discovered one of the actual reasons for the peer-review process sticking around? It forces CoTs to be legible unlocking secondary value?
Take into account exponential compute scaling and exponential efficiency scaling.
NP-hard problems have no known efficient algorithms, strong evidence suggests none exist
Can you back up this claim? I find the evidence I have seen pretty weak in a mathematical sense.
Yes, but it still makes sense to me. RL and 'llm thinking' are pretty basic heuristics at this point and I don't expect them to circumvent exponential running time requirements for many problems they implicitly solve at the same time.
Ok, but why do you think that AIs learn skills at a constant rate? Might it be that higher level skills need more time to learn because compute scales exponentially with time but for higher level skills data is exponentially more scarce and needs linearly in task length more context, that is, total data processed scales superexponentially with task level?
I think this is relevant: https://slatestarcodex.com/2017/07/24/targeting-meritocracy/ . 'Prevalence and tendency implies morality' - I don't think that's an argument that people gestured at here try to make.
Orange peel is a standard ingredient in Chinese cooking. Just be careful with pesticides.
Maybe this is related: A crucial step in the workflow of Getting Things Done is to clarify the task. Many of the tasks you mention are not clearly specified. I suppose morphing them into questions means that the task becomes to first clarify the task.