There is another sense in which I would not want to say that there is any particular hierarchy between natural/unnatural/rational constraints.
I think there's a lot to unpack here. I'm going to give it a preliminary go, anticipating that it's likely to be to a bit all over the place. The main thread I want to pull is what it means to impose a particular hierarchy between the constraints, and then see how this leads to many possible hierarchies in such a way that it feels like no particular hierarchy is privileged.
From a "natural" point of view, ...
Motivation 1 ('organisms-as-agents thesis') "says that organisms really do exhibit some or even all of the attributes of agency". Motivation 2 ('organism-as-agents heuristic') "says that it can be heuristically useful to treat organisms as if they were agents for certain intellectual purposes".
It's interesting that both motivations appear to be about modelling organisms as agents, as opposed to any other level of organisation. This feels like it misses some of the most interesting insights we might get from biological agency, namely those around agen...
"Social" is slightly too coarse-grained a tag. The thing we're actually interested in is "whether successfully predicting the behaviour of other members of its own species is a strong selection pressure". Social collaboration is one way this happens - another seems to be "deception" arms races (such as corvids stealing and hiding things from each other), or specific kinds of mating rituals. It also depends on the relative strength of other selection pressures - in most cases highly intelligent creatures also seem to have developed a "slack" in resources th...
Clem here - I was fellowship lead this year and have been a research affiliate and mentor for PIBBSS in the past. Thanks for posting this. As might be expected in my position, I'm much more bullish than you / most people on what is often called "blue sky" research. Breakthroughs in the our fundamental understanding of agency, minds, learning etc. seem valuable in a range of scenarios, not just in world dominated by an "intelligence explosion". In particular, I think that this kind of work (a) benefits a huge amount from close engagement with empirica... (read more)