I'm not sure that is the correct take in the context of Comparative Advantage.
It would not matter if the SI could produce more than humans in a direct comparison but what the opportunity cost for the SI might be. If the ASI is shifting efforts that would have produced more value to it than it gets from the $77 sunlight output AND that delta in value is greater than the lower productivity of the humans then the trade makes sense to the ASI.
Seems to me the questions here are about resource constraints and whether or not an ASI does or does not need to confront them in a meaningful way.
You're touching on one of the questions that occurred to me. What do the current and post-Jones transportation flows look like? While I agree that the law must shift some from shipping to truck, rail or pipeline I'm not sure I would expect massive changes here. Do you have some data on that point?
I think one clear aspect of the stories here, yours and John's, relates to what I'll call asymmetric information flows. Basically, the times at which the information, that no one is trying to keep secret, become known to the relevant parties.
Of course understanding what a good update frequency is for various situations should be is a tricky thing itself.
If I'm reading this correctly, then generally we're seeing a rather flat payoff curve over most "do good opportunities" and the rare max should stand out like a sore thumb when taking a good look. So those really should be things do-gooders will jump on quickly. (Note, that doesn't mean they are done quickly or that additional assistance is not important.)
While not as obvious, it probably also means that a lot of more mundane opportunities are getting ignored. That comes from an insight offered in one of my classes from years back asking why so much clumping (think fad type stuff here) exists when the marginal utility of the consumed good is pretty much equal to all the other goods that could have been consumer. In other words, when the opportunity cost is zero why is everyone doing the same thing?
I suspect we could see something like that in the "do good" space. Therefore, taking the path not followed could be a very good thing.
One point I'm not sure about with the idea of neutrality is neutrality of process or of outcome. Or would that distinction not matter to your interests here?
Interesting but I've just skim so will need to come back. With that caveat made, I seem to have had a couple of thought that keep recurring for me that seem compatible or complementary with your thoughts.
First, where do we define the margin between public and private. It strikes me that a fair amount of social strife does revolve around a tension here. We live in a dynamic world so thinking that the sphere of private actions will remain static seem unlikely but as the world changed (knowledge, applied knowledge driving technology change, movement of people resulting in cultural transmission and tensions...) will be forces resulting in a change in the line between public and private.
While I'm not entirely sure it is the best framing, I do think of this in the form of externalities. Negative externalities are the more challenging form. What I think starts happening is that we live in t=0 and some set of private activities are producing very little negative impacts on others. But we find by t=10 some of the elements in that set of private activities are now producing a large enough total negative external effect that:
At some point either most accept that a new definition of "private" exists and the old ways have changed or society reached the point that those who have not adapted will be treated as criminal and removed from society.
The other thing I've been thinking about is related. One hears the where's my flying car, it's the 21st Century already quip now and then. But I think a better one might be: It's the 21st Century, why am I still living under and 18th Century form of government?
I think these relate some of your post in that a lot of the social conflict you point to is driven by the shifting margin between public and private sphere of action. As that margin shifts people use the government to address those new conflicts within the society. But few if any governments differ substantially from those that have existed for centuries. I would characterize that vision of government, even when thinking of representative democracies, as that of an actor/agent. Government takes actions, just like the private members of society do. It should function, as you say, in a neutral way. Part of the failing there comes from government, being an actor/agent, then has its own interests, agendas and biases.
That government as an active participant contrasts a bit with how I think most people think of markets. Markets don't really do anything. They are simply an environment in which active entities come and interact with each other. Markets don't set price or quality or even really type of item -- these are all unplanned outputs. The market itself is indifferent to all those, it's neutral in the sense you use that term.
Well, in the 21st Century might we not think that how governments are structured might also shift? While I am far from sure that the shift would be correctly called divestiture or privatization (which seems most people think of when talking about fixing government -- or for some calling for increasing what its already doing) I do think the shift might be away from an acting entity and more into some type of passive environment that has some commonality with markets. In a very real sense governments are already a type of market setting but not a price/money exchange one (the representatives are not quite but out bids and offers on votes) but clearly these is an demand mediation and supply process going on. But currently the market-like aspect of government is about integrating voter/members of society demands and then the government makes a decision and takes the actions it wants. I would think some areas might be suitable for taking out the government being the actor and let the actions be decentralized among the people. Probably not individual action, I suspect some sub-agent presence will exist to reduce organizational/transaction costs but certainly the process would look more market-like and be a more neural setting. That might well then remove a lot of the divisiveness and conflict we see with the existing "old school" forms of government.
That is all probably a bit poorly written and expressed but it's a quick dump of a couple of not fully thought out ideas.
Years ago when I was hanging out with day traders there was a heuristic they all seemed to hold. If their trading model was producing winning trades two out of three times they thought the model was good and could be used. No one ever suggested why that particular rate was the shared meme/norm -- why not 4 out of 5 or 3 out of 5. I wonder if empirically (or just intuitively over time) they simply approximated the results in this post.
Or maybe just a coincidence, but generally when money is at stake I think the common practices will tend to reflect some fundamental fact of the environment.
Could you clarify a bit here. Is Hanson talking about specific cultures or all of the instances of culture?
Thanks that does help clarify the challenges for me.
Yes, all those conjectures are possible as we don't yet know what the reality will be -- it is currently all conjecture.
The counter argument to yours I think is just what opportunities is the AI giving up to do whatever humans might be left to do? What is the marginal value of all the things this ASI might be able to be doing that we cannot yet even conceive of?
I think the suggestion of a negative value is just out of scope here as it doesn't fit into theory of comparative advantage. That was kind of the point of the OP. It is fine to say comparative advantage will not apply but we lack any proof of that and have plenty of examples where it actually does hold even when there is a clear absolute advantage for one side. Trying to reject the proposition by assuming it away seems a weak argument.