All of jmh's Comments + Replies

There's a Korean expression that basicly seems to be "the look is right" or "the look fits" which seems in line with your comment. The same outfit, hat, shoes, glasses, jacket or even car for different people create a different image in other's heads. There is a different message getting sent.

So if the overall point for the post is about the signaling then I suspect it is very important to consider the device one chooses to send messages like this. In other words, yes breaking some social/cultural standards to make certain points is fine but thought needs ... (read more)

I'm reminded of the old Star Trek episode with the super humans that were found in cryosleep that then took over the Enterprise.

While I do agree that this could be one potential counter to AI (unless the relative speed things overwhelm) but also see a similar type of risk from the engineered humans. In that view, the program needs to be something that is widely implemented (which would also make it potentially a x-risk case itself) or we could easily find ourselves having created a ruler class that views ordinary humans as subhuman on not deserving of full... (read more)

I'm a bit conflicted on the subject of death penalty. I do agree with the view some solution is needed for incorrigible cases where you just don't want that person out in general society. But I honestly don't know if killing them versus imprisoning them for life is more or less humane. In terms of steelmanning the case I think one might explore this avenue. Which is the cruelest punishment?

But I would also say one needs to consider alternatives to either prison or death. Historically it was not unheard of to exile criminals to near impossible to escape locations -- Australia possibly being a best example. 

In some ways I think one can make that claim but in an important ways, to me, numbers don't really matter. In both you still see the role of government as an actor, doing things, rather than an institutional form that enables people to do things. I think the US Constitution is a good example of that type of thinking. It defines the powers the government is suppose to have, limiting what actions it can and cannot take. 

I'm wondering what scope might exist for removing government (and the bureaucracy that performs the work/actions) from our social and p... (read more)

Did the Ask Question type post go away? I don't see it any more. So I will ask here since it certainly is not worthy of a post (I have no good input or thoughts or even approaches to make some sense of it). Somewhat prompting the question was the report today about MS just revealing it's first quantum chip, and the recent news about Google's advancement in its quantum program (a month or two back).

Two branches of technology have been seen as game, or at least potential game changers: AI/AGI and quantum computing. The former often a topic here and certainly... (read more)

Thanks. It was an interesting view. Certainly taking advantage of modern technologies and, taken at face value, seem to have resulted in some positive results. Has me thinking of making a visit just to talk with some of the people to see get some first hand accounts and views just how much that is changing the views and "experience" of government (meaning people experience as they live under a government).

I particularly liked the idea of government kind of fading into the background and being generally invisible. I think in many ways people see markets in ... (read more)

Does anyone here ever think to themselves, or out loud, "Here I am in the 21st Century. Sure, all the old scifi stories told me I'd have a shiny flying car but I'm really more interested in where my 21st Century government is?"

For me that is premised on the view that pretty much all existing governments are based on theory and structures that date at least back to the 18th Century in the West. The East might say they "modernized" a bit with the move from dynasties (China, Korea, Japan) to democratic forms but when I look at the way those governments and po... (read more)

With regards to thinking about what comes next, you might find these two links, if you didn't already come across them, of some interest.

https://www.atlanticcouncil.org/content-series/atlantic-council-strategy-paper-series/three-worlds-in-2035/ hypothesizes 3 global futures for 2035.

https://www.atlanticcouncil.org/content-series/atlantic-council-strategy-paper-series/welcome-to-2035/ offers results from a survey about various outcomes of states that might obtain in (by???) 2035. I didn't find much surprising here but some of the questions I had not g... (read more)

I could be way off on this, but I cannot help but core here is less about complexity than it is about efficiency. The most efficient processes do all appear to be a bit simpler than they probably are. It's a bit like watching an every talented craftsman working and thinking "The looks easy." Then when you try you find out it was much more difficult and complicated than it appeared. The craftsman's efficiency in action (ability to handle/deal with the underlying complexity) masked the truth a bit.

I've had similar experiences where my intuition tells me to be cautious but I could not say why. When I've ignored those intuitions I've generally paid a price. So now I do give them consideration. 

In such situations it is probably good to take some time to sit back and try to identify some of the things that triggered the response. We are very good at pattern matching but also really good at filtering. Could be that intuitions like "getting bad vibes" is all about the interaction of the two.

But that is a pretty difficult task, we're asking our self to go back and review all the details we ignored and filtered. But I suspect it is a very good thing to try doing.

The desires-values schema reminds me of Hirshmann's The Passions and the Interests and the problem he (IIRC) sees Machiavelli dealing with. Machiavelli is advising kings who he sees as more driven by their passions but to be successful need to be driven more by their interests.

Anyone in the alignment space take a look at Machiavelli in the light of how to get some thing that is more powerful (king versus the advisor) better aligned? Probably need to stretch things a bit to see that type of alignment as being aligned with the general welfare of the kingdom but seems like he was dealing with a similar problem when looked at from certain angles.

I like the point about the need for some type of external competitive measure but as you say, they might not be a MMA gym where you need one.

Shifting the metaphor, I think your observation of the sucker punch fits well with the insight that for those with only a hammer, all problems look like nails. The gym would be someone with a screwdriver or riveter as well as the hammer. But even lacking the external check, we should always ask ourselves "Is this really a nail?" I might only have a hammer but if this isn't a nail while the results might be better than... (read more)

I agree with the view that punishment is not really a great deterrent as many crimes are not committed from a calculated cost-benefit perspective. I do think we need to apply that type of thinking towards what we might do with that insight/fact of things.

On that point, would like to see more on your claim that we would get better bang for the buck as it were from more investment in preventing crimes. In this regard I'm thinking about the contrast between western legal views and places like China as well as the estimates on the marginal pecuniary costs of p... (read more)

Agree. There is that old saying about even fools learning from their own mistakes but wise men learning from the mistakes of others. But if everyone is trying to hide their mistakes, that might limit how much learning the wise can do.

I had not really thought about this before, but after seeing your comment the question struck me if social/cultural norms about social status and "loosing face" don't impact scientific advancement.

Nice write up and putting some light on something I think I have intuitively been doing but not quite realizing it. Particularly the impact on growth of wealth. 

I was thinking that a big challenge for a lot of people is the estimated distribution - which is likely why so many non-technical rationales are given by many people. Trying to assess that is hard and requires a lot of information about a lot of things -- something the insurance companies can do (as suggested by another comment) but probably overwhelms most people who buy insurance.

With that t... (read more)

What is the price of the past? Kind of leading question but I've found myself wondering at times about the old saying about those who don't know the past are doom to repeat it. 

It's not that I don't think there is a good point to that view. However, when I look at the world around me I often see something that is vastly different from that view. I've come to summarize that as those who cannot let go of the past will never escape it. The implication is that not only those "clingy" people but also those around them will continue living whatever past it ... (read more)

Yes, all those conjectures are possible as we don't yet know what the reality will be -- it is currently all conjecture.

The counter argument to yours I think is just what opportunities is the AI giving up to do whatever humans might be left to do? What is the marginal value of all the things this ASI might be able to be doing that we cannot yet even conceive of? 

I think the suggestion of a negative value is just out of scope here as it doesn't fit into theory of comparative advantage. That was kind of the point of the OP. It is fine to say comparative... (read more)

I'm not sure that is the correct take in the context of Comparative Advantage.

It would not matter if the SI could produce more than humans in a direct comparison but what the opportunity cost for the SI might be. If the ASI is shifting efforts that would have produced more value to it than it gets from the $77 sunlight output AND that delta in value is greater than the lower productivity of the humans then the trade makes sense to the ASI.

Seems to me the questions here are about resource constraints and whether or not an ASI does or does not need to confront them in a meaningful way.

You're touching on one of the questions that occurred to me. What do the current and post-Jones transportation flows look like? While I agree that the law must shift some from shipping to truck, rail or pipeline I'm not sure I would expect massive changes here. Do you have some data on that point?

I think one clear aspect of the stories here, yours and John's, relates to what I'll call asymmetric information flows. Basically, the times at which the information, that no one is trying to keep secret, become known to the relevant parties.

Of course understanding what a good update frequency is for various situations should be is a tricky thing itself.

If I'm reading this correctly, then generally we're seeing a rather flat payoff curve over most "do good opportunities" and the rare max should stand out like a sore thumb when taking a good look. So those really should be things do-gooders will jump on quickly. (Note, that doesn't mean they are done quickly or that additional assistance is not important.) 

While not as obvious, it probably also means that a lot of more mundane opportunities are getting ignored. That comes from an insight offered in one of my classes from years back asking why so much ... (read more)

One point I'm not sure about with the idea of neutrality is neutrality of process or of outcome. Or would that distinction not matter to your interests here?

Interesting but I've just skim so will need to come back. With that caveat made, I seem to have had a couple of thought that keep recurring for me that seem compatible or complementary with your thoughts.

First, where do we define the margin between public and private. It strikes me that a fair amount of social strife does revolve around a tension here. We live in a dynamic world so thinking that the sphere of private actions will remain static seem unlikely but as the world changed (knowledge, applied knowledge driving technology change, movement of people... (read more)

Years ago when I was hanging out with day traders there was a heuristic they all seemed to hold. If their trading model was producing winning trades two out of three times they thought the model was good and could be used. No one ever suggested why that particular rate was the shared meme/norm -- why not 4 out of 5 or 3 out of 5. I wonder if empirically (or just intuitively over time) they simply approximated the results in this post.

Or maybe just a coincidence, but generally when money is at stake I think the common practices will tend to reflect some fundamental fact of the environment. 

Could you clarify a bit here. Is Hanson talking about specific cultures or all of the instances of culture?

Thanks that does help clarify the challenges for me.

I was just scrolling through Metaculus and its predictions for the US Elections. I noticed that pretty much every case was a conditional If Trump wins/If doesn't win. Had two thought about the estimates for these. All seem to suggest the outcomes are worse under Trump. But that assessment of the outcome being worse is certainly subject to my own biases, values and preferences. (For example, for US voters is it really a bad outcome if the probability of China attacking Taiwan increases under Trump? I think so but other may well see the costs necessary to re... (read more)

Had something of a similar reaction but the note about far-UV not having the same problems as other UV serilization (i.e., also harmful to humans) I gather the point is about locality. UV in ducks will kill viri in the air system. But the spread of an airborn illness goes host-to-target before it passed through the air system.

As such seems that while the in-duct UV solution would help limit spread, it's not going to do much to clean the air in the room while people are in it exhailing, coughing or sneezing, talking.... 

I suspect it does little to prot... (read more)

Quick comment regarding research.

If far-UV is really so great, and not that simple, I would assume that any company that would be selling and installing might not be some small Mom and Pop type operation. If that holds, why are the companies that want to promote and sell the systems using them and then collecting the data?

Or is would that type of investment be seen as too costly even for those with a direct interest in producing the results to bolster sales and increase the size of the network/ecosystem?

I think perhaps a first one might be:

On what evidence do I conclude what I think is know is correct/factual/true and how strong is that evidence? To what extent have I verified that view and just how extensively should I verify the evidence?

After that might be a similar approach to the implications or outcomes of applying actions based on what one holds as truth/fact.

I tend to think of rationality as a process rather than endpoint. Which isn't to say that the destination is not important but clearly without the journey the destination is just a thought or dream. That first of a thousand steps thing.

What happens when Bob can be found in or out of the set of bald things at different times or in different situations, but we might not understand (or even be well aware) of the conditions that drive Bob's membership in the set when we're evaluating baldness and Bob?

Can membership in baldness turn out to be some type of quantum state thing?

That might be a basis for separating the concept of fuzzy language and fuzzy truth.But I would agree that if we can identify all possible cases where Bob is or is not in the set of baldness one might claim truth is no longer fuzzy but one needs to then prove that knowledge of all possible states has been established I think.

I really like the observation in your Further Thoughts point. I do think that is a problem people need to look at as I would guess many will view the government involvement from a acting in public interests view rather than acting in either self interest (as problematic as that migh be when the players keep changing) or from a special interest/public choice perspective.

Probably some great historical analysis already written about events in the past that might serve as indicators of the pros and cons here. Any historians in the group here?

Strong upvote based on the first sentence. I often wonder why people think an ASI/AGI will want anything that humans do or even see the same things that biological life sees as resources. But it seems like under the covers of many arguments here that is largely assumed true.

I am a bit confused on point 2. Other than trading or doing it your selfs what other ways are you thinking about getting something?

That is certainly a more directly related, non-obvious aspect for verification. Thanks.

I assumed John was pointing at verifying that perhaps the chemicals used in the production of the chair might have some really bad impact on the environmnet, start causing a problem with the food chain eco system and make food much scarcers for everyone -- including the person who bought the chair -- in the meaningfully near future. Something a long those lines. 

As you note, verifying the chair functions as you want -- as a place to sit that is comfortable -- is pretty easy. Most of us probably do that without even really thinking about it. But will t... (read more)

In terms of the hard to verify aspect, while it's true that any one person will face any number of challenges do we live in a world where one person does anything on their own?

How would the open-source model influence outcomes? When pretty much anyone can take a look, and persumable many do, does the level of verifcation, or ease of verification, improve in your model?

Kind of speculative on my part and nothing I've tried to research for the comment. I am wondering is the tort version of reasonableness is a good model for new, poorly understood technologies. Somewhat thinking about the picture in https://www.lesswrong.com/posts/CZQYP7BBY4r9bdxtY/the-best-lay-argument-is-not-a-simple-english-yud-essay distinguising between narrow AI and AGI.

Tort law reasonableness seems okay for narrow AI. I am not so sure about the AGI setting though. 

So I wonder if a stronger liability model would not be better until we have a good... (read more)

I find this rather exciting -- and clearly the cryonics implications are positive. But beyond that, and yes, this is really scifi down the road thinking here, the implications for education/learning and treatment of things like PTSD seems huge. Assuming we can figure out how to control these. Of course I'm ignoring some of the real down sides like manipulation of memory for bad reasons or an Orwellean application. I am not sure those types of risks at that large in most open societies.

Thanks. Just took a quick glance as the abstract but looks interesting. Will have something to read while waiting at the airport for a flight tomorrow.

Is that thought one that is generally shared for those working in the field of memory or more something that is new/cutting edge? It's a very interesting statement so if you have some pointers to a (not too difficult) a paper on how that works, or just had the time to write something up, I for one would be interested and greatful.

Actually checking those hypotheses statistically would be a pretty involved project; subtle details of accounting tend to end up relevant to this sort of thing, and the causality checks are nontrivial. But it's the sort of thing economists have tools to test.

Yes, it would be a challenge statistically, and measurment a challenge as well. It's not really about subtle accounting details but the economic costs -- opportunity costs, subjective costs, expected costs. Additionally, economics has been trying to explain the existance, size and nature of the f... (read more)

First, I have to note this is way more than I can wrap my head around in one reading (in fact it was more than I could read in one sitting so really have not completed reading it) but thank you for posting this as it presents a very complicated subject in a framework I find more accessible that prior discussings here (or anywhere else I've looked at). But then I'm just a curious outsider to this issue who occasionally explores the discussion so information overload is normal I think.

I particularly like the chart and how it laid out the various states/outcomes.

I think it would be more correct to say that is a part of the literature related to the theory of the firm. The theory of the firm covers a lot of ground and in some ways various branches have somewhat challenging relationships with their internal logic and approaches.

I don't find this as convincing as others for a number or reasons. Caveat: I did a rather shallow read of the post and have not done deep thinking about the resonse below.  

First, most of the managers I've worked with, and how I was as a manager, don't act like the dominance seekers you're describing. Not a claim that it doesn't exist, just that in my personal experience it doesn't seem to be something that seems to have been a big driver within the companies. 

Second, I think the assumption of economic inefficiency exists therefore these big comp... (read more)

And then we also have the whole moral hazzard problem with those types of incentives. Could I put myself at a little risk of some AI damages that might be claimed to have much broader potential?

That touches on a view I've been holding for a while now. One often hears the phrase, those that forget the past are doomed to repeat it (or close to that). But it struck me one that that many seem to hold on to the past, never letting it go and so dooming themselves and everyone else to continued living in that past. When we're never getting past the injustices of the past we keep them in the present and keep living them. I think this might be part of why we see many of the existing conflicts in the world -- from the racial issues in the USA, the wars and... (read more)

Still reading and thanks for the write up. Much better than I could do myself and have been thinking it's time to revisit and see where things stand.

But think this is an obvious type so wanted to mention it for your edit. "In other words, if your biological age is lower than your biological age, you’re doing great." I assum you mean lower than your chronoloical age there.

So was farther along than I thought. Quick question on the reprogramming aspect. Certainly tissue complexity is a problem when the reaction rates are different and we probably really need ... (read more)

I do agree with your point but think you are creating a bit of a strawman here. I think the OP goal was to present situations in which we need to consider AI liability and two of those situations would be where Coasean barganing is possible and where it fails do the the (relatively) Judgement Proof actor. I'd also note that legal trends have tended to be to always look for the entity with the deepest pockets that you have some chance of blaming.

So while the example of the gun is a really poor case to apply Coase for I'm not sure that really detracts from t... (read more)