All of jmh's Comments + Replies

Perhaps I'm off on this but wanted to just ask. How much of that term being a contradiction is driven by the lack of a good underlying model of something. I'll use the house temperature example here.

We have a very good model (by assumption here) of energy transmission from the exterior to the interior. We have a controller (thermostat) that only measures external features, say outside temp and maybe light and a time duration for when and how long to run either the air conditioner or heating. With a good model (and probably a good initialization at installa... (read more)

It's an interesting post and on some levels seems both correct and, to me at least, somewhat common sense.

Still I have a small tingle in the back of my head asking "is this magic really from intelligence or something else?" Or perhaps intelligence (perhaps not all that exceptional) and something else. It seems like in a number of the cases we're presented a somewhat narrow frame of the situation. If the magic is not highly correlated with, or better a function largely of, intelligence I wonder exactly how meaningful this is regarding ASI.

Perhaps this is a bit of mirror image but accepted even just being myself actually included not opening up about some things so feeling safe not being vulnerable. 

I'm not sure I'm understanding your claim here. I don't think you're suggesting a "if you build it they will come" type result from being in a dominant position. But I also don't quite get why being in a dominant position in one space has much (other that deep pockets) really impacts the success in doing things others are not doing -- or leading a group in doing things others are not leading that group. 

Probably not the type of reference you were thinking about regarding reprogramming and impact on aging issues but I suspect it's in the area you were thinking. I'm pretty sure it's been mentioned here on LW before in other posts/comments. Interesting idea but implementation is problematic to say the least -- but really hoping someone can figure it out.

Since a lot of this is way beyond my skill sets and knowledge, when you're looking at the dynamic interplay aspect, is that purely internal to the cell or do you also look at the extra-cellular "communications"? If so, are you familiar with the Conboy's plasma dilution experiments?

Yes, I agree. But actions that make people dislike you less may have no impact on people liking you or may have a positive impact (increase you likeability). How does one go about choosing which element from the action set to put in your model to produce the results you present?

I suppose a better way to put my take here is that you're presenting a limited/narrow model but the rhetoric implies some type of general model. I think that is a bit dangerous to people trying to both understand their social anxiety and do something if they are looking to change it (which seems a bit implied by the use of anxiety rather than just saying introverted).

I like the general thrust of the post here but not sure I agree with some of the assumptions and conclusions that seem to follow from assumptions. I think you're correct that there as some real asymmetries that get lost when confusing avoiding dis-like with getting more liked. But I think that comes from disliked and liked not being different ends of a common spectrum. I suspect they are somewhat orthogonal to one another. 

So the distribution you draw is imposed on a projection of the dislike vector and the like vector which can allow shifting from the insecure to the secure distribution producing no change in the like number in the tail, or arguably no change in the dislike number, with an increase in the like.

I would add best literature survey piece for the given subject area as another type of reference material people should submit.

My sympathies to you and your family. This is one of life's sadness that we all experience at some point. While I don't think this will help easy anything (what could I say to do so?) I have to say when may father died it was too soon -- or felt that way. He lived a long life, died from aging (at nearly 102) and lasted a few days after the pain got too much for him that morphine was regularly administered by the hospice nurse. It was time, and expected. When it happened it was still too soon. 

It was easier for me when my mother died about 6 months lat... (read more)

Perhaps a more general way to approach the question would be can one identify the existing  comparative advantages related to the wizard power related task to be performed.

If you only want parts which are easy to clearly describe, you can hire someone to CAD them up... or you could describe them to an LLM and have it emit files that you can convert to your CAD of choice.

Don't know if this is still the case but 15 years ago something very similar existed with CAD/CAM production. The CAD design could be passed through a processor to generate the G-code ... (read more)

The inference goes beyond the definition and claims, additionally, that in a case like this diagram, the agent will only take actions that "follow the arrows," changing the world state from less-preferred to more-preferred at each step.

Hmm. I'm not sure about that inference but might be due to absence of specific argument you're making that inference from.

The preference arrow to me is simply pointing in a direction. The path between the less preferred state to more preferred state is pretty much a black box at this point. So I think the inference is more l... (read more)

Perhaps it is more terminology as I didn't mean to suggest some existing tendency was merely coincidental. I suspect, and would agree with the points you make, that some underlying structures will produce that fewer king opportunities and more wizard opportunities world. 

I see a lot of complications in the interactions here related to the kings and wizards framing that can muddy things up. But was really just observing that these are not quiet as simple as choose wizard power over king power.  I think one finds more wizard opportunities (to be fo... (read more)

Do you have an indications that those without the clinical signs of depression (or at least doctor approved state) won't become acclimated to the drug in a way those that perhaps need it for a balanced state don't?

I suppose asking a bit differently here, what are the gears here that do the work and how well one might think they they match up with one's own system that is in place?

Interesting though about using it to improve one's performance rather than just as an antidepressant or aid to quit smoking. the wiki has some good info but interesting that it do... (read more)

Interesting distinction to draw out. I some what come away with a view that wizards get to play in the positive-sum game world while kings are stuck in a zero- to negative-sum game. I'm not completely sure that is a good modeling though might see the game of thrones hitting it's equilibrium at a lower number of kings than one might expect to occur with wizards (resources are not infinite so at some point wizards will also just be competing against one another for stuff -- and possible followers for their own parades).

So maybe the bit is that we will tend t... (read more)

Current AI, even after ChatGPT, earns roughly ten‑thousand dollars per H100‑GPU‑year.

What does that $10K number actually represent? An average across all AI? The marginal GPU earning across all AI? An estimate of either from one of the big AIs?

I always find the use of "X% of the vote" in US elections to make some general point about overall acceptability or representation of the general public problematic. I agree it's a true statement but leaves out the important aspect of turn out for the vote. 

I have to wonder if, particularly the USA, would not be quite as divided if all the reporting provided percentage of vote of the eligible voting population rather than the percentage of votes cast. I think there is a big problem with just ignoring the non-vote information the is present (or expecti... (read more)

Would I be incorrect in reading into the post that, at least looked at through the lens of geopolitical status and power, one might not view the post WWII international order than promoted a lot of investment in the developing countries economic and health was planting the seeds of the West's vision of the modern world order?

Or do you see the current state of things as occurring independently of relative population changes?

I mostly put this question through the same filter I do the question of Chinese vs. US hegemony/empire. China has a long history of empire and knows how to do it well. The political bureaucracy in China is well developed for preserving both itself and the empire (even within changes at the top/changes of dynasty). Culturally and socially the population seems to be well acclimated to being ruled rather than seeing government as the servant of the people (which I not quite the same as saying they are resigned to abusive totalitarianism, the empire has to be ... (read more)

That is pretty much what I'm trying to accomplish and want to try to increase the rate I am building the working vocabulary.

I do agree with both you and Vaughn. Reading should (very hard for me now) really help improving the recall once I can read and have a sufficient understanding of the statement and larger text. Texting is (I have been able to do some) good for me in that it tends to keep the exchange short and sentence structure more simple and short (which means I typically will have a reasonable grasp of the general meaning so can better infer what the unknown word or unrecalled word likely means.)

Literacy seems to make sense to me but I might be missing something in the post. Writing is language and language is communication so at least two sides.

As more people learned to read, they also learned to write, and written communications increases. However, even with modest literacy one can read a long sentence. Or can do that when it is written by a good/skilled writer. But being able to read does not really lead to writing skills in most cases I suspect.

As more people started communicating via writing (think things like schools and education expansion) the skill level of the average writer likely declined. That probably lead to training next generation writes to write in a more simple sentence structure.

There's a Korean expression that basicly seems to be "the look is right" or "the look fits" which seems in line with your comment. The same outfit, hat, shoes, glasses, jacket or even car for different people create a different image in other's heads. There is a different message getting sent.

So if the overall point for the post is about the signaling then I suspect it is very important to consider the device one chooses to send messages like this. In other words, yes breaking some social/cultural standards to make certain points is fine but thought needs ... (read more)

I'm reminded of the old Star Trek episode with the super humans that were found in cryosleep that then took over the Enterprise.

While I do agree that this could be one potential counter to AI (unless the relative speed things overwhelm) but also see a similar type of risk from the engineered humans. In that view, the program needs to be something that is widely implemented (which would also make it potentially a x-risk case itself) or we could easily find ourselves having created a ruler class that views ordinary humans as subhuman on not deserving of full... (read more)

I'm a bit conflicted on the subject of death penalty. I do agree with the view some solution is needed for incorrigible cases where you just don't want that person out in general society. But I honestly don't know if killing them versus imprisoning them for life is more or less humane. In terms of steelmanning the case I think one might explore this avenue. Which is the cruelest punishment?

But I would also say one needs to consider alternatives to either prison or death. Historically it was not unheard of to exile criminals to near impossible to escape locations -- Australia possibly being a best example. 

In some ways I think one can make that claim but in an important ways, to me, numbers don't really matter. In both you still see the role of government as an actor, doing things, rather than an institutional form that enables people to do things. I think the US Constitution is a good example of that type of thinking. It defines the powers the government is suppose to have, limiting what actions it can and cannot take. 

I'm wondering what scope might exist for removing government (and the bureaucracy that performs the work/actions) from our social and p... (read more)

Did the Ask Question type post go away? I don't see it any more. So I will ask here since it certainly is not worthy of a post (I have no good input or thoughts or even approaches to make some sense of it). Somewhat prompting the question was the report today about MS just revealing it's first quantum chip, and the recent news about Google's advancement in its quantum program (a month or two back).

Two branches of technology have been seen as game, or at least potential game changers: AI/AGI and quantum computing. The former often a topic here and certainly... (read more)

Thanks. It was an interesting view. Certainly taking advantage of modern technologies and, taken at face value, seem to have resulted in some positive results. Has me thinking of making a visit just to talk with some of the people to see get some first hand accounts and views just how much that is changing the views and "experience" of government (meaning people experience as they live under a government).

I particularly liked the idea of government kind of fading into the background and being generally invisible. I think in many ways people see markets in ... (read more)

Does anyone here ever think to themselves, or out loud, "Here I am in the 21st Century. Sure, all the old scifi stories told me I'd have a shiny flying car but I'm really more interested in where my 21st Century government is?"

For me that is premised on the view that pretty much all existing governments are based on theory and structures that date at least back to the 18th Century in the West. The East might say they "modernized" a bit with the move from dynasties (China, Korea, Japan) to democratic forms but when I look at the way those governments and po... (read more)

With regards to thinking about what comes next, you might find these two links, if you didn't already come across them, of some interest.

https://www.atlanticcouncil.org/content-series/atlantic-council-strategy-paper-series/three-worlds-in-2035/ hypothesizes 3 global futures for 2035.

https://www.atlanticcouncil.org/content-series/atlantic-council-strategy-paper-series/welcome-to-2035/ offers results from a survey about various outcomes of states that might obtain in (by???) 2035. I didn't find much surprising here but some of the questions I had not g... (read more)

I could be way off on this, but I cannot help but core here is less about complexity than it is about efficiency. The most efficient processes do all appear to be a bit simpler than they probably are. It's a bit like watching an every talented craftsman working and thinking "The looks easy." Then when you try you find out it was much more difficult and complicated than it appeared. The craftsman's efficiency in action (ability to handle/deal with the underlying complexity) masked the truth a bit.

I've had similar experiences where my intuition tells me to be cautious but I could not say why. When I've ignored those intuitions I've generally paid a price. So now I do give them consideration. 

In such situations it is probably good to take some time to sit back and try to identify some of the things that triggered the response. We are very good at pattern matching but also really good at filtering. Could be that intuitions like "getting bad vibes" is all about the interaction of the two.

But that is a pretty difficult task, we're asking our self to go back and review all the details we ignored and filtered. But I suspect it is a very good thing to try doing.

The desires-values schema reminds me of Hirshmann's The Passions and the Interests and the problem he (IIRC) sees Machiavelli dealing with. Machiavelli is advising kings who he sees as more driven by their passions but to be successful need to be driven more by their interests.

Anyone in the alignment space take a look at Machiavelli in the light of how to get some thing that is more powerful (king versus the advisor) better aligned? Probably need to stretch things a bit to see that type of alignment as being aligned with the general welfare of the kingdom but seems like he was dealing with a similar problem when looked at from certain angles.

I like the point about the need for some type of external competitive measure but as you say, they might not be a MMA gym where you need one.

Shifting the metaphor, I think your observation of the sucker punch fits well with the insight that for those with only a hammer, all problems look like nails. The gym would be someone with a screwdriver or riveter as well as the hammer. But even lacking the external check, we should always ask ourselves "Is this really a nail?" I might only have a hammer but if this isn't a nail while the results might be better than... (read more)

I agree with the view that punishment is not really a great deterrent as many crimes are not committed from a calculated cost-benefit perspective. I do think we need to apply that type of thinking towards what we might do with that insight/fact of things.

On that point, would like to see more on your claim that we would get better bang for the buck as it were from more investment in preventing crimes. In this regard I'm thinking about the contrast between western legal views and places like China as well as the estimates on the marginal pecuniary costs of p... (read more)

Agree. There is that old saying about even fools learning from their own mistakes but wise men learning from the mistakes of others. But if everyone is trying to hide their mistakes, that might limit how much learning the wise can do.

I had not really thought about this before, but after seeing your comment the question struck me if social/cultural norms about social status and "loosing face" don't impact scientific advancement.

Nice write up and putting some light on something I think I have intuitively been doing but not quite realizing it. Particularly the impact on growth of wealth. 

I was thinking that a big challenge for a lot of people is the estimated distribution - which is likely why so many non-technical rationales are given by many people. Trying to assess that is hard and requires a lot of information about a lot of things -- something the insurance companies can do (as suggested by another comment) but probably overwhelms most people who buy insurance.

With that t... (read more)

What is the price of the past? Kind of leading question but I've found myself wondering at times about the old saying about those who don't know the past are doom to repeat it. 

It's not that I don't think there is a good point to that view. However, when I look at the world around me I often see something that is vastly different from that view. I've come to summarize that as those who cannot let go of the past will never escape it. The implication is that not only those "clingy" people but also those around them will continue living whatever past it ... (read more)

Yes, all those conjectures are possible as we don't yet know what the reality will be -- it is currently all conjecture.

The counter argument to yours I think is just what opportunities is the AI giving up to do whatever humans might be left to do? What is the marginal value of all the things this ASI might be able to be doing that we cannot yet even conceive of? 

I think the suggestion of a negative value is just out of scope here as it doesn't fit into theory of comparative advantage. That was kind of the point of the OP. It is fine to say comparative... (read more)

I'm not sure that is the correct take in the context of Comparative Advantage.

It would not matter if the SI could produce more than humans in a direct comparison but what the opportunity cost for the SI might be. If the ASI is shifting efforts that would have produced more value to it than it gets from the $77 sunlight output AND that delta in value is greater than the lower productivity of the humans then the trade makes sense to the ASI.

Seems to me the questions here are about resource constraints and whether or not an ASI does or does not need to confront them in a meaningful way.

You're touching on one of the questions that occurred to me. What do the current and post-Jones transportation flows look like? While I agree that the law must shift some from shipping to truck, rail or pipeline I'm not sure I would expect massive changes here. Do you have some data on that point?

I think one clear aspect of the stories here, yours and John's, relates to what I'll call asymmetric information flows. Basically, the times at which the information, that no one is trying to keep secret, become known to the relevant parties.

Of course understanding what a good update frequency is for various situations should be is a tricky thing itself.

If I'm reading this correctly, then generally we're seeing a rather flat payoff curve over most "do good opportunities" and the rare max should stand out like a sore thumb when taking a good look. So those really should be things do-gooders will jump on quickly. (Note, that doesn't mean they are done quickly or that additional assistance is not important.) 

While not as obvious, it probably also means that a lot of more mundane opportunities are getting ignored. That comes from an insight offered in one of my classes from years back asking why so much ... (read more)

One point I'm not sure about with the idea of neutrality is neutrality of process or of outcome. Or would that distinction not matter to your interests here?

Interesting but I've just skim so will need to come back. With that caveat made, I seem to have had a couple of thought that keep recurring for me that seem compatible or complementary with your thoughts.

First, where do we define the margin between public and private. It strikes me that a fair amount of social strife does revolve around a tension here. We live in a dynamic world so thinking that the sphere of private actions will remain static seem unlikely but as the world changed (knowledge, applied knowledge driving technology change, movement of people... (read more)

Years ago when I was hanging out with day traders there was a heuristic they all seemed to hold. If their trading model was producing winning trades two out of three times they thought the model was good and could be used. No one ever suggested why that particular rate was the shared meme/norm -- why not 4 out of 5 or 3 out of 5. I wonder if empirically (or just intuitively over time) they simply approximated the results in this post.

Or maybe just a coincidence, but generally when money is at stake I think the common practices will tend to reflect some fundamental fact of the environment. 

Could you clarify a bit here. Is Hanson talking about specific cultures or all of the instances of culture?

Thanks that does help clarify the challenges for me.

I was just scrolling through Metaculus and its predictions for the US Elections. I noticed that pretty much every case was a conditional If Trump wins/If doesn't win. Had two thought about the estimates for these. All seem to suggest the outcomes are worse under Trump. But that assessment of the outcome being worse is certainly subject to my own biases, values and preferences. (For example, for US voters is it really a bad outcome if the probability of China attacking Taiwan increases under Trump? I think so but other may well see the costs necessary to re... (read more)

Had something of a similar reaction but the note about far-UV not having the same problems as other UV serilization (i.e., also harmful to humans) I gather the point is about locality. UV in ducks will kill viri in the air system. But the spread of an airborn illness goes host-to-target before it passed through the air system.

As such seems that while the in-duct UV solution would help limit spread, it's not going to do much to clean the air in the room while people are in it exhailing, coughing or sneezing, talking.... 

I suspect it does little to prot... (read more)

Quick comment regarding research.

If far-UV is really so great, and not that simple, I would assume that any company that would be selling and installing might not be some small Mom and Pop type operation. If that holds, why are the companies that want to promote and sell the systems using them and then collecting the data?

Or is would that type of investment be seen as too costly even for those with a direct interest in producing the results to bolster sales and increase the size of the network/ecosystem?

I think perhaps a first one might be:

On what evidence do I conclude what I think is know is correct/factual/true and how strong is that evidence? To what extent have I verified that view and just how extensively should I verify the evidence?

After that might be a similar approach to the implications or outcomes of applying actions based on what one holds as truth/fact.

I tend to think of rationality as a process rather than endpoint. Which isn't to say that the destination is not important but clearly without the journey the destination is just a thought or dream. That first of a thousand steps thing.