Posts

Sorted by New

Wiki Contributions

Comments

I just can't get over that it's Game Stop. I remember when Blockbuster finally crashed in January 2010, I was convinced Game Stop was next and I shorted them then. Their valuation did decline a tiny bit and I made a small amount of money. 11 years later, I can't believe this company still exists. 

Helping someone establish a dictatorship is still a high cost action that I think requires being more persuasive than convincing someone to do their job without decisively proving you're actually their boss. 

The distinction in this specific case here is between intelligence and persuasiveness. To the extent that some elements of persuasiveness are inherently embodied, as in people are more likely to trust you if you're also a person, that is at best orthogonal to intelligence.

More generally, "effectiveness" as some general purpose quality of agents that can do things is limited by the ability to acquire and process information, but also by the ability to act on it. You may know that being tall makes you more likely to be elected to office, but if you can't make yourself any taller, you can't use the information to make your campaign more likely to succeed. 

As a more fantastical but maybe more relevant example, people often mention something like turning the moon into comptronium. Part of doing that is knowing how to do it. But we already know how to do it. We understand at the level of fusion and fission how to transmute elements into different elements, and we understand, given some elements that act as semiconductors, how to produce general-purpose computational processors. The actual reason we can't do it, aside from not wanting to disrupt the earth's orbit and potentially end human civilization, is (1) there is inherent propagation delay in moving material from wherever it is created to wherever it needs to be used and this delay is much greater when the distances to move are greater than planet-scale, (2) machines that can actually transmute rocks to silicon don't presently exist and there is non-zero manufacturing delay in creating them, and (3) we have no means of harnessing sufficient energy to actually transmute matter at the necessary scale. 

Can gaining more information solve these problems? Maybe. There might exist unknown physics that enable easier or faster methods than we presently know of, but there is non-zero propagation delay in creation of new knowledge of physics as well. You have to conduct experiments. At high-energy, sub-particle scale, these have become extremely expensive and time consuming. AI threat analysis tends to get around this one by proposing they can just simulate physics to such perfect fidelity that experimentation is no longer necessary, but this seem question-begging because you need to already know rules of physics that haven't been discovered yet to be able to do this. 

While presumably a collection of brains better than human brains can figure out a way to make this happen faster, maybe even decades rather than centuries faster, "foom" type analyses that claim the ability to recursively rewrite one's own source code better than the original coder means it will happen in days or even hours come across more as mysticism than real risk analysis. 

To expand, I actually think it applies much more to AI than to animals. Part of the advantage of being an animal is our interface to the rest of the world is extremely flexible regarding the kinds of inputs it can accept and outputs it can produce. Software systems often crash because xml doesn't specify whether you can include whitespace in a message or not. Part of why AlphaGo isn't really "intelligent" isn't anything about the intrinsic limitations of what types of functions its network architecture can potentially learn and represent. It isn't intelligent because it can't even accept an input that isn't a very specific encoding of a Go board and can't produce any outputs except moves in a game of Go. 

It's isn't like a dog and more like a dog that can only eat one specific flavor of one specific brand of dog food. Much of the practical difficulty in creating general purpose software systems is just that there is no general purpose communication protocol. It's why we have succeeded so far in producing things that can accept and produces images and text, because they analogize well to how animals communicate with the rest of the world, so we understand them and can create digital encodings of them. But even those still rely upon character set encodings, pixel metadata specifications, and video codecs that themselves have no ability to learn or adapt. 

Although this is probably true in general, it degrades when trying to get people to do something extremely high-cost like destroy all of humanity. You either need to be very persuasive or trick them about the cost. It's hard to get people to join ISIS knowing they're joining ISIS. It's a lot easier to get them to click on ransomware that can be used to fund ISIS.

In order to qualify as a non-profit, a foundation needs to have decisions made by a board, not a single individual. 

I'm not sure to what extent this also plays in to vaccine production specifically, but the requirement for being a foundation at all is you need to give 5% of your endowment annually to charitable causes. If vaccine production is not being carried out by qualifying 501(c)(3) non-profits, then any money you give them doesn't count toward that requirement. 

Someone who actually knows something about taxonomic phylogeny of neural traits would need to say for sure, but the fact that many species share neural traits doesn't necessarily mean those traits evolved many times independently as flight did. They could have inherited the traits from a common ancestor. I have no idea if anyone has any clue whether "data efficient learning" falls into the came from a single common ancestor or evolved independently in many disconnected trees categories. It is not a trait that leaves fossil evidence. 

It might be instructive to consider some specific examples. An internal combustion engine is particularly easy to analyze, for instance. The efficiency with which fuel is turned to kinetic energy is almost entirely determined by temperature gradient between the engine and its surroundings. This means that, apart from better cooling technologies, the only way to make an engine more efficient is to make it less powerful. In practice, we also make entire vehicles more fuel efficient by making them lighter and more aerodynamic. 

But the "goal" of an engine in many cases is just to produce the most power. If you're trying to win a race, get into orbit, you're limited by how much thrust or torque you can generate per unit of mass and time. This goal is directly antithetical to efficiency. 

While I don't think this generalizes to the extent that efficiency as a goal is always antithetical to whatever you're really trying to optimize, it is at least in most cases orthogonal to whatever you're actually trying to optimize. 

Answer by DTXJan 27, 202150

I can't point to any single good canonical example, but this definitely comes up from time to time in comment threads. There's the whole issue that computers can't act in the world at all unless they're physically connected to hardware controllers that can interface with some physical system we actually care about being broken or misused. Usually, the workaround there is AI will be so persuasive that they can just get people with bodies to do the dirty work that requires being able to actually touch stuff in order to repurpose manufacturing plants or whatever it is we're worried they might do. 

That does seem like there is a missing step in there somewhere. I don't think the bottleneck right now to building out a terrorist organization is that the recruiters aren't smart enough, but AI threat tends to just use "intelligence" as a shorthand for good at literally anything.

Strangely enough, actual AI doomsday fiction doesn't seem to do this. Usually, the rogue AI directly controls military hardware to begin with, or in a case like Ex Machina, Eva is able to manipulate people at least in part because she is able to convincingly take the form of an attractive embodied woman. A sufficiently advanced AI could presumably figure out that being an attractive woman helps, but if the technology to create convincing artificial bodies doesn't exist, you can't use it. This tends to get handwaved away by assuming sufficiently advanced AI can invent whatever nonexistent technology they need from scratch. 

I'd expect a Pareto distribution for charitable donations, not log-normal, and that's exactly what the histogram looks like:

Looks like alpha >> 2, so the variance is infinite.

Load More