One reason is just that eating food is enjoyable. I limit the amount of food I eat to stay within a healthy range, but if I could increase that amount while staying healthy, I could enjoy that excess.
I think there are two aspects to the enjoyment of food. One is related to satiety. I enjoy the feeling of sating my appetite, and failing to sate it leaves me with te negative experience of craving food (negative if I don't satisfy those cravings.
But the other aspect is just the enjoyment of eating each individual bite of food. Not the ...
I think tailcalled's point here is an important one. You've got very different domains with very different dynamics, and it's not apriori obvious that the same general principle is involved in making all of these at first glance dangerous systems relatively safe. It's not even clear to me that they are safer than you'd expect. Of course that depends on how safe you'd expect them to be.
Many people have lost their money from crypto scams. Catastrophic nuclear war hasn't happened yet, but it seems like we may have had some close...
When I was trekking in Qinghai my guide suggested we do a hike around a lake on our last day on the way back to town. It was just a nice easy walk around the lake. But there were tibetan nomads (nomadic yak herders, he just referred to them as nomads) living on the shore of the lake, and each family had a lot of dogs (Tibetan Mastiffs as well as a smaller local dog they call "three eyed dogs"). Each time we got near their territory the pack would come out very aggressively.
He showed me how to first always have some stones ready, and secon...
I don't think you've highlighted the casual factor here. It's not at all clear that the reason bees and ants have a more effective response to predators than do flocks of birds is that the bees are individually less intelligent than the birds.
There's a very clear evolutionary/game theoretic explanation for the difference between birds and bees here: specifically the inclusive fitness of individual bees is tied to the outcome of the collective whereas the inclusive fitness of the birds is not.
In a game theoretic framework we might say that the payoff ...
I like the three suggested approaches instead of giving advice directly. All three seem like good ideas.
However, all three of your approaches seem like things that could still be done in combination with giving advice. "Before giving advice, try to fully understand the situation by asking questions" seems like a reasonable way to implement your first suggestion, for instance. Personal experiences can be used to give context for why you are giving the advice you are giving, and clearing up misconceptions can be an important first step befo...
Is there a reason you are thinking of to expect that transition to happen at exactly the tail end of the distribution of modern human intelligence? There don't seem, as far as I'm aware, to have been any similar transitions in the evolution of modern humans from our chimp-like ancestors. If you look at proxies, like stone tools from homo-habilis to modern humans you see very slow improvements that slowly, but exponentially, accelerate in the rate of development.
I suspect that most of that improvement, once cultural transition took off at...
Presumably it's outputting the thing that's right where GR is wrong, in which case you should be able to tell, at least in so much as it's consistent with GR in all the places that GR has been tested.
Maybe it outputs something that's just to hard to understand so you can't actually tell what it's predictions are, in which case you haven't learned anything from your test.
Additional to the effect of parental investment on the selection pressure favoring longer lives (and thus a lower rate of aging) in humans, is potentially the effect of grand-parental investment. If in humans grandparents have a large impact on the rate of survival and reproduction* in their grandchildren, then the selection pressure for survival gets pushed to even higher ages, potentially into the ~60's/70s. The importance of grandparents seems to be relatively unique to humans.
I've seen enough evidence (related to the grandmother hypothesis ...
One of the things related to food that I noticed reading the story was that you still need primary food production even in the world of the duplicator, since it duplicates the food exactly as it is, food will still go bad. The duplicate is just as old as the original. Sure, canned beans will last a while so you can keep duplicating them for years, probably, without concern, but if you buy a loaf of bread you will only be able to duplicate it and eat the product for the same length of time that it would usually take your bread to get moldy.
You d...
I feel like this post misses one of the most important ways in which a tradition stays alive, that is through contact with the world.
The knowledge in a tradition of knowledge is clearly about something, and the test of that knowledge is to bring it into contact with the thing it is about.
As an example, a tradition of knowledge about effective farming can stay alive without the institutions discussed in the post through the action of individual farmers. If a farmer has failed to correctly learn the knowledge of the tradition, he'll fail to efficiently...
I like the idea, and at least with current AI models I don't think there's anything to really worry about.
Some concerns people might have:
Given economic growth I'd expect current 20 year olds to on average be richer than current 80 year olds by the time they are 80. If that doesn't happen, something has probably gone wrong, unless it's because of something like "more people are living to 80 by spending money on healthcare during their 50's/60's/70s".
This reminds me of a bit from Feynman's Lectures on Physics:
"What is this law of gravitation? It is that every object in the universe attracts every other object with a force which for any two bodies is proportional to the mass of each and varies inversely as the square of the distance between them. This statement can be expressed mathematically by the equation F=Gmm'/r^2. If to this we add the fact that an object responds to a force by accelerating in the direction of the force by an amount that is inversely proportional to the mass of t...
"The average shareholder definitely does not care about the value of R&D to the firm long after their deaths, or I suspect any time at all after they sell the stock."
This was addressed in the post: the price of the stock today (when its being sold) is a prediction of its future value. Even if you only care about the price that you can sell it at today, that means that you care about at least the things that can lead to predictably greater value in the future, including R&D, because the person you're selling to cares about those things.
Also wo...
Spandrels certainly exist. But note the context of what X is in the quoted text:
"a chunk of complex purposeful functional circuitry X (e.g. an emotion)"
a chunk of complex purposeful functional circuitry cannot be a spandrel. There are edge cases that are perhaps hard to distinguish, but the complexity of a feature is a sign of its adaptiveness. Eyes can't be spandrels. The immune system isn't a spandrel. Even if we didn't understand what they do, the very complexity and fragility of these systems necessitates that they are ada...
The wealthy may benefit from the existence of low-skilled labour, but compared to what? Do they benefit more than they would from the existence of high-skilled labour?
Yes, they benefit from low skilled labour as compared to no labour at all, but high skilled labour, being more productive, is an even greater benefit. If it weren't, it couldn't demand a higher wage.
If "the wavefunction is real, but it is a function over potential configurations, only one of which is real." then you have the real configuration interacting with potential configurations. I don't see how you can say something isn't real (if only one of them is real then the others aren't) is interacting with something that is. If that "potential" part of the wave function can interact with the other parts of the wave function, then it's clearly real in every sense that the word "real" means anything at all.
I know they're just cartoons and I get the gist, but the graphs labelled "naive scenario" and "actual performance" are a little confusing.
The X axis seems to be measuring performance, with benchmarks like "high schooler" and "college student", but in that case, what's the Y axis? Is it the number of tasks that the model performs at that particular level? Something like that?
I think it would be helpful if you labeled the Y axis, even with just a vague label.
Re: the dark matter analogy. I think the analogy works well, but would just like to point out that even in theories where dark matter doesn't interact even with the weak force, and there is some other force that it does interact with that's analogous to electromagnetism, so it could bind together to form an earth-like planet, it still interacts with gravity, and if this earth-sized dark matter planet really did overlap with ours, we'd feel it's gravity and the earth would seem to be twice as massive as it is. Or, to state it slightly differentl...
Related to this topic, with a similar outlook but also more discussion of specific approaches going forward, is Vitalik's recent post on techno-optimism:
https://vitalik.eth.limo/general/2023/11/27/techno_optimism.html
There is a lot at the link, but just to give a sense of the message here's a quote:
"To me, the moral of the story is this. Often, it really is the case that version N of our civilization's technology causes a problem, and version N+1 fixes it. However, this does not happen automatically, and requires intentional human effort. The ozone layer i...
Thanks for a good comment. My oversimplified thought process was that a 10x increase in energy usage for the brain would equate to a ~2x increase in total energy usage. Since we're able to maintain that kind of energy use during exercise, and elite athletes can maintain that for many hours/day, it seems reasonable that the heart and other organs could maintain this kind of output.
However, the issue you bring up, of actually getting that much blood to the brain, evacuating waste products, doing the necessary metabolism there, and dealing w...
"Extrapolating the historic 10x fall in $/FLOP every 7.7 years for 372 years yields a 10^48x increase in the amount of compute that can be purchased for that much money (we recognize that this extrapolation goes past physical limits)."
If you are aware that this extrapolation goes past physical limits, why are you using it in your models? Why not use a model where compute plateaus after it reaches those physical limits? That seems more useful than a model that knowingly breaks the laws of physics.
I agree that it's plausible just from priors that ASI could find a way to eat the sun. The matter is there, and while it's strongly gravitationally bound in a way that's inconvenient, there's nothing physically impossible about getting it out of that arrangement into one that's more convenient to using fusion reactors or something.
But an analysis of how plausible the scenario is would certainly have made the post more valuable. There are plausible proposals for how to get the fuel present in the sun out such that it could be used more efficient... (read more)