= 27d567bc2d48d9f40e9ccc20ea370b15)
jacob_cannell comments on An overall schema for the friendly AI problems: self-referential convergence criteria - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
Comments (110)
This. Values evolve, like everything else. Evolution will continue in the posthuman era.
Evolution requires selection pressure. The failures have to die out. What will provide the selection pressure in the posthuman era?
Economics. Posthumans still require mass/energy to store/compute their thoughts.
I'm not sure that's true. Imagine some glorious postbiological future in which people (or animals or ideas or whatever) can reproduce without limit. There are two competing replicators A and B, and the only difference is that A replicates slightly faster than B. After a while there will be vastly more of A around than of B, even if nothing dies. For many purposes, that might be enough.
So, in this scenario, what evolved?
The distribution of A and B in the population.
I don't think this is an appropriate use of the word "evolution".
Why not? It's a standard one in the biological context. E.g.,
which according to a talk.origins FAQ is from this textbook: Helena Curtis and N. Sue Barnes, Biology, 5th ed. 1989 Worth Publishers, p.974
If there are mistakes made or the environment requires adaptation, a sufficiently flexible intelligence can mediate the selection pressure.
The end result still has to be for the failures to die or be castrated.
There is no problem with saying that values in future will "change" or "drift", but "evolve" is more specific and I'm not sure how will it work.
Memetic evolution, not genetic.
I understand that. Memes can die or be castrated, too :-/
In your earlier comment you said "evolution requires selection pressure". There is of course selection pressure in memetic evolution. Completely eliminating memetic selection pressure is not even wrong - because memetic selection is closely connected to learning or knowledge creation. You can't get rid of it.
"Evolve" has (at least) two meanings. One is the Darwinian one where heritable variation and selection lead to (typically) ever-better-adapted entities. But "evolve" can also just mean "vary gradually". It could be that values aren't (or wouldn't be, in a posthuman era) subject to anything much like biological evolution; but they still might vary. (In biological terms, I suppose that would be neutral drift.)
Well, we are talking about the Darwinian meaning, aren't we? "Vary gradually", aka "drift" is not contentious at all.
I'm not sure we are talking specifically about the Darwinian meaning, actually. Well, I guess you are, given your comment above! But I don't think the rest of the discussion was so specific. Kaj_Sotala said:
which seems to me to describe a situation of gradual change in our values that doesn't need to be driven by anything much like biological evolution. (E.g., it could happen because each generation's people constantly make small more-or-less-deliberate adjustments in their values to suit the environment they find themselves in.)
(Kaj's comment does actually describe a resource-constrained situation, but the resource constraints aren't directly driving the evolution of values he describes.)
We're descending into nit-pickery. The question of whether values will change in the future is a silly one, as the answer "Yes" is obvious. The question of whether values will evolve in the Darwinian sense in the posthuman era (with its presumed lack of scarcity, etc.) is considerably more interesting.
I agree that it's more interesting. But I'm not sure it was the question actually under discussion.