Besides its core claim, I think one reading of the post can be that you need to understand what you want before giving it to a powerful optimizer - not AI in this case, but society.
I've been hinting at similar conclusions for a while, and people have written books about how you can't live on spread alone for 1000s of years. You can try to incorporate spirituality in science, but I predict that you will be unsuccessful. The more scientific somebody is, the less they tend to care about everything spiritual. They might ask you to prove it to them, or tell you that Jung was a hack, or downplay psychology because it's not a "hard science" and therefore not real. I even find that some people consider the death of humanity as good thing as less humanity means less errors. I've also seen some people who don't understand the problem even when it's explained to them, they can't tell that something is missing. By objective metrics, things are "improving", and the high rates of mental illness (which should tell them that something is seriously wrong) just seem like another problem that they can "solve with science". "their chemical balance is out of whack, is all!". Allegedly, their lives are good so they should be happy and if they're not happy then their brains are wrong and we should fix them)
I don't think science is a good framework for non-scientific things. If you wrap spirituality in science, you kill whatever substance you had by reducing it to something mundane and mechanical. Like if you were to describe love in terms of chemical reactions and conclude that it's what love "really is". And it frankly wouldn't matter even if you were correct, as what you need for well-being isn't correctness but belief in substance or whatever the opposite of disillusionment and nihilism is.
What you seek is joy, fulfillment and wisdom, so why not aim at that directly? Using science to fix the problems that science caused feels a bit like putting out a fire using fire. Let me also warn you that meta-science is worse than science. The more degrees of separation to reality, the worse you're off mentally.
Mistaking a metric for what it represents is a form of separation, not living in the moment is a form of separation, living in the map instead of the territory is a form of separation, being objective or attempting to be an observer is a form of separation. Meta-science is two layers of disconnect. Speaking of which, I belive that the tendency to model how other people model oneself has gotten much worse lately. It's too many pointers (simulacra perhaps?)
1Great points - thanks for your thoughts on this! 2 questions:
1) Do you think it may be better to "wrap science in spirituality" instead? Or should we just leave them segregated as they are today?
2) My suggestion here was that we adjust what science is so that it no longer creates the problems you are pointing at. Specifically perhaps we can relax the "3rd person objective observer" paradigm and give more weight to 1st person perspective as well. I do believe that science can be quite spiritual and can generate well-being when it's driven by genuine wonder, curiosity and intention to make life more wonderful. Does this fit with your thoughts?
My reply turned out to be a bit long. Perhaps you can jsut skim it for the aspects you care about?
I think wrapping science in spirituality can work, but for people in this community, it's probably tempting to think in objective, well-defined tokens rather than thinking in the concepts and subjective tokens which align with how the brain works and with ones own values.
But the rules of the mind and the rules of mathematics are entirely different, and ones "objective" quality of life matters much less for well-being than their subjective worldview does, which is partly why we're not really getting any happier.
If you try walking, and thinking consciously about every movement you make, you will probably find that walking becomes much harder. Your mind also has its own symbolic language which is much more efficient than mathematics for many things, and science is sufficently inhuman that it's destructive not just to human errors, but to human nature in general.
My suggestion here was that we adjust what science is so that it no longer creates the problems you are pointing at
You could partly do that by correctly stating that everything is relative, and thus that an absolutist worldview might not be ideal. But moving further than that is difficult as people have an almost religious view of science. They think that the subjective doesn't matter much, that things are only worth something if you can prove them, that science can discriminate between good and evil or morality and immorality.
People also seem to either reject reality, or desperately attempt to construct a morally correct hypothesis which explains the pattern they see in reality, and then make up excuses as the hypothesis repeatedly fails to predict the future. The idea that people seek the truth is a lie, they can only be objective about things that they don't care about too much, which is why controversies repeatedly form around things connected to politics and morality. People who don't realize this don't even have a basic understanding of themselves (or other people, or humanity in general) which is the actual cause of our problems.
science can be quite spiritual and can generate well-being when it's driven by genuine wonder, curiosity and intention to make life more wonderful
It can when you put the latter first, so that science becomes second. That feeling of wonder literally requires a lack of complete understanding. The people who enjoy science the most are those who know it the least, and they will become disillusioned until they once again meet something that they don't understand, which causes an explosion in possibilities bigger than what you can wrap your mind around. "The first gulp from the glass of natural sciences will make you an atheist, but at the bottom of the glass God is waiting for you", the latter part is only true because we don't understand quantum science. It's our good luck that we didn't manage to "solve" science.
And again, we have to watch out for Goodhart's law. If you "improve" peoples lives in a way which makes them miserable, then it's not an improvement, even if the metrics state otherwise. Another reason I dislike scientific views is that I think it results in Moloch. the only winning move is not to play, and the only way to avoid playing is if the optimal solution remains unknown. Psychologically speaking, having full information about something seems really undesirable. Possibly because you do the mental equivalence of when the state reduces human lives to numbers on a spreadsheet.
I'm autistic so thinking objectively has always been easy for me, I appear "better" at scientific thought than most, which is why I'm so conscious of all the pitfalls one can run into. Anyway, I think the many psychological issues appearing in society are directly connected to the domination of scientific thought and the death of religion, and thus that you're trying to solve the problem with the tools which caused it, and that you only consider this a good idea because you feel something wonderful in science... Which actually exists in yourself (or in your relation to science). The whole "beauty in mathematics" is, I think, the brains reaction to symmetry and harmonious patterns, making it beautiful in the same way that music is beautiful. Of course, a lot of things can be made possible through science, so it's not incorrect to perceive a lot of hidden value waiting to be discovered. It just has to be for the sake of humanity instead of at the cost of humanity.
But everywhere in society I see a hatred of humanity and attempts of destroying it. Usually in order to make humanity "better" or "more moral", which translates into destroying aspects of human nature or replacing them with less human ones. An easy example which is not too controversial is destroying "laziness" and making people into efficient workers. Which also hints at the fact that the optimization of "productivity" and the optimization of "humanity" go in two different directions, meaning that we'll start our own darwinistic process of destroying human aspects (as the genetic "fitness" values inhuman/objective things). Which is ironic as the purpose of technology is improving human life, rather than to, say, replace it.
2Thanks for this - and sorry I missed it earlier. I marked a couple of your statements that especially hit me.
Overall great analysis - I'm only be a bit more positive about these things myself. Mainly:
- I do think science is a tool that may be qualitatively different than all other tools we have - just looking at the impact it managed to have on the world. And in so far as it is "special," it's worth considering whether it can solve our problem
- I don't think it's ever possible to completely understand the world - and so there will always be "God at the bottom of the glass" - no matter how much we do solve. I also don't think there is anything bad about completely understanding some aspect of the world - our wonder just transitions to another aspect. Planetary motion no longer creates wonder, we take it for granted, and use it as a workhorse on which we build other things that do generate wonder (like "why are the laws what they are?"). The things we completely understand we just stop noticing really - but we start noticing other things. Driving a car becomes mundane once you're good at it - but the road and places you can visit become exciting.
That's alright! And I'm happy some of it resonated with you.
While science seems to solve a lot of problems, I think it creates new problems as it solves the old ones, and that most of the problems were trying to solve now wouldn't be as bad if they weren't amplified by technology. I think science will always both solve and create problems, and that this is unavoidable because science itself is unbiased and free-for-all, and that they problems will get worse since technology is a power-amplifier (by the way, as technology allows for stronger tools over time, society needs more laws and restrictions in order to keep people from being able to harm one another, and if you try to design a video game in which players progress like this you will notice that no player is going to enjoy the end-game)
I also don't think we should give science the entire credit. The knowledge of humanity is mainly improved by few, extremely intelligent people (Einstein, Newton, Tesla, Hawking, and so on). Even before the scientific method, a few highly influential people accounted for most changes in the world. Most peoples education consist of studying other peoples discoveries and theories, with the hope that they can reach a level where they provide more benefit than harm. This almost makes common people sound superficial, but that's also because science is getting harder to use. 200 years ago you'd likely be alright if you could operate a shovel and a wheelbarrow, whereas you're at a disadvantage today if you can't make it through college.
It is impossible to understand the world. At best you can make a mental model which can predict it because, in some sense, your internal model is a bisimulation (I'm not sure if that's the right term, but the idea should be close). We refine theories so that they predict the world with less and less error. But the human brain already does this by its own, and intelligence isn't actually required, nor is understanding. It's just trial and error, you try things, and if their outcome is good, you keep them, and otherwise you get rid of them. But isn't this how darwinism works? And how cultures and traditions originate? But they don't need to know why something works, only that it does. We don't give tradition much credit because traditional explanations are irrational, but as far as pure results go I think traditions do quite well. Most arguments against tradition aren't rational but rather moralistic, and society doesn't seem aware of this, but morality and truth are at great conflict.
Besides the psychological consequences of understanding something (a kind of disillusionment), I think Moloch might be the greater danger. Moloch seems to me a result of legibility, and glorifying science makes people think that legibility (and order) are good-in-themselves, i.e. something where more is always better. This is not the case. I didn't yet know this when I wrote my previous comment, but the issue is known as "high modernity". Nassim Taleb has written about how forcing orderliness on society is dangerous, and it's my personal belief that a lack of legibility is what holds back Moloch. The game theoretical collapse of society is only happening now, in the modern age, since the modern age is what has made it possible, by creating enough order and simplicity (or given the illusion of these) that we now have enough information available that the dilemmas are visible. And now that they're visible you're either required to join them or to put yourself at a disadvantage by not joining them. I believe that online social media is unhealthy, whereas real-life socialization is often healthy, because the latter is more chaotic (literally) and has less visible metrics that one could start optimizing for.
To generalize, most undesirable mechanics in life are caused by excess legibility, and by the belief that something singular is good and ought to be optimized for. Rationality, legibility, morality, equality, happiness... Whatever metric we choose, the outcome will be terrible. The alignment problem in AI should perhaps teach us that focusing on anything singular is bad, i.e. that balance is key. But Taoists knew this more than 2500 years ago. I've often been told that religious people are just stupid and that the category of people who speak of Heaven/Nature/GNON and warn against "playing god" don't include hyper intelligent people, but from my limited experience with rationalism and science, this is wrong.
Of course, this community will largely disagree with me, since it believes that society is clearly improving while I believe it's clearly getting worse.
I don't think science is a good framework for non-scientific things. If you wrap spirituality in science, you kill whatever substance you had by reducing it to something mundane and mechanical.
I find it somewhat difficult to understand exactly what you mean here and in the rest of the comment. Could you maybe define the terms "science", "spirituality" and "non-scientific things" as you are using them here?
What you seek is joy, fulfillment and wisdom, so why not aim at that directly? Using science to fix the problems that science caused feels a bit like putting out a fire using fire. Let me also warn you that meta-science is worse than science. The more degrees of separation to reality, the worse you're off mentally.
Are you recommending here that people should not use science in their attempts to pursue joy, fulfillment and wisdom?
And when you say "The more degrees of separation to reality, ...", what is the thing that you are talking about that is being separated from reality?
Could you maybe define the terms
I think that science is a way of thinking and a way of doing things, as well as a way of encoding something else. If I were to describe a beautiful painting just in writing alone, then I would have encoded the image in text, reducing something visual to something linguistic. But in that process, the image would be completely destroyed, so this "encoding" is impossible. Likewise, if I define science, the definition would be a pointer at best, and not science itself.
My understanding of spirituality is something like psychological well-being and tacit knowledge and wisdom. This is rather vague perhaps, but communicating wisdom is inherently difficult, and the text is only, at best, a pointer. If you replace your intuition of substance (like your personal experience of love) with your scientific understanding (that love is 'merely' chemical reactions), your brain might lose faith in its own experience, regard it as empty, or destroy the subjective value associated with it.
What I'm describing here is how the mind relate to its own sensory inputs, ideas and associations, which are all more fundamental to your brain than your conscious beliefs (for instance, even if you believe that a phobia of yours is irrational, your mind may still believe it).
Are you recommending here that people should not use science
I think science can be used to the extent that it can help, but that people often consider science to be universal, or attempt to "encode" everything as science so that they feel at home working with the problem (when the only tool you have is a hammer... ). If your aim is well-being, then well-being should have a higher priority than science. Too often, I see that people are unhappy, unfulfilled, and unwise because their perspective on life is too objective, too logical, and too rigid. These characteristics apply to logic and math and scientific thought, but not to human life. You may miss this if you think that math and logic are more fundamental than life, or that things are only "real" if they can be encoded and reasoned with.
what is the thing that you are talking about that is being separated from reality?
Your perception. If you see a cat with your own eyes, that's a direct experience. At first it's raw sensory input, then your brain categorizes it as "a cat", which is an idea. If you later remember that you saw a cat, then your memory is a pointer to the experience you had. We seem to use more and more pointers, and forget that they're pointers. For instance, you may think that science is good in itself, but it's only good because we can use it for things which we consider valuable, so it's these things that we're actually aiming at, and not science. (I do realize that this is no longer a disconnect with your perception, but the distance between concepts and ideas, thought of kind of like a graph of nodes and edges, in which distance can be defined)
Another example of a disconnect is when people say "lying is bad" instead of "the consequences of lying are bad" or when they say "X is bad because it's illegal" instead of "X is illegal because it's bad". These may be obvious enough, but our culture definitely have a lot of shorthands which people don't realize are shorthands, and a lot of derived ideas that people mistake for the root that they stem from. I believe that Simulacra Levels and their Interactions may be a consequence of this
We optimize for GDP, and forget about actual well-being and psychological happiness of the population.
Who do you think optimizes for GDP and how would they act differently if they wouldn't optimize for GDP?
While politicians do care about GDP, it's not the central metric to which they optimize. Politicians are likely better understand as goodharting on winning elections and even that they don't really (see Dominic Cummings for how politicians fail to optimize for winning elections).
The FED has a dual mandate of caring about the employment rate and stable prices. Eliezer wrote about how it would be better if they would actually focus on NGDP but they don't.
Saying that any Western government is goodharting GDP, requires a lot of ignorance of what's actually going on.
1
[cross-posted from my blog https://pchvykov.com/blog]
I see a problem with current science. It’s not the reproducibility crisis, nor the toxic work culture, nor the misaligned incentive. But in another sense, it is all of these – or perhaps the root cause behind them. It’s hard to name it exactly, but in a way, it’s the dissociation between the romanticism of a selfless “quest for truth,” and the career-success incentives that run academia. In another way, it’s Goodhart’s law (optimizing for citation counts rather than for meaningful progress). In yet another way, it may be an issue with the scientific method itself (or at least how it’s taught). But I think the overarching issue is with trying to remove our humanity from the scientific process.
Let’s begin with something concrete – Goodhart’s law (“when a metric becomes a target, it ceases to be a good metric”). This is a major (perhaps the major) problem of the western social order, which relies heavily on metrics. We optimize for GDP, and forget about actual well-being and psychological happiness of the population. We optimize for profits, and forget about the social value we are creating. We optimize for grades, and forget about learning. We optimize for longevity, and forget about meaning. And so in academia, we optimize for citations, and forget about progress, about building a better world. Goodhart’s law comes up because the things we actually value cannot be accurately encoded into metrics – they will always be slightly misaligned. People’s ingenuity will then always find a way a way to leverage this misalignment to game the metric. But metrics and reproducible measurements are at the core of the scientific method, thereby seeding the problem (it's easier to write a paper showing some high metric score than one that actually matters).
At the same time, upon reflection we somehow have an intuitive understanding of whether something is or isn’t a “good” metric, or whether it achieved some “desired” outcomes. So what metric inside us tells us what’s good? For this I like the dialectic of “measuring” vs “sensing” – the former being about finding the value of a pre-defined metric, and the latter about pattern-recognition. Perhaps one can say that in these terms, science is more about measuring, while art is about sensing? Either way, measuring has a linear quality to it – it’s the “forward path” of the scientific method. In contrast, sensing is inherently a feedback loop – where question informs the answer, which in turn changes the question, and so on. Such dynamics may mitigate Goodhart’s law as the metric (i.e., question) here is never static, but continuously updated and refined in response to the outcomes. The problem I’m referring to is insufficient amount of this feedback in science and its application.
But I think we must take it deeper than this. We must include not just the question, but the observer themselves in this feedback loop. To see this, we need to ask ourselves honestly why do we study the questions that we do? The real reason is often circumstantial – “my PhD adviser worked on this,” or “there was a job available for this research,” or “this topic sells well.” But if we go beyond all this and honestly ask ourselves which questions are really important? Which questions are worth our time, our effort, and money? Which questions really make meaningful progress towards a better world? Such sincere inquiry quickly makes us realize that the answers are far from obvious, are very subjective, and are highly sensitive to personal and cultural values, traumas, fears and hopes. And these are the foundation of our science. Thus, we cannot pretend to be the “objective observers,” standing outside the scientific method – we are part of it. And as such, we must have the humility to ourselves become subjects to it. On the one hand, thus goes back to the willingness to update our beliefs about the world, our behavior, our personality, our sense of self even, in response to new evidence – which is already hard enough. But on the other, it also means asking the questions that actually matter to us, that actually have the potential to change our lives (cf. active learning in ML). I think this is the only way to really do “honest science.”
And once we come to this, we basically come to spirituality (in some idealized sense). If science is the study of the external, then spirituality is the study of the internal (-Carl Jung, Carl Sagan, Fritjof Capra, etc.). But if we allow our science to be guided by the quest for personal transformation, for greater joy, and for a better world, then the distinction begins to blur. I believe that this integration of science with spirituality, with our humanity, with our inner goals and aspirations, and paradoxically, with our subjectivity, is the only way to overcome Goodhart’s law and get fulfilling outcomes. The notion of a separate objective observer in the scientific method is an impossible idealization – and therefore misleading, leaving much of academia to study incremental technical minutiae that has little relevance to our lives. Even the relevant technological achievements often end up divorced from generating greater well-being. It is no accident that many of the early scientists were motivated by their faith – they studied the external to better understand the internal. I find that the schism between the scientific and the spiritual that emerged since has been unproductive for both domains, just as any hard disciplinary boundaries lead to siloed inefficient work.
So what do we do? Well, I’m not really sure. The rationalism movement may be on the right track here. Personally I think it would be fun to build a research institute that really focuses on enabling this feedback between research results and our personal inner values. I’ve been quite interested in Eastern Philosophy and mindfulness practices for a while – and these might give a good approach to really help learn from and internalize our scientific insights. On the other hand, complexity science (my research field) may be a good framework to research all these questions more systematically, perhaps in the context of “science of science.” The key, I think, is to integrate the theory and the practice here – to “walk our talk.” I would love to see science become a joyful and deep practice that leads its adepts to ever-greater personal fulfillment and wisdom, not mere knowledge. Scientists serve the role of shamans in modernity, and so their wisdom and personal attainment (or lack thereof) spreads to the rest of the human tribe.