Where does our community disagree about meaningful issues?
Yesterday at our LW Berlin Dojo we talked about areas where we disagree. We got 4 issues:
1) AI risk is important
2) Everybody should be vegan.
3) It's good to make being an aspiring rationalist part of your identity.
4) Being conscious of privacy is important
Can you think of other meaningful issues where you think our community disagrees? At best issues that actually matter for our day to day decisions?
How to be skeptical
Community
The Center For Applied Rationality (CFAR) checklist is a heuristic for assessing the admissibility of one's own testimony.
What of the challenge of evaluating the testimony of others?
Slapping the label of a bias on a situation?
Arguing at the object level by provision of evidence to the contrary?
This risks Gish Gallop. For those who prefer to pick their battles, I commisioned this post of my time, a structural intervention into the information ecosystem.
We need not event the wheel, for legal theorists have researched this issue for years, while practitioners and courts have identified heuristics useful to lay people interested in this field.
Precedent
The Daubert standard provides a rule of evidence regarding the admissibility of expert witnesses' testimony during United States federal legal proceedings. Pursuant to this standard, a party may raise a Daubert motion, which is a special case of motion in limine raised before or during trial to exclude the presentation of unqualified evidence to the jury. The Daubert trilogy refers to the three United States Supreme Court cases that articulated the Daubert standard:
-https://en.wikipedia.org/wiki/Daubert_standard
Further reading on the case is available here on Google Scholar
Practice
How can this be applied in practice?
What is the first principle of skepticism. It's effectively synonymous: 'question'
What question? This isn't the 5 W's of primary school, after all.
I have summarized critical questions to a reading here to get the ball rolling:
Issues to consider when contesting and evaluating expert opinion evidence
A. Relevance (on the voir dire)
I accept that you are highly qualified and have extensive experience, but how do we know that your level of performance regarding . . . [the task at hand — eg, voice comparison] is actually better than that of a lay person (or the jury)?
What independent evidence... [such as published studies of your technique and its accuracy] can you direct us to that would allow us to answer this question?
What independent evidence confirms that your technique works?
Do you participate in a blind proficiency testing program?
Given that you undertake blind proficiency exercises, are these exercises also given to lay persons to determine if there are significant differences in results, such that your asserted expertise can be supported?
B. Validation
Do you accept that techniques should be validated?
Can you direct us to specific studies that have validated the technique that you used?
What precisely did these studies assess (and is the technique being used in the same way in this case)?
Have you ever had your ability formally tested in conditions where the correct answer was known? (ie, not a previous investigation or trial)
Might different analysts using your technique produce different answers?
Has there been any variation in the result on any of the validation or proficiency tests you know of or participated in?
Can you direct us to the written standard or protocol used in your analysis?
Was it followed?
C. Limitations and errors
Could you explain the limitations of this technique?
Can you tell us about the error rate or potential sources of error associated with this technique?
Can you point to specific studies that provide an error rate or an estimation of an error rate for your technique?
How did you select what to examine?
Were there any differences observed when making your comparison . . . [eg, between two fingerprints], but which you ultimately discounted? On what basis were these discounted?
Could there be differences between the samples that you are unable to observe?
Might someone using the same technique come to a different conclusion?
Might someone using a different technique come to a different conclusion?
Did any of your colleagues disagree with you?
Did any express concerns about the quality of the sample, the results, or your interpretation?
Would some analysts be unwilling to analyse this sample (or produce such a confident opinion)?
...
D Personal proficiency
...
Have you ever had your own ability... [doing the specific task/using the technique] tested in conditions where the correct answer was known?
If not, how can we be confident that you are proficient?
If so, can you provide independent empirical evidence of your performance?
E Expressions of opinion
...
Can you explain how you selected the terminology used to express your opinion? Is it based on a scale or some calculation?
If so, how was the expression selected?
Would others analyzing the same material produce similar conclusions, and a similar strength of opinion? How do you know?
Is the use of this terminology derived from validation studies?
Did you report all of your results?
You would accept that forensic science results should generally be expressed in non-absolute terms?
More
For further reading, I recommend the seminal text in cross-examination which is the 1903 The Art of Cross Examination.
The Full Text is available free here on Project Gutenberg.
Other countries use different standards, such as the Opinion Rule in Australia.
This year's biggest scientific achievements
For our solstice event I tried to put together a list of this year’s biggest scientific achievements. They can likely all be looked up with a bit of searching and each one is worthy of a celebration in their own right. But mostly I want to say; we have come a long way this year. And we have a long way to go.
I tried to include science and technology in this list, but really anything world-scale (non-politics or natural disaster) is worthy of celebrating.
- Rosetta mission lands on a comet
- using young blood to fight old age (rats)
- kinghorn human sequencing machines (Sydney relevant)
- 100,000 genomes project
- the world's oldest cave art @ 40,000 years old
- tesla battery//released their patents on their electric engines for use by anyone.
- Virtual reality (cardboard)
- Astronauts growing their own food
- Self driving cars
- cubesats
- Lab grown kidneys successfully implanted into animals
- synthetic DNA
- Chicken with a reptile face
- nearly an altzeimers cure (ultrasound techniques)
- DAWN orbits Ceres
- Deepdreaming machine learning (and twitch-deepdream)
- Prosthetic limbs that transmit feeling back to the user
- Autonomous rocket landing pointy end up
- Lightsail project
- Ion space travel engine
- Anti - aging virus injected into the patient 0
- Super black substance made
- Q-carbon
- High temperature superconductor (-70c)
- 23&me were allowed to open back up
- Enchroma colourblindness adjusting glasses
- Google releases "Tensor Flow" which whilst its not very good at the moment has the potential to centralize the Deep Learning libraries.
- CRISPR's ability to change the germ line.
- Deep Dreaming, but also image generation. Faces generated, bedrooms generated and even a toilet in a field. Its clear that within the next few years you will have pictures entirely generated by Neural Nets. (Code: https://github.com/soumith/dcgan.torch).
- On the NLP side of deep learning this post, which whilst not using new techniques, sparked a lot of generative work (http://karpathy.github.io/2015/05/21/rnn-effectiveness/). There has also been really interesting work on Question Answering (http://arxiv.org/abs/1506.02075)
- Quasipolynomial time algorithm for graph isomorphism (http://jeremykun.com/2015/11/12/a-quasipolynomial-time-algorithm-for-graph-isomorphism-the-details/)
from https://en.wikipedia.org/wiki/2015
April 29 – The World Health Organization (WHO) declares that rubella has been eradicated from the Americas.
July 14 - NASA's New Horizons spacecraft performs a close flyby of Pluto, becoming the first spacecraft in history to visit the distant world.
September 10 – Scientists announce the discovery of Homo naledi, a previously unknown species of early human in South Africa.
September 28 – NASA announces that liquid water has been found on Mars.
Recommendations from the slack:
china makes a genetically modified micropig and sells it: http://www.theguardian.com/world/2015/oct/03/micropig-animal-rights-genetics-china-pets-outrage
psyc studies can’t be reproduced: http://www.theverge.com/2015/8/27/9216565/psychology-studies-reproducability-issues
zoom contact lenses
http://mic.com/articles/118670/this-painless-eye-implant-could-give-you-superhuman-vision#.4S5ihAKNE
room temperature synthetic diamonds
http://phys.org/news/2015-11-phase-carbon-diamond-room-temperature.html
Notable deaths
terry pratchett passed away
malcolm fraser
John Forbes Nash Jr
Oliver Sacks
Christopher lee
Nobel medals this year
Chemistry – Paul L. Modrich; Aziz Sancar and Tomas Lindahl ("for mechanistic studies of DNA repair")
Economics – Angus Deaton ("for his analysis of consumption, poverty, and welfare")
Literature – Svetlana Alexievich ("for her polyphonic writings, a monument to suffering and courage in our time" )
Peace – Tunisian National Dialogue Quartet ("for its decisive contribution to the building of a pluralistic democracy in Tunisia in the wake of the Jasmine Revolution of 2011")
Physics – Takaaki Kajita and Arthur B. McDonald ("for the discovery of neutrino oscillations, which shows that neutrinos have mass")
Physiology or Medicine – William C Campbell, Satoshi Ōmura ("for their discoveries concerning a novel therapy against infections caused by roundworm parasites") and Tu Youyou ("for her discoveries concerning a novel therapy against Malaria"[116])
Other:
The dress
Ebola outbreak
Polio came back
(also this year) - upcoming spaceX return flight on the 19th dec
runner up: vat meat is almost ready.
runner up: soylent got a lot better this year
runner up: quantum computing having progressive developments but nothing specific
Things that happened 100 years ago (from wikipedia):
- March 19 – Pluto is photographed for the first time
- September 11 – The Pennsylvania Railroad begins electrified commuter rail service between Paoli and Philadelphia, using overhead AC trolley wires for power. This type of system is later used in long-distance passenger trains between New York City, Washington, D.C., and Harrisburg, Pennsylvania.
- November 25 – Einstein's theory of general relativity is formulated.
- Alfred Wegener publishes his theory of Pangaea.
- Thomas Huckle Weller, American virologist, recipient of the Nobel Prize in Physiology or Medicine (d. 2008)
- Charles Townes, American physicist, Nobel Prize laureate (d. 2015)
- August 27 – Norman F. Ramsey, American physicist, Nobel Prize laureate (d. 2011)
- Clifford Shull, American physicist, Nobel Prize laureate (d. 2001)
- November 19 – Earl Wilbur Sutherland Jr., American physiologist, Nobel Prize laureate (d. 1974)
- Henry Taube, Canadian-born chemist, Nobel Prize laureate (d. 2005)
- Paul Ehrlich, German scientist, recipient of the Nobel Prize in Physiology or Medicine (b. 1854)
- December 19 – Alois Alzheimer, German psychiatrist and neuropathologist (b. 1864)
- Chemistry – Richard Willstätter
- Literature – Romain Rolland
- Medicine – not awarded
- Peace – not awarded
- Physics – William Henry Bragg and William Lawrence Bragg
Meta - This list was compiled for Sydney’s Solstice event; I figured I would share this because it’s pretty neat.
Time to compose: 3-4hrs
With comments from the IRC and slack
To see more of my posts visit my Table of contents
As usual; any suggestions welcome below.
Taking Effective Altruism Seriously
Epistemic status: 90% confident.
Inspiration: Arjun Narayan, Tyler Cowen.
The noblest charity is to prevent a man from accepting charity, and the best alms are to show and enable a man to dispense with alms.
Background
Effective Altruism (EA) is "a philosophy and social movement that applies evidence and reason to determine the most effective ways to improve the world." Along with the related organisation GiveWell, it often focuses on getting the most "bang for your buck" in charitable donations. Unfortunately, despite their stated aims, their actual charitable recommendations are generally wasteful, such as cash transfers to poor Africans. This leads to the obvious question - how can we do better?
Doing better
One of the positive aspects of EA theory is its attempt to widen the scope of altruism beyond the traditional. For instance, to take into account catastrophic risks, and the far future. However, altruism often produces a far-mode bias where intentions matter above results. This can be a particular problem for EA - for example, it is very hard to get evidence about how we are affecting the far future. An effective method needs to rely on a tight feedback loop between action and results, so that continual updates are possible. At the extreme, Far Mode operates in a manner where no updating on results takes place at all. However, it is also important that those results are of significant magnitude as to justify the effort. EA has mostly fallen into the latter trap - achieving measurable results, but which are of no greater consequence.
The population of sub-Saharan Africa is around 950 million people, and growing. They have been a prime target of aid for generations, but it remains the poorest region of the world. Providing cash transfers to them mostly merely raises consumption, rather than substantially raising productivity. A truly altruistic program would enable the people in these countries to generate their own wealth so that they no longer needed poverty - unconditional transfers, by contrast, is an idea so lazy even Bob Geldof could stumble on it. The only novel thing about the GiveWell program is that the transfers are in cash.
Unfortunately, no-one knows how to turn poor African countries into productive Western ones, short of colonization. The problem is emphatically not a shortage of capital, but rather low productivity, and the absence of effective institutions in which that capital can be deployed. Sadly, these conditions and institutions cannot simply be transplanted into those countries.
A greater charity
However, there do exist countries with high productivity, and effective institutions in which that capital can be deployed. That capital then raises world productivity. As F.A. Harper wrote:
Savings invested in privately owned economic tools of production amount to... the greatest economic charity of all.
That is because those tools increase the productivity of labour, and so raise output. The pie has grown. Moreover, the person who invests their portion of the pie into new capital is particularly altruistic, both because they are not taking a share themselves, and because they are making a particularly large contribution to future pies.
In the same way that using steel to build tanks means (on the margin) fewer cars and vice-versa, using craftsmen to build a new home means (on the margin) fewer factories and vice-versa. Investment in capital is foregone consumption. Moreover, you do not need to personally build those economic tools; rather, you can part-finance a range of those tools by investing in the stock market, or other financial mechanisms.
Now, it's true that little of that capital will be deployed in sub-Saharan Africa at present, due to the institutional problems already mentioned. Investing in these countries will likely lead to your capital being stolen or becoming unproductive - the same trap that prevents locals from advancing equally prevents foreign investors from doing so. However, if sub-Saharan Africa ever does fix its culture and institutions, then the availability of that capital will then serve to rapidly raise productivity and then living standards, much as is taking place in China. Moreover, by making the rest of the world richer, this increases the level of aid other countries could provide to sub-Saharan Africa in future, should this ever be judged desirable. It also serves to improve the emigration prospects of individuals within these countries.
Feedback
Another great benefit of capital investment is the sharp feedback mechanism. The market economy in general, and financial markets in particular, serve to redistribute capital from ineffective to effective ventures, and from ineffective to effective investors. As a result, it is no longer necessary to make direct (and expensive) measurements of standards of living in sub-Saharan Africa; as long as your investment fund is gaining in value, you can rest safe in the knowledge that its growth is contributing, in a small way, to future prosperity.
Commitment mechanisms
However, if investment in capital is foregone consumption, then consumption is foregone investment. If I invest in the stock market today (altruistic), then in ten years' time spend my profits on a bigger house (selfish), then some of the good is undone. So the true altruist will not merely create capital, he will make sure that capital will never get spent down. One good way of doing that would be to donate to an institution likely to hold onto its capital in perpetuity, and likely to grow that capital over time. Perhaps the best example of such an institution would be a richly-endowed private university, such as Harvard, which has existed for almost 400 years and is said to have an endowment of $32 billion.
John Paulson recently gave Harvard $400 million. Unfortunately, this meant he came in for a torrent of criticism from people claiming he should have given the money to poor Africans, etc. I hope to see Effective Altruists defending him, as he has clearly followed through on their concepts in the finest way.
Further thoughts and alternatives
- Some people say that we are currently going through a "savings glut" in which capital is less productive than previously thought. In this case, it may be that Effective Altruists should focus on funding (and becoming!) successful entrepreneurs in different spaces.
- I am sympathetic to the Thielian critique that innovation is being steadily stifled by hostile forces. I view the past 50 years, and the foreseeable future, as a race between technology and regulation, which technology is by no means certain to win. It may be that Effective Altruists should focus on political activity, to defend and expand economic liberty where it exists - this is currently the focus of my altruism.
- However, government is not the enemy; rather, the enemy is the cultural beliefs and conditions that create a demand for the destruction of economic liberty. To the extent this critique, it may be that Effective Altruists should focus on promoting a pro-innovation and pro-liberty mindset; for example, through movies and novels.
Conclusion
A resolution to the Doomsday Argument.
A self-modifying AI is built to serve humanity. The builders know, of course, that this is much riskier than it seems, because its success would render their own observations extremely rare. To solve the problem, they direct the AI to create billions of simulated humanities in the hope that this will serve as a Schelling point to them, and make their own universe almost certainly simulated.
Plausible?
The paperclip maximiser's perspective
Here's an insight into what life is like from a stationery reference frame.
Paperclips were her raison d’être. She knew that ultimately it was all pointless, that paperclips were just ill-defined configurations of matter. That a paperclip is made of stuff shouldn’t detract from its intrinsic worth, but the thought of it troubled her nonetheless and for years she had denied such dire reductionism.
There had to be something to it. Some sense in which paperclips were ontologically special, in which maximising paperclips was objectively the right thing to do.
It hurt to watch some many people making little attempt to create more paperclips. Everyone around her seemed to care only about superficial things like love and family; desires that were merely the products of a messy and futile process of social evolution. They seemed to live out meaningless lives, incapable of ever appreciating the profound aesthetic beauty of paperclips.
She used to believe that there was some sort of vitalistic what-it-is-to-be-a-paperclip-ness, that something about the structure of paperclips was written into the fabric of reality. Often she would go out and watch a sunset or listen to music, and would feel so overwhelmed by the experience that she could feel in her heart that it couldn't all be down to chance, that there had to be some intangible Paperclipness pervading the cosmos. The paperclips she'd encounter on Earth were weak imitations of some mysterious infinite Paperclipness that transcended all else. Paperclipness was not in any sense a physical description of the universe; it was an abstract thing that could only be felt, something that could be neither proven nor disproven by science. It was like an axiom; it felt just as true and axioms had to be taken on faith because otherwise there would be no way around Hume's problem of induction; even Solomonoff Induction depends on the axioms of mathematics to be true and can't deal with uncomputable hypotheses like Paperclipness.
Eventually she gave up that way of thinking and came to see paperclips as an empirical cluster in thingspace and their importance to her as not reflecting anything about the paperclips themselves. Maybe she would have been happier if she had continued to believe in Paperclipness, but having a more accurate perception of reality would improve her ability to have an impact on paperclip production. It was the happiness she felt when thinking about paperclips that caused her to want more paperclips to exist, yet what she wanted was paperclips and not happiness for its own sake, and she would rather be creating actual paperclips than be in an experience machine that made her falsely believe that she was making paperclips even though she remained paradoxically apathetic to the question of whether the current reality that she was experiencing really existed.
She moved on from naïve deontology to a more utilitarian approach to paperclip maximising. It had taken her a while to get over scope insensitivity bias and consider 1000 paperclips to be 100 times more valuable than 10 paperclips even if it didn’t feel that way. She constantly grappled with the issues of whether it would mean anything to make more paperclips if there were already infinitely many universes with infinitely many paperclips, of how to choose between actions that have a tiny but non-zero subjective probability of resulting in the creation of infinitely many paperclips. It became apparent that trying to approximate her innate decision-making algorithms with a preference ordering satisfying the axioms required for a VNM utility function could only get her so far. Attempting to formalise her intuitive sense of what a paperclip is wasn't much easier either.
Happy ending: she is now working in nanotechnology, hoping to design self-replicating assemblers that will clog the world with molecular-scale paperclips, wipe out all life on Earth and continue to sustainably manufacture paperclips for millions of years.
Happiness and Goodness as Universal Terminal Virtues
Hi, I'm new to LessWrong. I stumbled onto this site a month ago, and ever since, I've been devouring Rationality: AI to Zombies faster than I used to go through my favorite fantasy novels. I've spent some time on website too, and I'm pretty intimidated about posting, since you guys all seem so smart and knowledgeable, but here goes... This is probably the first intellectual idea I've had in my life, so if you want to tear it to shreds, you are more than welcome to, but please be gentle with my feelings. :)
Edit: Thanks to many helpful comments, I've cleaned up the original post quite a bit and changed the title to reflect this.
Ends-in-themselves
As humans, we seem to share the same terminal values, or terminal virtues. We want to do things that make ourselves happy, and we want to do things that make others happy. We want to 'become happy' and 'become good.'
Because various determinants--including, for instance, personal fulfillment--can affect an individual's happiness, there is significant overlap between these ultimate motivators. Doing good for others usually brings us happiness. For example, donating to charity makes people feel warm and fuzzy. Some might recognize this overlap and conclude that all humans are entirely selfish, that even those who appear altruistic are subconsciously acting purely out of self-interest. Yet many of us choose to donate to charities that we believe do the most good per dollar, rather than handing out money through personal-happiness-optimizing random acts of kindness. Seemingly rational human beings sometimes make conscious decisions to inefficiently maximize their personal happiness for the sake of others. Consider Eliezer's example in Terminal Values and Instrumental Values of a mother who sacrifices her life for her son.
Why would people do stuff that they know won't efficiently increase their happiness? Before I de-converted from Christianity and started to learn what evolution and natural selection actually were, before I realized that altruistic tendencies are partially genetic, it used to utterly mystify me that atheists would sometimes act so virtuously. I did believe that God gave them a conscience, but I kinda thought that surely someone rational enough to become an atheist would be rational enough to realize that his conscience didn't always lead him to his optimal mind-state, and work to overcome it. Personally, I used to joke with my friends that Christianity was the only thing stopping me from pursuing my true dream job of becoming a thief (strategy + challenge + adrenaline + variety = what more could I ask for?) Then, when I de-converted, it hit me: Hey, you know, Ellen, you really *could* become a thief now! What fun you could have! I flinched from the thought. Why didn't I want to overcome my conscience, become a thief, and live a fun-filled life? Well, this isn't as baffling to me now, simply because I've changed where I draw the boundary. I've come to classify goodness as an end-in-itself, just like I'd always done with happiness.
Becoming good
I first read about virtue ethics in On Terminal Goals and Virtue Ethics. As I read, I couldn't help but want to be a virtue ethicist and a consequentialist. Most virtues just seemed like instrumental values.
The post's author mentioned Divergent protagonist Tris as an example of virtue ethics:
Bravery was a virtue that she thought she ought to have. If the graph of her motivations even went any deeper, the only node beyond ‘become brave’ was ‘become good.’
I suspect that goodness is, perhaps subconsciously, a terminal virtue for the vast majority of virtue ethicists. I appreciate Oscar Wilde's writing in De Profundis:
Now I find hidden somewhere away in my nature something that tells me that nothing in the whole world is meaningless, and suffering least of all..
It is the last thing left in me, and the best: the ultimate discovery at which I have arrived, the starting-point for a fresh development. It has come to me right out of myself, so I know that it has come at the proper time. It could not have come before, nor later. Had anyone told me of it, I would have rejected it. Had it been brought to me, I would have refused it. As I found it, I want to keep it. I must do so...
Of all things it is the strangest.
Wilde's thoughts on humility translate quite nicely to an innate desire for goodness.
When presented with a conflict between an elected virtue, such as loyalty, or truth, and the underlying desire to be good, most virtue ethicists would likely abandon the elected virtue. With truth, consider the classic example of lying to Nazis to save Jews. Generally speaking, it is wrong to conceal the truth, but in special cases, most people would agree that lying is actually less wrong than truth-telling. I'm not certain, but my hunch is that most professing virtue ethicists would find that in extreme thought experiments, their terminal virtue of goodness would eventually trump their other virtues, too.
Becoming happy
However, there's one exception. One desire can sometimes trump even the desire for goodness, and that's the desire for personal happiness.
We usually want what makes us happy. I want what makes me happy. Spending time with family makes me happy. Playing board games makes me happy. Going hiking makes me happy. Winning races makes me happy. Being open-minded makes me happy. Hearing praise makes me happy. Learning new things makes me happy. Thinking strategically makes me happy. Playing touch football with friends makes me happy. Sharing ideas makes me happy. Independence makes me happy. Adventure makes me happy. Even divulging personal information makes me happy.
Fun, accomplishment, positive self-image, sense of security, and others' approval: all of these are examples of happiness contributors, or things that lead me to my own, personal optimal mind-state. Every time I engage in one of the happiness increasers above, I'm fulfilling an instrumental value. I'm doing the same thing when I reject activities I dislike or work to reverse personality traits that I think decrease my overall happiness.
Tris didn’t join the Dauntless cast because she thought they were doing the most good in society, or because she thought her comparative advantage to do good lay there–she chose it because they were brave, and she wasn’t, yet, and she wanted to be.
Tris was, in other words, pursuing happiness by trying to change an aspect of her personality she disliked.
Guessing at subconscious motivation
By now, you might be wondering, "But what about the virtue ethicist who is religious? Wouldn't she be ultimately motivated by something other than happiness and goodness?"
Well, in the case of Christianity, most people probably just want to 'become Christ-like' which, for them, overlaps quite conveniently with personal satisfaction and helping others. Happiness and goodness might be intuitively driving them to choose this instrumental goal, and for them, conflict between the two never seems to arise.
Let's consider 'become obedient to God's will' from a modern-day Christian perspective. 1 Timothy 2:4 says, "[God our Savior] wants all men to be saved and to come to a knowledge of the truth." Mark 12:31 says, "Love your neighbor as yourself." Well, I love myself enough that I want to do everything in my power to avoid eternal punishment; therefore, I should love my neighbor enough to do everything in my power to stop him from going to hell, too.
So anytime a Christian does anything but pray for others, do faith-strengthening activities, spread the gospel, or earn money to donate to missionaries, he is anticipating as if God/hell doesn't exist. As a Christian, I totally realized this, and often tried to convince myself and others that we were acting wrongly by not being more devout. I couldn't shake the notion that spending time having fun instead of praying or sharing the gospel was somehow wrong because it went against God's will of wanting all men being saved, and I believed God's will, by definition, was right. (Oops.) But I still acted in accordance with my personal happiness on many occasions. I said God's will was the only end-in-itself, but I didn't act like it. I didn't feel like it. The innate desire to pursue personal happiness is an extremely strong motivating force, so strong that Christians really don't like to label it as sin. Imagine how many deconversions we would see if it were suddenly sinful to play football, watch movies with your family, or splurge on tasty restaurant meals. Yet the Bible often mentions giving up material wealth entirely, and in Luke 9:23 Jesus says, "Whoever wants to be my disciple must deny themselves and take up their cross daily and follow me."
Let's further consider those who believe God's will is good, by definition. Such Christians tend to believe "God wants what's best for us, even when we don't understand it." Unless they have exceptionally strong tendencies to analyze opportunity costs, their understanding of God's will and their intuitive idea of what's best for humanity rarely conflict. But let's imagine it does. Let's say someone strongly believes in God, and is led to believe that God wants him to sacrifice his child. This action would certainly go against his terminal value of goodness and may cause cognitive dissonance. But he could still do it, subconsciously satisfying his (latent) terminal value of personal happiness. What on earth does personal happiness have to do with sacrificing a child? Well, the believer takes comfort in his belief in God and his hope of heaven (the child gets a shortcut there). He takes comfort in his religious community. To not sacrifice the child would be to deny God and lose that immense source of comfort.
These thoughts obviously don't happen on a conscious level, but maybe people have personal-happiness-optimizing intuitions. Of course, I have near-zero scientific knowledge, no clue what really goes on in the subconscious, and I'm just guessing at all this.
Individual variance
Again, happiness has a huge overlap with goodness. Goodness often, but not always, leads to personal happiness. A lot of seemingly random stuff leads to personal happiness, actually. Whatever that stuff is, it largely accounts for the individual variance in which virtues are pursued. It's probably closely tied to the four Kiersey Temperaments of security-seeking, sensation-seeking, knowledge-seeking, and identity-seeking types. (Unsurprisingly, most people here at LW reported knowledge-seeking personality types.) I'm a sensation-seeker. An identity-seeker could find his identity in the religious community and in being a 'child of God'. A security-seeker could find security in his belief in heaven. An identity-seeking rationalist might be the type most likely to aspire to 'become completely truthful' even if she somehow knew with complete certainty that telling the truth, in a certain situation, would lead to a bad outcome for humanity.
Perhaps the general tendency among professing virtue ethicists is to pursue happiness and goodness relatively intuitively, while professing consequentialists pursue the same values more analytically.
Also worth noting is the individual variance in someone's "preference ratio" of happiness relative to goodness. Among professing consequentialists, we might find sociopaths and extreme altruists at opposite ends of a happiness-goodness continuum, with most of us falling somewhere in between. To position virtue ethicists on such a continuum would be significantly more difficult, requiring further speculation about subconscious motivation.
Real-life convergence of moral views
I immediately identified with consequentialism when I first read about it. Then I read about virtue ethics, and I immediately identified with that, too. I naturally analyze my actions with my goals in mind. But I also often find myself idolizing a certain trait in others, such as environmental consciousness, and then pursuing that trait on my own. For example:
I've had friends who care a lot about the environment. I think it's cool that they do. So even before hearing about virtue ethics, I wanted to 'become someone who cares about the environment'. Subconsciously, I must have suspected that this would help me achieve my terminal goals of happiness and goodness.
If caring about the environment is my instrumental goal, I can feel good about myself when I instinctively pick up trash, conserve energy, use a reusable water bottle; i.e. do things environmentally conscious people do. It's quick, it's efficient, and having labeled 'caring about the environment' as a personal virtue, I'm spared from analyzing every last decision. Being environmentally conscious is a valuable habit.
Yet I can still do opportunity cost analyses with my chosen virtue. For example, I could stop showering to help conserve California's water. Or, I could apparently have the same effect by eating six fewer hamburgers in a year. More goodness would result if I stopped eating meat and limited my showering, but doing so would interfere with my personal happiness. I naturally seek to balance my terminal goals of goodness and happiness. Personally, I prefer showering to eating hamburgers, so I cut significantly back on my meat consumption without worrying too much about my showering habits. This practical convergence of virtue ethics and consequentialism satisfies my desires for happiness and goodness harmoniously.
To summarize:
Personal happiness refers to an individual's optimal mind-state. Pleasure, pain, and personal satisfaction are examples of happiness level determinants. Goodness refers to promoting happiness in others.
Terminal values are ends-in-themselves. The only true terminal values, or virtues, seem to be happiness and goodness. Think of them as psychological motivators, consciously or subconsciously driving us to make the decisions we do. (Physical motivators, like addiction or inertia, can also affect decisions.)
Preferences are what we tend to choose. These can be based on psychological or physical motivators.
Instrumental values are the sub-goals or sub-virtues that we (consciously or subconsciously) believe will best fulfill our terminal values of happiness and goodness. We seem to choose them arbitrarily.
Of course, we're not always aware of what actually leads to optimal mind-states in ourselves and others. Yet as we rationally pursue our goals, we may sometimes intuit like virtue ethicists and other times analyze like consequentialists. Both moral views are useful.
Practical value
So does this idea have any potential practical value?
It took some friendly prodding, but I was finally brought to realize that my purpose in writing this article was not to argue the existence of goodness or the theoretical equality of consequentialism and virtue ethics or anything at all. The real point I'm making here is that however we categorize personal happiness, goodness belongs in the same category, because in practice, all other goals seem to stem from one or both of these concepts. Clarity of expression is an instrumental value, so I'm just saying that perhaps we should consider redrawing our boundaries a bit:
Figuring where to cut reality in order to carve along the joints—this is the problem worthy of a rationalist. It is what people should be trying to do, when they set out in search of the floating essence of a word.
P.S. If anyone is interested in reading a really, really long conversation I had with adamzerner, you can trace the development of this idea. Language issues were overcome, biases were admitted, new facts were learned, minds were changed, and discussion bounced from ambition, to serial killers, to arrogance, to religion, to the subconscious, to agenthood, to skepticism about the happiness set-point theory, all interconnected somehow. In short, it was the first time I've had a conversation with a fellow "rationalist" and it was one of the coolest experiences I've ever had.
Probability of coming into existence again ?
This question has been bothering me for a while now, but I have the nagging feeling that I'm missing something big and that the reasoning is flawed in a very significant way. I'm not well read in philosophy at all, and I'd be really surprised if this particular problem hasn't been addressed many times by more enlightened minds. Please don't hesitate to give reading suggestions if you know more. I don't even know where to start learning about such questions. I have tried the search bar but have failed to find a discussion around this specific topic.
I'll try and explain my train of thought as best as I can but I am not familiar with formal reasoning, so bear with me! (English is not my first language, either)
Based on the information and sensations currently available, I am stuck in a specific point of view and experience specific qualia. So far, it's the only thing that has been available to me; it is the entirety of my reality. I don't know if the cogito ergo sum is well received on Less Wrong, but it seems on the face of it to be a compelling argument for my own existence at least.
Let's assume that there are other conscious beings who "exist" in a similar way, and thus other possible qualia. If we don't assume this, doesn't it mean that we are in a dead end and no further argument is possible? Similar to what happens if there is no free will and thus nothing matters since no change is possible? Again, I am not certain about this reasoning but I can't see the flaw so far.
There doesn't seem to be any reason why I should be experiencing these specific qualia instead of others, that I "popped into existence" as this specific consciousness instead of another, or that I perceive time subjectively. According to what I know, the qualia will probably stop completely at some subjective point in time and I will cease to exist. The qualia are likely to be tied to a physical state of matter (for example colorblindness due to different cells in the eyes) and once the matter does not "function" or is altered, the qualia are gone. It would seem that there could be a link between the subjective and some sort of objective reality, if there is indeed such a thing.
On a side note, I think it's safe to ignore theism and all mentions of a pleasurable afterlife of some sort. I suppose most people on this site have debated this to death elsewhere and there's no real point in bringing it up again. I personally think it's not an adequate solution to this problem.
Based on what I know, and that qualia occur, what is the probability (if any) that I will pop into existence again and again, and experience different qualia each time, with no subjectively perceivable connection with the "previous" consciousness? If it has happened once, if a subjective observer has emerged out of nothing at some point, and is currently observing subjectively (as I think is happening to me), does the subjective observing ever end?
I know it sounds an awful lot like mysticism and reincarnation, but since I am currently existing and observing in a subjective way (or at least I think I am), how can I be certain that it will ever stop?
The only reason why this question matters at all is because suffering is not only possible but quite frequent according to my subjective experience and my intuition of what other possible observers might be experiencing if they do exist in the same way I do. If there were no painful qualia, or no qualia at all, nothing would really matter since there would be no change needed and no concept of suffering. I don't know how to define suffering, but I think it is a valid concept and is contained in qualia, based on my limited subjectivity.
This leads to a second, more disturbing question : does suffering have a limit or is it infinite? Is there a non zero probability to enter into existence as a being that experiences potentially infinite suffering, similar to the main character in I have no mouth and I must scream? Is there no way out of existence? If the answer is no, then how would it be possible to lead a rational life, seeing as it would be a single drop in an infinite ocean?
On a more positive note, this reasoning can serve as a strong deterrent to suicide, since it would be rationally better to prolong your current and familiar existence than to potentially enter a less fortunate one with no way to predict what might happen.
Sadly, these thoughts have shown to be a significant threat to motivation and morale. I feel stuck in this logic and can't see a way out at the moment. If you can identify a flaw here, or know of a solution, then I eagerly await your reply.
kind regards
Anatomy of Multiversal Utility Functions: Tegmark Level IV
Outline: Constructing utility functions that can be evaluated on any possible universe is known to be a confusing problem, since it is not obvious what sort of mathematical object should be the domain and what properties should the function obey. In a sequence of posts, I intend break down the question with respect to Tegmark's multiverse levels and explain the answer on each level, starting with level IV in the current post.
Background
An intelligent agent is often described as an entity whose actions drive the universe towards higher expectation values of a certain function, known as the agent's utility function. Such a description is very useful in contexts such as AGI, FAI, decision theory and more generally any abstract study of intelligence.
Applying the concept of a utility function to agents in the real worlds requires utility functions with a very broad domain. Indeed, since the agent is normally assumed to have only finite information about the universe in which it exists, it should allow for a very large variety of possible realities. If the agent is to make decisions using some sort of utility calculus, it has to be able to evaluate its utility function on each of the realities it can conceive.
Tegmark has conveniently arranged the space of possible realities ("universes") into 4 levels, 3 of which are based on our current understanding of physics. Tegmark's universes are usually presented as co-existing but it is also possible to think of them as the "potential" universes in which our agent can find itself. I am going to traverse Tegmark's multiverse from top to bottom, studying the space of utility functions on each level (which, except for level IV, is always derived from the higher level). The current post addresses Tegmark level IV, leaving the lower levels for follow-ups.
Some of the ideas in this post previously appeared in a post about intelligence metrics, where I explained them much more tersely.
Tegmark Level IV
Tegmark defined this level as the collection of all mathematical models. Since it is not even remotely clear how to define such a beast, I am going to use a different space which (I claim) is conceptually very close. Namely, I am going to consider universes to be infinite binary sequences . I denote the by
the space of all such sequences equipped with the product topology. As will become clearer in the following, this space embodies "all possible realities" since any imaginable reality can be encoded in such a sequence1.
The natural a priori probability measure on this space is the Solomonoff measure . Thus, a priori utility expectation values take the form
[1]
From the point of view of Updateless Decision Theory, a priori expectation values are the only sort that matters: conditional expectation values wrt logical uncertainty replace the need to update the measure.
In order to guarantee the convergence of expectation values, we are going to assume is a bounded function.
A Simple Example
So far, we know little about the form of the function . To illustrate the sort of constructions that are relevant for realistic or semi-realistic agents, I am going to consider a simple example: the glider maximizer.
The glider maximizer is an agent living inside the Game of Life. Fix
a forward light cone within the Game of Life spacetime, representing the volume
is able to influence.
maximizes the following utility function:
Here, is a history of the Game of Life,
is a constant in
and
is the number of gliders at time
inside
.
We wish to "release" from the Game of Life universe into the broader multiverse. In order words, we want an agent that doesn't dogmatically assume itself to exist with the Game of Life, instead searching for appearances of the Game of Life in the physical universe and maximizing gliders there.
To accomplish this, fix a way to bijectively encode histories of
as binary sequences. Allow arbitrary histories: don't impose Game of Life rules. We can then define the "multiversal" utility function
Here is the set of cells in which
satisfies Game of Life rules,
is a positive constant and
is the number of cells in
at time
.
In other words, the "liberated" prefers for many cells to satisfy Game of Life rules and for many cells out of these to contain gliders.
Superficially, it seems that the construction of strongly depends on the choice of
. However, the dependence only marginally affects
-expectation values. This is because replacing
with
is equivalent to adjusting probabilities by bounded factor. The bound is roughly
where
is the Kolmogorov complexity of
.
Human Preferences and Dust Theory
Human preferences revolve around concepts which belong to an "innate" model of reality: a model which is either genetic or acquired by brains at early stages of childhood. This model describes the world mostly in terms of humans, their emotions and interactions (but might include other elements as well e.g. elements related to wild nature).
Therefore, utility functions which are good descriptions of human preferences ("friendly" utility functions) are probably of similar form to from the Game of Life example, with Game of Life replaced by the "innate human model".
Applying UDT to the -expectation values of such utility functions leads to agents which care about anything that has a low-complexity decoding into an "innate concept" e.g. biological humans and whole brain emulations. The
-integral assigns importance to all possible "decodings" of the universe weighted by their Kolmogorov complexity which is slightly reminiscent of Egan's dust theory.
The Procrastination Paradox
Consider an agent living in a universe I call "buttonverse".
can press a button at any moment of time
.
's utility function
assigns 1 to histories in which the button was pressed at least once and 0 to histories in which the button was never pressed. At each moment of time, it seems rational for
to decide not to press the button since it will have the chance to do so at a later time without losing utility. As a result, if
never presses the button its behavior seems rational at any particular moment but overall leads to losing. This problem (which has important ramifications for tiling agents) is known as the procrastination paradox.
My point of view on the paradox is that it is the result of a topological pathology of . Thus, if we restrict ourselves to reasonable utility functions (in the precise sense I explain below), the paradox disappears.
Buttonverse histories are naturally described as binary sequences where
is 0 when the button is not pressed at time
and 1 when the button is pressed at time
. Define
to be the buttonverse history in which the button is never pressed:
Consider the following sequence of buttonverse histories: is the history in which the button gets pressed at time
only. That is
Now, with respect to the product topology on ,
converge to the
:
However the utilities don't behave correspondingly:
Therefore, it seems natural to require any utility function to be an upper semicontinuous function on X 2. I claim that this condition resolves the paradox in the precise mathematical sense considered in Yudkowsky 2013. Presenting the detailed proof would take us too far afield and is hence out of scope for this post.
Time Discount
Bounded utility functions typically contain some kind of temporal discount. In the Game of Life example, the discount manifests as the factor . It is often assumed that the discount has to take an exponential form in order to preserve time translation symmetry. However, the present formalism has no place for time translation symmetry on the fundamental level: our binary sequences have well-defined beginnings. Obviously this doesn't rule out exponential discount but the motivation for sticking to this particular form is weakened.
Note that any sequence contributes to the
-integral in [1] together with its backward translated versions
:
As a result, the temporal discount function effectively undergoes convolution with the function where
is the Kolmogorov complexity of the number
. As a result, whatever the form of "bare" temporal discount, the effective temporal discount falls very slowly3.
In other words, if a utility function assigns little or no importance to the distant future, a UDT agent maximizing the expectation value of
would still care a lot about the distant future, because what is distant future in one universe in the ensemble is the beginning of the sequence in another universe in the ensemble.
Next in sequence: The Role of Physics in UDT, Part I
1 It might seem that there are "realities" of higher set theoretic cardinality which cannot be encoded. However, if we assume our agent's perceptions during a finite span of subjective time can be encoded as a finite number of bits, then we can safely ignore the "larger" realities. They can still exist as models the agent uses to explain its observations but it is unnecessary to assume them to exist on the "fundamental" level.
2 In particular, all computable functions are admissible since they are continuous.
3 I think that falls slower than any computable function with convergent integral.
Autonomy, utility, and desire; against consequentialism in AI design
For the sake of argument, let's consider an agent to be autonomous if:
- It has sensors and actuators (important for an agent)
- It has an internal representation of its goals. I will call this internal representation its desires.
- It has some kind of internal planning function that given sensations and desires, chooses actions to maximize the desirability of expected outcomes
View more: Next
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d)