Testing [pollid:1090]
I'm working through the udacity deep learning course right now, and I'm always trying to learn more things on the MIRI research guide. I'm in a fairly different timezone, but my schedule is pretty flexible. Maybe we can work something out?
This raises a really interesting point that I wanted to include in the top level post, but couldn't find a place for. It seems plausible/likely that human savants are implementing arithmetic using different, and much more efficient algorithms than those used by neurotypical humans. This was actually one of the examples I considered in support of the argument that neurons can't be the underlying reason humans struggle so much with math.
This is a really broad definition of math. There is regular structure in kinetic tasks like throwing a ball through a hoop. There's also regular structure in tasks like natural language processing. One way to describe that regular structure is through a mathematical representation of it, but I don't know that I consider basketball ability to be reliant on mathematical ability. Would you describe all forms of pattern matching as mathematical in nature? Is the fact that you can read and understand this sentence also evidence that you are good at math?
It's the average({4-2}/2), rather than the sum, since the altruistic agent is interested in maximizing the average utility.
The tribal limitations on altruism that you allude to are definitely one of the tendencies that much of our cultural advice on altruism targets. In many ways the expanding circle of trust, from individuals, to families, to tribes, to cities, to nation states, etc. has been one of the fundamental enablers of human civilization.
I'm less sure about the hard trade-off that you describe. I have a lot of experience being a member of small ...
Fiction is written from inside the head of the characters. Fiction books are books about making choices, about taking actions and seeing how they play out, and the characters don't already know the answers when they're making their decisions. Fiction books often seem to most closely resemble the problems that I face in my life.
Books that have people succeed for the wrong reasons I can put down, but watching people make good choices over and over and over again seems like a really useful thing. Books are a really cheap way to get some of the intuitive adva...
...On a very basic level, I am an algorithm receiving a stream of sensory data.
So, do you trust that sensory data? You mention reality, presumably you allow that objective reality which generates the stream of your sensory data exists. If you test your models by sensory data, then that sensory data is your "facts" -- something that is your criterion for whether a model is good or not.
I am also not sure how do you deal with surprises. Does sensory data always wins over models? Or sometimes you'd be willing to say that you don't believe your own e
Fact just isn't an epistemological category that I have, and it's not one that I find useful. There are only models.
So how you choose between different models, then? If there are no facts, what are your criteria? Why is the model of lizard overlords ruling the Earth any worse than any other model?
You use expressions like "because it's always been true in the past", but what do you mean by "true"?
My primary criterion is consistency. On a very basic level, I am an algorithm receiving a stream of sensory data. I make models to predic...
...it's not obvious to me that children are a good investment
I think you're engaging in nirvana fallacy. Children are not a good investment compared to what?
Again -- let's take a medieval European peasant. He has no ability to accumulate capital because he's poor, because his lord will just take his money if he notices it, and because once in a while an army passes through and basically grabs everything that isn't nailed down. He doesn't have any apprentices because peasants don't have apprentices (and apprentices leave once they learn the craft, anyway).
I don't understand why you think that human allegiances have to be founded on the nuclear family.
They don't have to be, but I think that empirical evidence points to family ties binding more tight than others.
Okay, but that doesn't necessarily matter. The ties don't have to be tight, they just have to be adequate. Also, the parent->child bond is typically tighter than the "child->parent" bond. But even if we add an uncertainty cost to forming non-parent child relationships, it's not obvious to me that children are a good investment. C...
That, actually, depends on the circumstances. But in any case, do you really suggest making friends as a good solution to who-will-feed-me problem? Don't forget that they will get old, too.
Human tribes have been a thing for about as long as there have been humans. People with an important role in the tribe don't starve to death. And yes, friends age, and so do children. You can make friends that aren't the same age as you. I don't understand why you think that human allegiances have to be founded on the nuclear family.
...The reality is precisely what i
Parents often devote significant resources to caring for special needs children who are unlikely to grow into good providers.
All the more reason to have a large extended family. These children will grow into adults who continue to need extra support, and there's no reason for parents to support them on their own. The more siblings you have to help out the better.
From a selfish perspective, the correct decision isn't to have more children. It's to kill or disown the ones who not only won't repay your investment, but will actually compete with you for ...
Women in general were low status. Many of their concerns and desires were ignored unless they happened to match concerns and desires that benefitted men. The fact that women didn't have alternatives to being a mother was just a special case of that..
How did men benefit? Did all men benefit? Were the men also constrained by cultural roles that served to benefit women?
...women's desires were considered irrelevant by society.
This is too strong a statement
Almost any statement interpreted while ignoring connotation is too strong. "Women's desires w
Being low status has always meant being vulnerable to social violence, and ascribing status is one of the ways that societies create and maintain social norms. The attractiveness of a position in society is dictated by the value and status society ascribes to it, and that valuation is always a set of "external reasons". Particularly low status groups or members of society, who are perceived as different or in violation of important social norms are often ascribed the status of "criminal" or "enemy" and are left especially vuln...
I don't see humans commonly engaging in a lot of decades off long-term thinking
No particular need. First, it is what happens by default if you don't take heroic birth control measures (remember, no pill or effective condoms), second, it's culturally ingrained, that's what everyone does
I'm a little uncomfortable classifying infanticide as heroic, but that aside I feel like your claim is shifting. At first you claimed that people choose to have children because they are making an optimal selfish long-term retirement decision and that they choose to hav...
That doesn't explain why people choose to have small families.
No, but that explains why that choice exists.
Not really. Humans have exercised control over family size for thousands of years via all sorts of different mechanisms. Modern birth control is certainly more convenient than the vast majority of ancient mechanisms, but it's not clear that the increase in convenience is why the modern world is a lot less excited than the ancient one about the command "Go forth and multiply."
...Would you seriously argue that people choose to have child
I'm not sure that this is true, or maybe I'm not sure that considering things on average is a good measure of surprise. Finding out you were wrong about something is much more surprising than learning something in the first place. Limited reasoners tend to discard alternative hypotheses when something fits the data well enough. Learning that the earth was flying through space around the sun even though it really doesn't feel like it is was much more surprising to me than it would have been if I hadn't seen the ground so stubbornly sitting still for most of...
Birth control is widely available;
That doesn't explain why people choose to have small families. In the Iliad the 50 rooms filled with Priam's sons are a mark of his wealth and power, guaranteeing the success and continuity of his bloodline. They aren't an accident. In developing nations people are proud of their large families and they regard as unfortunate people who only have a few children. Birth control may enable the transition, but it doesn't explain the stark difference in attitude.
...Social safety nets (and middle-class wealth) reduce the need
But face it. You're weird. And I mean that in a bad way, evolutionarily speaking. How many of you have kids? Damn few. The LessWrong mindset is maladaptive. It leads to leaving behind fewer offspring.
It's surprisingly not weird. Birthrates in the developed world have plummeted precisely because achievements other than sex and children have become markers of status. Having a large family is no longer seen as an indicator of high status but as something that makes you a bit of a cultural oddball, and that attitude is spreading. Cultural evolution hap...
I feel like this may be a semantics issue. I think that order implies information. To me, saying that a system becomes more ordered implies that I know about the increased order somehow. Under that construction, disorder (i.e. the absence of detectable patterns) is a measure of ignorance and disorder then is closely related to entropy. You may be preserving a distinction between the map and territory (i.e. between the system and our knowledge of the system) that I'm neglecting. I'm not sure which framework is more useful/productive.
I think it's definitely an important distinction to be aware of either way.
But if I know that all the gas molecules are in one half of the container, then I can move a piston for free and then as the gas expands to fill the container again I can extract useful work. It seems like if I know about this increase in order it definitely constitutes a decrease in entropy.
The 2nd law is never violated, not even a little. Unfortunately the idea that entropy itself can decrease in a closed system is a misconception which has become very widespread. Disorder can sometimes decrease in a closed system, but disorder has nothing to do with entropy!
Could you elaborate on this further? Order implies regularity which implies information that I can burn to extract useful work. I think I agree with you, but I'm not sure that I understand all the implications of what I'm agreeing with.
Meditation (empirical/practical emphasis), and more broadly the psychology associated with executive function and attentional control.
Set theory, topology, deep learning. Probably most math/computer science topics.
Anything that someone thinks they have a really good intuitive explanation for. Omniscience was one of my life goals when I was growing up.
Physics, quantum mechanics, related math concepts like linear algebra, abstract vector spaces, differential equations, calculus.
Much of the material in the LW sequences.
Optimization and machine learning. Also, shell scripting, python, perl, matlab, computability, numerical methods, basic data structures and algorithms.
More randomly: electrochemical energy storage, Li-ion batteries, distance running, dog training, Christian theology, Latin, English/American literature, poetry.
I like this idea. There are lots of things that I know and even more things that I'm interested in knowing, but I'm not sure I understand how it would play out.
How much tutoring experience do you have? What sorts of resources would there be for tutors? How long do you see tutor relationships lasting? What does it look like to tutor someone in Python Programming. Is this person trying to learn python on their own? Are they following a guide I'm familiar with? I know calculus like the back of my hand, but that doesn't mean I have lesson plans mapped out. Tu...
I'm an advocate of this approach in general for a number of reasons, and it's typically how I explain the idea of FAI to people without seeming like a prophet of the end times. Most of the reasons I like value-learning focus on what happens before a super-intelligence or what happens if a super-intelligence never comes into being.
I am strongly of the opinion that real world testing and real world application of theoretical results often exposes totally unanticipated flaws, and it seems like for the value-learning problem that partial/incomplete solutions ...
Just happened across this article summary today about people using atomic spectra to look for evidence of dark matter. I don't know that they've found anything yet, but it's sort of neat how closely related your proposal here is to their research.
The true pattern (i.e. the many-particle wavefunction) is smooth. The issue is that the pattern depends on the positions of every electron in the atom. The variational principle gives us a measure of the goodness of the wavefunction, but it doesn't give us a way to find consistent sets of positions. We have to rely on numerical methods to find self-consistent solutions for the set of differential equations, but it's ludicrously expensive to try to sample the solution space given the dimensionality of that space.
It's really difficult to solve large systems ...
I really like this topic, and I'm really glad you brought it up; it probably even deserves its own post.
There are definitely some people who are trying this, or similar approaches. I'm pretty sure it's one of the end goals of Stephen Wolfram's "New Kind of Science" and the idea of high-throughput searching of data for latent mathematical structure is definitely in vogue in several sub-branches of physics.
With that being said, while the idea has caught people's interest, it's far from obvious that it will work. There are a number of difficulties ...
Noise is certainly a problem, but the biggest problem for any sort of atomic modelling is that you quickly run into an n-body problem. Each one of of n electrons in an atom interacts with every other electron in that atom and so to describe the behavior of each electron you end up with a set of 70 something coupled differential equations. As a consequence, even if you just want a good approximation of the wavefunction, you have to search through a 3n dimensional Hilbert space and even with a preponderance of good experimental data there's not really a good way to get around the curse of dimensionality.
That's not really true. You can write a review article as one of your first publications and use it to lay out what you intend to work on. People won't take your review article as seriously as they will one written by Dr. Bigshot et al., but there certainly aren't any rules against it.
Also, the NSF is thrilled if you're a beginner and you're doing any sort of popular outreach. They love pop science blogs.
Thanks so much for your thoughtful response. This clarifies the position dramatically and makes it sound much more attractive. If I have any further questions related to my application specifically, I'll certainly let you know.
It's sort of not that useful though. This is a description of the "shovel-ready" projects and those are actually pretty straight-forward. If you fit into one of those categories, you'd basically be under a single person with a well-defined discipline and you can get a pretty good sense of who you'd be working for by scanning a half-dozen paper abstracts if you're not already familiar with them. There's a decent chance you're actually funded directly out of the individual professor's research grant. It's pretty business as usual.
But being a post-...
Candidates should have a PhD in a relevant field
I'm really curious as to what constitutes a relevant field. The 3 people you list are an economist, a conservation biologist, and a someone with a doctorate in geography. Presumably those are relevant fields, but I don't know what they have in common exactly.
I don't know what to think about this. You're new and you have sort of unconventional funding and a really broad mission statement. I'm not really sure what sort of research you're looking for or what journals it would be published in. I can't tell how...
Leplen, thank you for your comments, and for taking the time to articulate a number of the challenges associated with interdisciplinary research – and in particular, setting up a new interdisciplinary research centre in a subfield (global catastrophic and existential risk) that is in itself quite young and still taking shape. While we don’t have definitive answers to everything you raise, they are things we are thinking a lot about, and seeking a lot of advice on. While there will be some trial and error, given the quality and pooled experience of the acad...
This is phenomenally clear thinking and has clarified something I've been struggling to understand for the last 10 years. Thank you.
So the extent to which various traits are adaptive vs. maladaptive is an interesting question. There are a lot of hidden trade-offs, especially when you start discussing cognitive heuristics. Modern life also has some fairly different selection pressures than our species has historically been exposed to, so maybe some of those instincts are getting out-dated.
But all of that is secondary to a much larger consideration. Evolution doesn't share my goals. Evolution designed my brain for gene propagation. It does a decent job at survival, resource acquisition,...
Given how you have set this problem up, what do you think will be the relative prices of the 4 contracts you specified?
I understand that we're capable of calculating P(A|B), but if P(A|B) isn't on the market, then the market won't reflect the value of P(A|B). So I don't understand your statement that the market will somehow get the answer wrong because of its estimate of P(A|B). The market makes no value estimate of that quantity.
Your market, as stated, is really strange in a lot of ways. By having the contracts include "Bush wins" or "Clinton wins" the market is essentially predicting itself. It's going to have really strong attractors for a landslide ...
The normalization is because we want to compare what happens conditional on Hillary being elected to what happens conditional on Jeb is elected. These probabilities will not be comparable unless we normalize.
Why would we want to do this? Your contracts aren't structured in such a way that they encourage these sorts of conditional considerations. P(A|B) isn't on the market. P(A and B) is. Maybe you meant for your contracts to be "If Hillary is elected, the U.S. will be nuked." ?
if we normalize by dividing by the marginal probability that Hillary is elected, we get 1/3 which is equal to Pr [Nuked | Clinton Elected]. In other words, the prediction market estimates the wrong quantities.
Why are you doing this normalization? It doesn't seem related to the 4 contracts on your prediction market in an obvious way.
There was a common cause of Hillary being elected and the US being nuked. This common cause - whether Kim Jong-Un was still Great Leader of North Korea
I'm confused as to how Kim Jong-Un being leader of NK "causes" Hillary to be elected. That seems to go against state 5 in your table.
The reason that AI wants to turn the universe into paperclips is because it's the 2nd coming of Clippy.
I'm not sure that the number of possible states of the universe is relevant. I would imagine that the vast majority of the variation in that state space would be characterized by human indifference. The set of all possible combinations of sound frequencies is probably comparably enormous, but that doesn't seem to have precluded Pandora's commercial success.
I have to categorically disagree with the statement that people don't have access to their values or that their answers about what they value will tend to be erratic. I would wager that an overwhelming ...
How are human values categorically different from things like music preference? Descriptions of art also seem to to rely on lots of fairly arbitrary objects that it's difficult to simplify.
I'm also not sure what qualifies as unacceptably wrong. There's obviously some utility in having very crude models of human preferences. How would a slightly less crude model suddenly result in something that was "unacceptably" wrong?
I don't think that trying to solve the Schrödinger equation itself is particularly useful. The SE is a partial differential equation, and there's a whole logic of differential equations and boundary conditions, etc. that provides context for the SE. If you're serious about trying to understand quantum mechanics, I think the concept of Hilbert space/abstract vector spaces/linear algebra in general is a bigger conceptual shift than just being able to solve the particle in a box in function space. It's also just a really useful set of concepts that makes lear...
I don't understand this. Why should my utility function value me having a large income or having a large amount of money? What does that get me?
I don't have a good logical reason for why my life is a lot more valuable than anyone else's. I have a lot more information about how to effectively direct resources into improving my own life vs. improving the lives of others, but I can't come up with a good reason to have a dominantly large "Life of leplen" term in my utility function. Much of the data suggests that happiness/life quality isn't well co...
I'm really curious as to where you're getting the $500B number from. I felt like I didn't understand this argument very well at all, and I'm wondering what sorts of results you're imagining as a result of such a program.
It's worth noting that 1E30-1E40 is only the cost of simulating the neurons, and an estimate for the computational cost of simulating the fitness function is not given, although it is stated that the fitness function "is typically the most computationally expensive component". So the evaluation of the fitness function (which presu...
The normal distribution is just a model. You have to be very careful about expectations that happen at 6 sigma. Nothing guarantees your gaussian works well that far from the mean.
Because one requires only a theoretical breakthrough and the other requires engineering. Ideas iterate very quickly. Hardware has to be built. The machines that make the machines you want to use have to be designed, whole industries may have to be invented. A theoretical breakthrough doesn't have the same lag time.
If I work as a theorist and I have a brilliant insight, I start writing the paper tomorrow. If I work as an experimentalist and I have a brilliant insight, I start writing the grant to purchase the new equipment I'll need.
"There are 729,500 single women my age in New York City. My picture and profile successfully filtered out 729,499 of them and left me with the one I was looking for."
I know this is sort of meant as a joke, but I feel like one of the more interesting questions that could be addressed in an analysis like this is what percentage of the women in the dating pool could you actually have had a successful relationship with. How strong is your filter and how strong does it need to be? There's a tension between trying to find/obtain the best of many possib... (read more)