"Useful" is negatively correlated with "Correct theory"... on a grand scale.
Sure, having a correct theory has some positive correlation with "useful",
Which is it?
I think all the further you can go with this line of thought is to point out that lots of things are useful even if we don't have a correct theory for how they work. We have other ways to guess that something might be useful and worth trying.
Having a correct theory is always nice, but I don't see that our choice here is between having a correct theory or not having one.
Thank you for the detailed reply, I think I'll read the book and revisit your take on it afterward.
I suppose for me it's the sort of breathless enthusiastic presentation of the latest brainstorm as The Answer. Also I believe I am biased against ideas that proceed from an assumption that our minds are simple.
Still, in a rationalist forum, if one is to not be bothered by dismissing the content of material based on the form of its presentation, one must be pretty confident of the correlation. Since a few people who seem pretty smart overall think there might be something useful here, I'll spend some time exploring it.
I am wondering about the proposed eas...
Not to be discouraging, but is that really the "logical" reasoning used at the time? They use the word "rationalization" for a reason. "I can always work toward my goals tomorrow instead" will always be true.
Hopefully you had fun dancing, nothing wrong with it at all, but it does seem odd to be so self-congratulatory about deciding to go out and party.
Yes, I'm afraid this post is kind of impenetrable, although cousin_it's contribution helped. What is "RDS"?
Also, continually saying "People should..." do this and that and the other thing might be received better if you (meaning Michael, not Vladimir) start us off by doing a little of the desired analysis yourself.
If you're wondering whether I'm aware that I can figure out how to steal software licenses, I am.
ETA: I don't condemn those who believe that intellectual property rights are bad for society or immoral. I don't feel that way myself, though, so I act accordingly.
No specific use cases or examples, just throwing out ideas. On the one hand it would be cool if the notes one jots down could self-organize somehow, even a little bit. Now OpenCog is supposed by its creators to be a fully general knowledge representation system so maybe it's possible to use it as a sort of notation (like a probabilistic-logic version of mathematica? or maybe with a natural language front end of some kind? i think Ben Goertzel likes lojban so maybe an intermediate language like that)
Anyway, it's not really a product spec just one possible...
Thanks for the motivation, by the way -- I have toyed with the idea of getting Mathematica many times in the past but the $2500 price tag dissuaded me. Now I see that they have a $295 "Home Edition", which is basically the full product for personal use. I bought it last night and started playing with it. Very nifty program.
If the point of this essay was to advocate pharmaceutical research, it might have been more effective to say so, it would have made the process of digesting it smoother. Given the other responses I think I am not alone in failing to guess that this was pretty much your sole target.
I don't object to such research; a Bostrom article saying "it might not be impossible to have some effect" is weak support for a 10 IQ point avergage-gain pill, but that's not a reason to avoid looking for one. Never know what you'll find. I'm still not clear what th...
I'm still baffled about what you are getting at here. Apparently training people to think better is too hard for you, so I guess you want a pill or something. But there is no evidence that any pill can raise the average person's IQ by 10 points (which kind of makes sense, if some simple chemical balance adjustment could have such a dramatic effect on fitness it would be quite surprising). Are you researching a sci fi novel or something? What good does wishing for magical pills do?
The issue people are having is, that you start out with "sort of" as your response to the statement that math is the study of precisely-defined terms. In doing so, you decide to throw away that insightful and useful perspective by confusing math with attempts to use math to describe phenomena.
The pitfalls of "mathematical modelling" are interesting and worth discussing, but it actually doesn't help clarify the issue by jumbling it all together yourself, then trying to unjumble what was clear before you started.
Cool stuff. Good luck with your research; if you come up with anything that works I'll be in line to be a customer!
Well if you are really only interested in raising the average person's "IQ" by 10 points, it's pretty hard to change human nature (so maybe Bostrom was on the right track).
Perhaps if somehow video games could embed some lesson about rationality in amongst the dumb slaughter, that could help a little -- but people would probably just buy the games without the boring stuff instead.
I suppose the question is not whether it would be good, but rather how. Some quick brainstorming:
I think people are "smarter" now then they were, say, pre-scientific-method. So there may be more trainable ways-of-thinking that we can learn (for example, "best practices" for qualitative Bayesianism)
Software programs for individuals. Oh, maybe when you come across something you think is important while browsing the web you could highlight it and these things would be presented to you occasionally sort of like a "drill" to
Or: "Physics is not Math"
Um, so has Eurisko.
Perhaps a writeup of what you have discovered, or at least surmise, about walking that road would encourage bright young minds to work on those puzzles instead of reimplementing Eurisko.
It's not immediately clear that studying and playing with specific toy self-referential systems won't lead to ideas that might apply to precise members of that class.
You could use that feedback from the results of prior actions. Like: http://www.aleph.se/Trans/Individual/Self/zahn.txt
Interesting exercise. After trying for a while I completely failed; I ended up with terms that are completely vague (e.g. "comfort"), and actually didn't even begin to scratch the surface of a real (hypothesized) utility function. If it exists it is either extremely complicated (too complicated to write down perhaps) or needs "scientific" breakthroughs to uncover its simple form.
The result was also laughably self-serving, more like "here's roughly what I'd like the result to be" than an accurate depiction of what I do.
The re...
People on this site love to use fiction to illustrate their points, and a "biomoderate singularity managed by a superintelligent singleton" is very novel-friendly, so that's something!
Eliezer, in the ones I've seen so far I don't think you comes across very well. In particular you tend to ignore the point (or substance) of your partner's arguments which makes you look evasive or inattentive. There is also a fine line for viewers between confidence and arrogant pomposity and you often come across on the wrong side of that line. Hopefully this desire of yours to keep doing it reflects a commitment to improving, in which case keep at it. Perhaps asking a number of neutral parties about specifics would help you train for it... if you're...
If dark arts are allowed, it certainly seems like hundreds of millions of dollars spent on AI-horror movies like Terminator are a pretty good start. Barring an actual demostration of progress toward AI, I wonder what could actually be more effective...
Sometime reasonably soon, getting real actual physical robots into the uncanny valley could start to help. Letting imagination run free, I imagine a stage show with some kind of spookily-competent robot... something as simple as competent control of real (not CGI) articulated robots would be rather scary......
Apparently you and others have some sort of estimate of probability distribution over time leading you to being alarmed enough to demand action. Maybe it's say "1% chance in the next 20 years of hard takeoff" or something like that. Say what it is and how you got to it from "conceivability" or "non-impossibility". If there is a reasoned link that can be analyzed producing such a result, it is no longer a leap of faith; it can be reasoned about rationally and discussed in more detail. Don't get hung up on the number exactl...
Steven, I'm a little surprised that the paper you reference convinces you of a high probability of imminent danger. I have read this paper several times, and would summarize its relevant points thusly:
We tend to anthropomorphise, so our intuitive ideas about how an AI would behave might be biased. In particular, assuming that an AI will be "friendly" because people are more or less friendly might be wrong.
Through self-improvement, AI might become intelligent enough to accomplish tasks much more quickly and effectively than we expect.
This
One thing that might help change the opinion of people about friendly AI is to make some progress on it. For example, if Eliezer has had any interesting ideas about how to do it in the last five years of thinking about it, it could be helpful to communicate them.
A case that is credible to a large number of people needs to be made that this is a high-probability near-term problem. Without that it's just a scary sci-fi movie, and frankly there are scarier sci-fi movie concepts out there (e.g. bioterror). Making an analogy with a nuclear bomb is simply not...
For a continuation of the ideas in Beyond AI, relevant to this LW topic, see:
Hello all. I don't think I identify myself as a "rationalist" exactly -- I think of rationality more as a mode of thought (for example, when singing or playing a musical instrument, that is a different mode of thought, and there are many different modes of thought that are natural and appropriate for us human animals). It is a very useful mode of thought, though, and worth cultivating. It does strike me that the goals targeted by "Instrumental Rationality" are only weakly related to what I would consider "rationality" and ...
Well, if we really wanted to other-optimize we'd try to change your outlook on life, but I'm sure you get a lot of such advice already.
One thing you could try is making websites to sell advertising and maybe amazon clickthroughs. You would have to learn some new skills and have a little bit of discipline (and have some ideas about what might be popular). You could always start with the games you are interested in.
There's plenty of information out there about doing this. It will take a while to build up the income, and you may not be motivated enough to learn what you need to do to succeed.