Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Comment author: ryjm 30 December 2013 03:32:54AM 2 points [-]

For anyone interested in vipassana meditation, I would recommend checking out Shinzen Young. He takes a much more technical approach to the practice. This pdf by him is pretty good.

Comment author: ryjm 24 July 2013 06:14:57PM *  5 points [-]

Oh my god if we can get this working with org-mode and habitrpg it will be the ultimate trifecta. And I've already got the first two (here).

Seriously this could be amazing. Org-mode and habitrpg are great, but they don't really solve the problem of what to do next. But with this, you get the data collection power of org mode with the motivational power of habitrpg - then Familiar comes in, looks at your history (clock data, tags, agendas, all of the org mode stuff will be a huge pool of information that it can interact with easily because emacs) and does its thing.

It could tell habitrpg to give you more or less experience for things that are correlated with some emotion you've tagged an org mode item with. Or habits that are correlated with less clocked time on certain tasks. If you can tag it org mode you can track it with familiar, and familiar will then controls how habitrpg calculates your experience. Eventually you won't have that nagging feeling in the back of your head that says "Wow, I'm really just defining my own rewards and difficulty levels, how is this going to actually help me if I can just cheat at any moment?" - Maybe you can still cheat yourself, but Familiar will tell you exactly the extent of your bullshit. It basically solves the biggest problem of gamification! You'll have to actually fight for your rewards, since Familiar won't let you get away with getting tons of experience for tasks that are not correlated with anything useful. Sure it won't be perfectly automated, but it will be close enough.

It could sort your agenda by what you actually might get done vs shit that you keep there because you feel bad about not doing it - and org mode already has a priority system. It could tell you what habits (org-mode has these too) are useful and what you should get rid of.

It could work with magit to get detailed statistics about your commit history and programming patterns.

Or make it work with org-drill to analyze your spaced repetition activity! Imagine, you could have an org-drill file associated with a class you are taking and use it to compare test grades and homework scores and the clocking data from homework tasks. Maybe there is a correlation between certain failing flashcards and your recent test score. Maybe you are spending too much time on SRS review when it's not really helping. These are things that we usually suspect but won't act on, and I think seeing some hard numbers, even if they aren't completely right, will be incredibly liberating. You don't have to waste cognitive resources worrying about your studying habits or wondering if you are actually stupid, because familiar will tell you! Maybe it could even suggest flashcards at some point, based on commit history or wikipedia reading or google searches.

Maybe some of this is a little far fetched but god would it be fun to dig into.

Comment author: JonahSinick 31 May 2013 08:25:26PM *  6 points [-]

I personally am optimistic about the world's elites navigating AI risk as well as possible subject to inherent human limitations that I would expect everybody to have, and the inherent risk. Some points:

  1. I've been surprised by people's ability to avert bad outcomes. Only two nuclear weapons have been used since nuclear weapons were developed, despite the fact that there are 10,000+ nuclear weapons around the world. Political leaders are assassinated very infrequently relative to how often one might expect a priori.

  2. AI risk is a Global Catastrophic Risk in addition to being an x-risk. Therefore, even people who don't care about the far future will be motivated to prevent it.

  3. The people with the most power tend to be the most rational people, and the effect size can be expected to increase over time (barring disruptive events such as economic collapses, supervolcanos, climate change tail risk, etc). The most rational people are the people who are most likely to be aware of and to work to avert AI risk. Here I'm blurring "near mode instrumental rationality" and "far mode instrumental rationality," but I think there's a fair amount of overlap between the two things. e.g. China is pushing hard on nuclear energy and on renewable energies, even though they won't be needed for years.

  4. Availability of information is increasing over time. At the time of the Dartmouth conference, information about the potential dangers of AI was not very salient, now it's more salient, and in the future it will be still more salient.

  5. In the Manhattan project, the "will bombs ignite the atmosphere?" question was analyzed and dismissed without much (to our knowledge) double-checking. The amount of risk checking per hour of human capital available can be expected to increase over time. In general, people enjoy tackling important problems, and risk checking is more important than most of the things that people would otherwise be doing.

I should clarify that with the exception of my first point, the arguments that I give are arguments that humanity will address AI risk in a near optimal way – not necessarily that AI risk is low.

For example, it could be that people correctly recognize that building an AI will result in human extinction with probability 99%, and so implement policies to prevent it, but that sometime over the next 10,000 years, these policies will fail, and AI will kill everyone.

But the actionable thing is how much we can reduce the probability of AI risk, and if by default people are going to do the best that one could hope, we can't reduce the probability substantially.

Comment author: ryjm 01 June 2013 03:23:24AM *  2 points [-]

I've been surprised by people's ability to avert bad outcomes. Only two nuclear weapons have been used since nuclear weapons were developed, despite the fact that there are 10,000+ nuclear weapons around the world. Political leaders are assassinated very infrequently relative to how often one might expect a priori.

Why would a good AI policy be one which takes as a model a universe where world destroying weapons in the hands of incredibly unstable governments controlled by glorified tribal chieftains is not that bad of a situation? Almost but not quite destroying ourselves does not reflect well on our abilities. The Cold War as a good example of averting bad outcomes? Eh.

AI risk is a Global Catastrophic Risk in addition to being an x-risk. Therefore, even people who don't care about the far future will be motivated to prevent it.

This is assuming that people understand what makes an AI so dangerous - calling an AI a global catastrophic risk isn't going to motivate anyone who thinks you can just unplug the thing (and even worse if it does motivate them, since then you have someone who is running around thinking the AI problem is trivial).

The people with the most power tend to be the most rational people, and the effect size can be expected to increase over time (barring disruptive events such as economic collapses, supervolcanos, climate change tail risk, etc). The most rational people are the people who are most likely to be aware of and to work to avert AI risk. Here I'm blurring "near mode instrumental rationality" and "far mode instrumental rationality," but I think there's a fair amount of overlap between the two things. e.g. China is pushing hard on nuclear energy and on renewable energies, even though they won't be needed for years.

I think you're just blurring "rationality" here. The fact that someone is powerful is evidence that they are good at gaining a reputation in their specific field, but I don't see how this is evidence for rationality as such (and if we are redefining it to include dictators and crony politicians, I don't know what to say), and especially of the kind needed to properly handle AI - and claiming evidence for future good decisions related to AI risk because of domain expertise in entirely different fields is quite a stretch. Believe it or not, most people are not mathematicians or computer scientists. Most powerful people are not mathematicians or computer scientists. And most mathematicians and computer scientists don't give two shits about AI risk - if they don't think it worthy of attention, why would someone who has no experience with these kind of issues suddenly grab it out of the space of all possible ideas he could possibly be thinking about? Obviously they aren't thinking about it now - why are you confident this won't be the case in the future? Thinking about AI requires a rather large conceptual leap - "rationality" is necessary but not sufficient, so even if all powerful people were "rational" it doesn't follow that they can deal with these issues properly or even single them out as something to meditate on, unless we have a genius orator I'm not aware of. It's hard enough explaining recursion to people who are actually interested in computers. And it's not like we can drop a UFAI on a country to get people to pay attention.

Availability of information is increasing over time. At the time of the Dartmouth conference, information about the potential dangers of AI was not very salient, now it's more salient, and in the future it will be still more salient.

In the Manhattan project, the "will bombs ignite the atmosphere?" question was analyzed and dismissed without much (to our knowledge) double-checking. The amount of risk checking per hour of human capital available can be expected to increase over time. In general, people enjoy tackling important problems, and risk checking is more important than most of the things that people would otherwise be doing.

It seems like you are claiming that AI safety does not require a substantial shift in perspective (I'm taking this as the reason why you are optimistic, since my cynicism tells me that expecting a drastic shift is a rather improbable event) - rather, we can just keep chugging along because nice things can be "expected to increase over time", and this somehow will result in the kind of society we need. These statements always confuse me; one usually expects to be in a better position to solve a problem 5 years down the road, but trying to describe that advantage in terms of out of thin air claims about incremental changes in human behavior seems like a waste of space unless there is some substance behind it. They only seem useful when one has reached that 5 year checkpoint and can reflect on the current context in detail - for example, it's not clear to me that the increasing availability of information is always a net positive for AI risk (since it could be the case that potential dangers are more salient as a result of unsafe AI research - the more dangers uncovered could even act as an incentive for more unsafe research depending on the magnitude of positive results and the kind of press received. But of course the researchers will make the right decision, since people are never overconfident...). So it comes off (to me) as a kind of sleight of hand where it feels like a point for optimism, a kind of "Yay Open Access Knowledge is Good!" applause light, but it could really go either way.

Also I really don't know where you got that last idea - I can't imagine that most people would find AI safety more glamorous then, you know, actually building a robot. There's a reason why it's hard to get people to do unit tests and software projects get bloated and abandoned. Something like what Haskell is to software would be optimal. I don't think it's a great idea to rely on the conscientiousness of people in this case.

Comment author: JesseGalef 17 May 2013 02:09:54AM *  20 points [-]

Regarding the music: I found video game soundtracks to be especially perfect - after all, they're designed to be background music. But I think there's more to it than that. I've had years of conditioning such that when I hear the Warcraft II soundtrack I immediately get into a mindset of intense concentration and happiness.

Obviously it depends on your tastes and whether you have attachments to particular video games, but here are my favorites:

(non-video game music that go into the rotation)

Comment author: ryjm 19 May 2013 04:58:45PM *  0 points [-]

focus@will is pretty useful for me - I've never been into movie music, but the cinematic option was very inspiring for me. There is some science behind the project too.

Comment author: ryjm 31 January 2013 06:57:16PM *  3 points [-]

For the GTD stuff, I use emacs + org-mode + .emacs based on this configuration + mobile org.

Since I try to work exclusively in emacs, I can quickly capture notes and "things that need to get done" in their proper context, all of which is aggregated under an Agenda window. The Agenda window manages a collection of ".org" files which store the specific details of everything. MobileOrg syncs all these .org files to my phone. Combined with the GTD philosophy of never having anything uncategorized bouncing around in my mind, this system works very well for me.

Example workflow (a better and more complete example is in the configuration I linked above):

  1. At the end of class, Professor assigns a programming project due in a week. I pull out my phone and quickly capture a TODO item with a deadline in Mobileorg. Mobileorg syncs this to google calendar.
  2. I get home and pull up the agenda in emacs. The item referencing the programming project shows up in my "Tasks to refile" category (equivalent to "Inbox" in GTD terms), along with any other TODOs I captured while I was at school.
  3. I refile the project to an org file that contains all the information about my classes and define a NEXT item under it, which represent the next action I need to take on the project. When I start working on the project, I can attach any files related to it directly on the TODO item identifying the project.
  4. The NEXT item shows up on a list of NEXT items on the agenda. I can filter these by project (defined in the GTD way) or by the tag system.

It all seems very complicated, but all of this is literally a couple of keystrokes. And this barely scratches the surface (take a look at the aforementioned configuration to see what I mean).

Pros:

  • Forces you to learn emacs.
  • Easily configurable and incredibly robust.
  • Optimized for functionality rather than prettiness (i.e if you end up liking it, you'll know it wasn't because of the nice UI, which is usually the main selling point for any computer based organizational system).

Cons:

  • Forces you to learn emacs.
  • Takes a huge amount of effort to set up. I would compare it to setting up an Arch Linux system.
  • Can get messy if you don't know what you're doing.
  • Getting the syncing functionality isn't easy.

A spaced repetition package is also available for org-mode, which really ties the whole thing together for me.

EDIT: You can also overlay latex fragments directly in org-mode, which is really nice for notetaking. Whole .org files can be exported to latex as well.

Comment author: ryjm 18 December 2012 07:42:42PM 0 points [-]

Whether it is meant for entertainment or not I think the usefulness of these hypothetical scenarios (in the context of a community blog) is directly proportional to the precision of their construction.

Value Loading

3 ryjm 23 October 2012 04:47AM

This article was originally on the FHI wiki and is being reposted to LW Discussion with permission. All content in this article is credited to Daniel Dewey.

In value loading, the agent will pick the action:

Here A is the set of actions the agent can take, e is the evidence the agent has already seen, W is the set of possible worlds, and U is the set of utility functions the agent is considering.

The parameter C(u) is some measure of the 'correctness' of the utility u, so the term p(C(u)|w) is the probability of u being correct, given that the agent is in world w. A simple example is of an AI that completely trusts the programmers; so if u is some utility function that claims that giving cake is better than giving death, and w1 is a world where the programmers have said "cake is better than death" while w2 is a world where they have said the opposite, then p(C(u)|w1) = 1 and p(C(u) | w2) = 0.

There are several challenging things in this formula:

W : How to define/represent the class of all worlds under consideration

U : How to represent the class of all utility functions over such worlds

C : What do we state about the utility function: that it is true? believed by humans?

p(C(u)|w) : How to define this probability

: How to sum up utility functions (a moral uncertainty problem)

In contrast:

is mostly the classic AI problem. It is hard to predict what the world is like from evidence, but this is a well known and studied problem and not unique to the present research. There is a trick to it here in that the nature of w includes the future actions of the agent which will depend upon how good future states look to it, but this recursive definition eventually bottoms out like a game of chess (where what happens when I make a move depends on what moves I make after that). It may cause an additional exponential explosion in calculating out the formula though, so the agent may need to make probabilistic guesses as to its own future behaviour to actually calculate an action.

This value loading equation is not subject to the classical Cake or Death problem, but is vulnerable to the more advanced version of the problem, if the agent is able to change the expected future value of p(C(u)) through its actions.

Daniel Dewey's Paper

The above idea was partially inspired by a draft of Learning What to Value, a paper by Daniel Dewey. He restricted attention to streams of interactions, and his equation, in a simplified form, is:

where S is the set of all possible streams of all past and future observations and actions.

Comment author: RolfAndreassen 15 October 2012 09:19:39PM 0 points [-]

I did consider that the post was very well written, but then, it is precisely the child prodigies who have the greatest difficulty in high school. The language is not out of reach for a high-intelligence teenager who both reads and writes a lot.

In any case I sit corrected on the OP's age.

Comment author: ryjm 15 October 2012 10:46:56PM 0 points [-]

I understand, and I do think you gave good advice (I love pg's writing).

On a related note, I just get a little worried when these threads come up. We like to hide behind computing jargon and Spock-like introspection; this does help with efficient communication, but probably makes us look more resilient than we really are. These kind of LW discussion posts are probably of very high social value to the OP and the tone of the responses have more of an effect than we would like to admit.

So helping the OP to see hard truths is all well and good, but it seems to me that we could use a bit more finesse. It's easier to understand the root of a problem when we have such precise words for everything, but it also means our pontifications must be just as precise or miss the mark completely - possibly hitting something we weren't aiming for.

Comment author: RolfAndreassen 15 October 2012 03:35:36PM 2 points [-]

This may be a stupid question: How old are you? From the pattern of your posts I seem to detect a vague hint of an American high school. Please observe: American schools are severely broken. (1) Inability to function in such a place is not necessarily a sign of insanity. Consider whether, perhaps, you may simply be surrounded by hormonal teenagers with nothing better to do than assert idiotic status hierarchies. If you're not already familiar with it, Paul Graham's essay on nerds may be relevant to your interests.

That aside: Are you sure you are really trying to be rational, as opposed to performing some ritual of cognition? It's hard to use oneself as a control group, you certainly cannot do so double-blindly, but what happens if you drop your attempts at rationality and act without thought, or with less thought than you are currently using? For all you know, that may give even worse results; but at any rate it seems to me that your problem is desperately in need of a control, or baseline. When you debug, it's no use saying that the error must be somewhere in module X. Comment out module X and see if the crash still happens.

Footnote 1: I read this on the Internet, so it must be true.

Comment author: ryjm 15 October 2012 06:49:29PM 3 points [-]

He is older than 23 per this comment. But reading his posts, either you have some extremely high standards for high school students or I am terrible at estimating someone's level of education. (Unless you were measuring emotional maturity somehow).

In any case, I would find it pretty disheartening if someone asked me if I was in high school in a post about my own mental health. I'm sure you didn't mean to be rude, but I find it hard to believe that this response would be anything but patronizing or insulting to anyone who isn't a high school student.

Comment author: RichardKennaway 18 September 2012 10:46:04AM *  2 points [-]

So, yeah, I don't think you'd encourage anyone to read anything by calling it "not original".

I don't know anything about the friends ciphergoth is attempting to reach, but I observe that in religion, "original" would be the greater turn-off. In religion, every innovation is heretical, because it is an innovation. To be accepted it must be presented as "not original", either because it is exactly in accordance with official doctrine, or because it is a return to the true religion that the official doctrine has departed from. It is rare for a religion to successfully introduce a new prophet with the power to sweep away the old, and even then ("I am not come to destroy, but to fulfil") the pretence is maintained that no such thing has happened.

Comment author: ryjm 18 September 2012 02:20:30PM 0 points [-]

Someone who doesn't want to read science-y stuff because they have that kind of mindset is not going to suddenly become curious when someone tells them it's based on science-y stuff from less than 30 years ago.

I like to think of it temporally; that religion is much like rationalists facing the wrong direction. Both occasionally look over their shoulders to confirm their beliefs (although with theists it's more like throwing a homunculus into the distant past and using that for eyes), while most of the time the things we really care about and find exciting are in front of us. Original vs unoriginal with respect to modern thought is of no practical interest to someone with the "every innovation is heretical" mindset unless it is completely within their usual line of sight - heretical is code for "I don't want to keep looking over my shoulder", not "I hate the original on principle". So unless you put that "original" encouragement thousands of years ago where they can see it, where it's a matter of one in front and one behind, the distinction between which is the greater turn-off is not going to matter, or bait anyone into turning around - there is nothing in their usually observed world to relate it to.

View more: Next