Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Comment author: cousin_it 06 August 2017 10:24:33AM *  1 point [-]

If we want a measure of rationality that's orthogonal to intelligence, maybe we could try testing the ability to overcome motivated reasoning? Set up a conflict between emotion and reason, and see how the person reacts. The marshmallow test is an example of that. Are there other such tests, preferably ones that would work on adults? Which emotions would be easiest?

Comment author: Dagon 06 August 2017 09:30:40PM 0 points [-]

Why do you want a measure of rationality that's orthogonal to (measures of) intelligence? Whatever this reason is will likely lead you to a better phrasing of what aspects of behavior/capability you want to test for.

Comment author: casebash 03 August 2017 02:00:21PM 2 points [-]

Yes, they don't appear in the map, but when you see a mountain you think, "Hmm... this really needs to go in the map"

Comment author: Dagon 04 August 2017 04:32:39PM 0 points [-]

There are LOTS of maps which don't include mountains.

Comment author: whpearson 01 August 2017 07:58:01PM 0 points [-]

If the competition stops a person reproducing then sure it is a little violent. If it stops an idea reproducing, then I am not so sure I care about stopping all violence.

Poor competitors fail to reproduce, and that is the ultimate violence.

Failure to reproduce is not the ultimate violence. Killing someone and killing everyone vaguely related to them (including the bacteria that share a genetic code), destroying their culture and all its traces is far more violent.

Comment author: Dagon 01 August 2017 09:19:05PM 0 points [-]

If it stops an idea reproducing, then I am not so sure I care about stopping all violence.

Ideas have no agency. Agents competing for control/use of resources contain violence. I probably should back up a step and say "denial of goals is the ultimate violence". If you have a different definition (preferably something more complete than "no hitting"), please share it.

Comment author: Dagon 01 August 2017 05:54:30PM *  3 points [-]

I agree that there's something which draws us toward similar map structures. However, I'd avoid the word "natural" for describing this, as I suspect it's cultural or shared-experience rather than inherent in the territory. I therefore suspect that how one identifies "us" will result in very dissimilar map choices. Thee's likely to be much divergence between the categorization by ML models vs humans, and some variance between humans with wildly different cultural norms.

I might say "cultural shared expectations" as the thing you're trying to name.
edit: or maybe "comfortable map structures" or "focal structures"/"Schelling structures" (cf. https://en.wikipedia.org/wiki/Focal_point_(game_theory) ).

summary: I think these structures are from similarity of mapmakers, rather than anything in the territory.

Comment author: whpearson 31 July 2017 09:30:52PM 0 points [-]

Pro-social is not trying to take over the entire world or threatening . It is agreeing to mainly non-violent competition. Anti-social is genocide/pogroms, biocide, mind crimes, bio/nano warfare.

I'd rather no gating, but some gating might be required at different times.

Comment author: Dagon 31 July 2017 10:57:07PM 1 point [-]

mainly non-violent competition

Heh. If you think there's any such thing as "non-violent competition", you're not seeing through some levels of abstraction. All resource allocation is violent or has the threat of violence behind it.

Poor competitors fail to reproduce, and that is the ultimate violence.

Comment author: Dagon 25 July 2017 10:19:37PM 1 point [-]

Depending on the formality and requirements to qualify as "Friendly" as opposed to just "doesn't destroy humanity pretty quickly", I put it between 0% and 40%.

In response to Is Altruism Selfish?
Comment author: Dagon 25 July 2017 01:58:54PM 0 points [-]

People are only capable of acting upon their own utility function.

Kind of by definition. Utility function != self-interest, though. Many of us have expressions in our utility functions for other people's expressed or inferred desires.

I don't think I redefined "selfish".

I think you did. You picked a formal definition that does not match common usage. And you utterly failed to deconstruct why anyone would care either way.

Comment author: Dagon 12 July 2017 05:34:50PM 2 points [-]

I'm assuming you mean "new to you" ideas, not actually novel concepts for humanity as a whole. Both are rare, the latter almost vanishingly so. A lot of things we consider "new ideas" for ourselves are actually "new salience of an existing idea" or "change in relative weighting of previous ideas".

All to say that your rhetorical question is not the best way to think of this. Novelty isn't the hard part for ideas, it's application to decisions and weighting of competing ideas. Similarly, "recommendation" isn't the problem, it's one (imperfect) solution to the problem of decision-making costs (search costs in economic frameworks).

You're right it's not about time learning vs time practicing. It's about time in decision-making (including research, planning, and modeling results) vs time in action. Often called the explore vs exploit problem, and often framed as if it is knowledge being sought, but that's mostly because it's easier to come up with examples of that than examples of time spent deciding which framework fits better.

Comment author: cousin_it 07 July 2017 07:34:06AM *  1 point [-]

I've seen that article before, but can't quite understand it. Is there really a use for mixed sentences like "the probability that the probability that all ravens are black is 0.5 is 0.5"? It seems like both quantifiers and meta-probabilities are unnecessary, I can say all I want just by having a prior over states of the world with all its ravens. Relationships among multiple objects get folded into that as well.

Comment author: Dagon 08 July 2017 02:42:48PM 0 points [-]

Sure, but you can't actually hold the probability vector over all states with ravens. So you move up a level and summarize that set of probabilities to a smaller (and less precise) set.

All uncertainty is map, not territory. Anytime you are using probability, you're acknowledging that you're a limited calculator that cannot hold the complete state of the universe. If you could, you wouldn't need probability, you'd actually know the thing.

Meta-models are useful when specific models get cumbersome. Likewise meta-probability.

Comment author: Dagon 07 July 2017 04:57:17PM 9 points [-]

It's simply wrong to say that self-directed improvement doesn't work. Many people make lots of progress with self-reflection, reading, intentional practice, etc.

It would be equally correct (and equally misleading) to say "classes don't work". Personally, I dropped out of college after 2 years because I just wasn't getting enough from the structured, semi-useful coursework. There's LOTS of studies that show extremely weak correlation between class time (at reasonable margins, correcting for other factors) and later achievement.

Presumably, the best for any individual is some idiosyncratic mix of the two. In fact, it takes a fair bit of self-reflection and desire for improvement to identify the classes and groups which will help your goals, so there's no escaping that level.

View more: Next