Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Comment author: syllogism 16 March 2015 01:47:20AM *  10 points [-]

I don't think the Hamming advice is so great. It's akin to asking, "What are the highest salary professions? Why aren't you entering them?".

Academia is a market-place. Everyone wants high research impact, for a given expenditure of time. Some opportunities are higher-value than others, but as those opportunities appear, other researchers are going to identify them too.

So in academia, as in the economy, it's better to identify your comparative advantage --- both short-term, and long-term. You usually need to publish something quickly, so you need to know what you can do right away. But you also want to plan for the medium and long-term, too. It's a difficult trade-off.

Comment author: someonewrongonthenet 02 March 2014 11:01:37PM *  12 points [-]

I've had 6 years of formal spanish classes. When I speak spanish, I need to think of the phrase in english, and then translate each word to spanish, and it's all very awkward with no real fluency or soul. My accent is good, but I think that's just because the Spanish phenomes which are not present in English are present in Hindi.

I've had sporadic contact with Hindi via family members and movies. My Hindi is just as bad as my Spanish in terms of raw communicative power...but the nature of my knowledge is different. I can spontaneously and naturally say entire phrases with all the proper communicative cues (changes in pitch, expression) without any English in my mind. And I can directly understand the meaning of Hindi speech, whereas with Spanish I need to map it to English before grasping meaning.

So, if I said one sentence in Spanish, a Spanish speaker can immediately tell I'm not a native speaker. If I say one sentence in Hindu, for that one sentence I'm reported to sound exactly like a native speaker (but there is very little I can actually say, so I essentially sound like a child who has been speaking sentences for about 6 months. (People find it hilarious)).

So while I can communicate roughly equally poorly in both languages in a pinch, I can do more "natural" stuff like create/appreciate humor in Hindi, which I can never pull off in Spanish.

Given that experience, here are my suggestions:

Immersion substitution: Take a movie which you have seen the English version many times, and watch it in Hebrew. When an unfamiliar phrase appears, pause the movie, say the phrase, and try to figure out what it means (using the dictionary as a last resort). Once you get to the point where you can completely understand one movie in Hebrew, move on to another movie. (Side effects may include: Use of overly dramatic or poetic language in daily talk. This goes double from phrases which you learned from musical pieces). Unless you're totally at a loss, I'd suggest leaving any English subtitles off.

Repeat the phrases aloud. Don't be afraid to babble unfamiliar syllables like a baby. Whenever I go to India, I get a powerful instinctive urge to start babbling, saying random phrases, and repeating what people around me say. This draws weird looks, so I do it under my breathe, but I think it really helps/

Do not mentally translate from one language to another. Try to grab the meaning directly from the words, without going en-route to English. That is, as you think of the phrase, try to not activate the equivalent word in a language you already know, and instead try to strongly activate a non-verbal representation of the concept.

For example, if הַשְּׁקִיעָה means "sunset"...but literally means "the setting", then mentally envision a far-away object going downwards over the horizon. The english word "Sunset" most strongly brings to mind a warm orange glow because of the word sun, but "the setting" may well have a very different connotation to a Hebrew speaker, and might emphasize different aspects (the downward motion, the fact that it's now time to go inside and relax, etc). These small differences will become apparent from taking into consideration the literal translation, the figurative translation, and the context. Actively not translating will help these come clearer, because instead of just memorizing "sunset" you would focus on how close the word is to "שקיעה" (to sink), which you would have previously associated with downwards motions.

To the extent possible, try to do the same when speaking (it's harder because it can be a bit difficult to think in a communicative way without the aid of a language, but it will sound more natural in the end.)

Talk to yourself try to have an internal monologue in the target language, except move your mouth along with it. Movie-immersion is only listening...you need to simulate speaking immersion too, or you'll end up being able to understand but not speak (this has happened to me to some extent with Hindi).

I can't read Hindi, but I felt that reading written Spanish aloud made for a decent "immersion" substitute (in terms of how much I subjectively felt myself learning). You might be able to do that with hebrew if you've got the graphy-phenome mapping down. Writing spanish was much less helpful for me...the slower pace of the writing process had me instinctively using the time to apply the "rules" I had learned rather than doing it instinctively.

(Remember the fact that I can't speak Spanish or Hindi well enough to survive on my own in an immersed environment, when weighing my opinion)

Comment author: syllogism 04 March 2014 10:35:43AM 0 points [-]

I'm interested in developing better language learning software.

For the movie case, do you think these would be helpful? Any other ideas?

  • Read in the subtitles file before viewing, so that vocab can be checked and learned via spaced repetition
  • Option to slow down the dialogue, with pitch-shifting to keep it from sounding weird and bassy
Comment author: Luke_A_Somers 02 February 2014 04:36:25PM *  19 points [-]

I have been tossing around the idea of not-high-IQ rationalist fiction. Problem is, it's really hard to write. If they act rationally, people stop identifying the person as unintelligent. You get intelligence creep or an unsatisfying story.

The best route I can see is to make them well-substandard in intelligence. Rationalist!Forrest Gump, say.

ETA: Another problem is that adventures are usually sub-optimal. No one writes about the Amundsen expedition or equivalents (*) - they write about Scott expeditions.

*(except for Le Guin, who managed it because she's amazing)

Comment author: syllogism 03 February 2014 09:06:10AM *  7 points [-]

You'd go pretty far just telling the audience the character was unintelligent, by giving them unintelligent status markers. Give them a blue-collar career, and very low academic achievement, while also coming from a stable family and average opportunity.

It's been a while since I watched it, but do you think Ben Affleck's character in Good Will Hunting was rational, but of limited intelligence?

There are scattered examples of this sort of "humble working man, who lives honest and true" throughout fiction.

Comment author: scrafty 03 February 2014 05:53:01AM *  2 points [-]

He also likes arguing with Jeff Kaufman about effective altruism.

Comment author: syllogism 03 February 2014 08:59:43AM 1 point [-]

Can't say I'm impressed with his reasoning there.


Comment author: protest_boy 03 February 2014 12:42:19AM 0 points [-]

I'm not sure that he doesn't have "natural" skill or talent. I find the link now but I remember reading that he's extremely high IQ. (or something something eidetic memory something something?)

Motifs in his standup comedy routines are about how much smarter he is than everyone else, etc etc (anecdata)

Comment author: syllogism 03 February 2014 03:09:52AM 0 points [-]

It doesn't seem to me that he has that any more than ther Jeopardy! contenders.

Arthur Chu: Jeopardy! champion through exemplary rationality

22 syllogism 02 February 2014 08:02AM


I'm not sure I've ever seen such a compelling "rationality success story". There's so much that's right here.

The part that really grabs me about this is that there's no indication that his success has depended on "natural" skill or talent. And none of the strategies he's using are from novel research. He just studied the "literature" and took the results seriously. He didn't arbitrarily deviate from the known best practice based on aesthetics or intuition. And he kept a simple, single-minded focus on his goal. No lost purposes here --- just win as much money as possible, bank the winnings, and use it to self-insure. It's rationality-as-winning, plain and simple.

Comment author: Anatoly_Vorobey 03 January 2014 11:28:07AM 3 points [-]

Something that's always bugged me about being an academic is, we're terrible at communicating to people outside our field. This means that whenever I see a post using an NLP tool, they're using a crap tool.

Why is that, do you think? This doesn't seem to be the case in the ML community as far as I can judge (though I'm not an expert). What's special about NLP? What prevents the nltk people from doing what you did?

Comment author: syllogism 04 January 2014 04:18:05AM *  3 points [-]

In ML, everyone is engaging with the academics, and the academics are doing a great job of making that accessible, e.g. through MOOCs. ML is one of the most popular targets of "ongoing education", because it's popped up and it's a useful feather to have in your cap. It extends the range of programs you can write greatly. Many people realise that, and are doing what it takes to learn. So even if there are some rough spots in the curriculum, the learners are motivated, and the job gets done.

The cousin of language processing is computer vision. The problem we have as academics is that there is a need to communicate current best-of-breed solutions to software engineers, while we also communicate underlying principles to our students and to each other.

If you look at nltk, it's really a tool for teaching our grad students. And yet it's become a software engineering tool-of-choice, when it should never have been billed as industrial strength at all. Check out the results in my blog post:

  • NLTK POS tagger: 94% accuracy, 236s
  • My tagger: 96.8% accuracy, 12s

Both are pure Python implementations. I do no special tricks; I just keep things tight and simple, and don't pay costs from integrating into a large framework.

The problem is that the NLTK tagger is part of a complicated class hierarchy that includes a dictionary-lookup tagger, etc. These are useful systems to explain the problem to a grad student, but shouldn't be given to a software engineer who wants to get something done.

There's no reason why we can't have a software package that just gets it done. Which is why I'm writing one :). The key difference is that I'll be shipping one POS tagger, one parser, etc. The best one! If another algorithm comes out on top, I'll rip out the old one and put the current best one in.

That's the real difference between ML and NLP or computer vision. In NLP, we really really should be telling people, "just use this one". In ML, we need to describe a toolbox.

Comment author: syllogism 02 January 2014 05:18:06PM *  14 points [-]

I'm currently a post-doc doing language technology/NLP type stuff. I'm considering quitting soon to work full time on a start-up. I'm working on three things at the moment.

  • The start-up is a language learning web app: http://www.cloze.it . What sets it apart from other language-learning software is my knowledge of linguistics, proficiency with text processing, and willingness to code detailed language-specific features. Most tools want to be as language neutral as possible, which limits their scope a lot. So they tend to all have the same set of features, centred around learning basic vocab.

  • Something that's always bugged me about being an academic is, we're terrible at communicating to people outside our field. This means that whenever I see a post using an NLP tool, they're using a crap tool. So I wrote a blog post explaining a simple POS tagger that was better than the stuff in e.g. nltk (nltk is crap): http://honnibal.wordpress.com/2013/09/11/a-good-part-of-speechpos-tagger-in-about-200-lines-of-python/ The POS tagger post has gotten over 15k views (mostly from reddit), so I'm writing a follow up about a concise parser implementation. The parser is 500 lines, including the tagger, and faster and more accurate than the Stanford parser (the Stanford parser is also crap).

  • I'm doing minor revisions for a journal article on parsing conversational speech transcripts, and detecting disfluent words. The system gets good results when run on text transcripts. The goal is to allow speech recognition systems to produce better transcripts, with punctuation added, and stutters etc removed. I'm also working on a follow up paper to that one, with further experiments.

Overall the research is going well, and I find it very engaging. But I'm at the point where I have to start writing grant applications, and selling software seems like a much better expected-value bet.

Comment author: jockocampbell 17 November 2013 07:51:34PM 0 points [-]

Yes I agree, there is only a rough isomorphism between the mathematics of binary logic and the real world; binary logic seems to describe a limit that reality approaches but never reaches.

We should consider that the mathematics of binary logic are the limiting case of probability theory; it is probability theory where the probabilities may only take the values of 0 or 1. Probability theory can do everything that logic can but it can also handle those real world cases where the probability of knowing something is something other than 0 or 1, as is the usual case with scientific knowledge.

Comment author: syllogism 18 November 2013 05:11:34PM 0 points [-]

Yeah, I came across that idea in the Jaynes book, and was very impressed.

Comment author: jockocampbell 15 November 2013 10:48:42PM *  0 points [-]

Interesting discussion but I suspect an important distinction may be required between logic and probability theory. Logic is a special case of probability theory where values are restricted to only 0 and 1, that is to 0% and 100% probability. Within logic you may arrive at certain conclusions but generally within probability theory conclusions are not certain but rather assigned a degree of plausibility.

If logic provides, in some contexts, a valid method of reasoning then conclusions arrived at will be either 0% or 100% true. Denying that 100% confidence is ever rational seems to be equivalent to denying that logic ever applies to anything.

It is certainly true that many phenomena are better described by probability than by logic but can we deny logic any validity. I understand mathematical proofs as being within the realm of logic where things may often be determined as being either true or false. For instance Euclid is credited with first proving that there is no largest prime. I believe most mathematicians accept this as a true statement and that most would agree that 53 is easily proven to be prime.

Comment author: syllogism 17 November 2013 02:25:25PM *  6 points [-]

Denying that 100% confidence is ever rational seems to be equivalent to denying that logic ever applies to anything.

It's just saying that logic is a model that can't describe anything in the real world fully literally. That doesn't mean it's not useful. Abstracting away irrelevant details is bread and butter reductionism.

View more: Next