= 27d567bc2d48d9f40e9ccc20ea370b15)
pjeby comments on Practical Advice Backed By Deep Theories - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (112)
Actually, speaking as somebody who's done this, what I can tell you is that a huge amount of the experimenters get stuff wrong in their models and conclusions, because their terminology is at cross-purposes to what's really happening.
NLP, on the other hand, actually does have a vocabulary that matches the territory, but one that has been largely unexplored by experimental psychology, in much the way that hypnosis has had limited study. The catch in both is that you need a skilled operator to observe or produce many of the phenomena in question, because people differ in surface characteristics that have to be bypassed before you get to the similarities.
NLP's rep-systems and strategies models actually do have the necessary vocabulary and "behavioral caclulus" to discuss subjective experience, and in particular the parts needed to get past surface dissimilarities in processing.
I suggest "Neuro-linguistic Programming, Volume I", by Dilts et al, as an introduction for the theory-minded. A brief excerpt:
Another:
IOW, if you're looking for a vocabulary, run, don't walk, to get that book. It is generally considered the least-successful/popular book on NLP ever written, for precisely the same reason I'm recommending it to you: it's full of math, big words, and attempts at being precise.
(It is almost 30 years old, btw, so it shouldn't be considered the latest or greatest. There are a LOT of things in it that have been supplanted by more streamlined methods. However, the key underlying model of sensory representation strategy sequences (both in and out of consciousness) is just as valid today. There are just a lot more things known today about how we code things in those sensory represenations, and how to obtain information about them, install new representations, etc.)
This post was, to some extent, directed particularly at you. It would seem that you haven't taken my advice... I wish I knew of some good experimental results to back it up, as this would render it less ignorable.
What you're talking about above is not a concrete experimental result. Neither is it a standard causal theory, nor is it a causal theory that strikes me as particularly likely to be true in the absence of experimental validation. Nor is it valid math validly interpreted, or logic that seems necessarily true across lawful possible worlds. I don't care if it works for you and for other people you know; that doesn't show anything about the truth of the model; there's this thing called a placebo effect. The advice fails to meet the standard we're accustomed to, and that's why we're ignoring it. It is just one more theory on the Internet at this point, and one more set of orders delivered in a confident tone but not explained well enough to interpret at all, really.
I'm relieved to read this Eliezer, because I thought it was just me who perceived pjeby's advice as misguided.
I've been whining at him for a while, though my complaint isn't so much that his advice is misguided, as that he keeps offering pronouncements about how the mind works and how to make it work better, but evidence that his model and methods are sound seems sorely lacking (here, at least).
...much of which has come attached with things that are actually possible to investigate and test on your own, and a few people have actually posted comments describing their results, positive or negative. I've even pointed to bits of research that support various aspects of my models.
But if you're allergic to self-experimentation, have a strong aversion to considering the possibility that your actions aren't as rational as you'd like to think, or just don't want to stop and pay attention to what goes on in your head, non-verbally... then you really won't have anything useful to say about the validity or lack thereof of the model.
I think it's very interesting that so far, nobody has opposed anything I've said on the grounds that they tested it, and it didn't work.
What they've actually been saying is, they don't think it's right, or they don't think it will work, or that NLP has been invalidated, or ANYTHING at all other than: I tried thus-and-such using so-and-so procedure, and it appears that my results falsify this-or-that portion of the model you are proposing."
In a community of self-professed rationalists, I find that very interesting. Not as interesting, mind you, as I would an actual result falsifying a portion of my model, though.
Because that, I would actually LEARN something from. I could try and replicate the person's result, offer other things to try, or maybe even update my model. It does happen, pretty regularly -- and the updates are almost equally likely to come from:
A recent mainstream psych example would be Dweck's fixed/growth mindsets model, which I've now converted to a more specific model for change work that I call "or"/"more" thinking.
That is, a belief that "either I do this OR I fail" -- a digital control variable of avoidance -- is less useful than one where "the MORE I do this the more/closer I get": an analog variable under your control.
This is a much finer-grained distinction than my older notion that didn't include discrete/continuous, but focused strictly on the approach/avoidance aspect of the variables. It's also a more narrowly-focused understanding of the difference than Dweck's work, which speaks more about the effects of these mindsets than the mechanism of them, or how to change that mechanism in practice.
So now that I have this distinction, I've gone back and reviewed other things I've read that tie into this idea in one way or another, giving it more depth. That is, I can look at other discussions of "naturally successful" behavior, hypnotic techniques or NLP submodality techniques that link an increase in one thing to an increase in another, and so on.
In particular, I've found various techniques by Richard Bandler that describe how certain successful athletes and entertainers he worked with transformed "or" variables into "more" variables (although he didn't use those terms).
I'm now in the process of self-experimenting with some of those techniques, preparatory to selecting ones to add to my personal and training repertoire.
That, more or less is my method for model refinement: read about ideas, try ideas, figure out what works, update models, find relevant techniques, try techniques w/self, w/clients, get ideas about what other ideas might be worth investigating, rinse and repeat.
Is it "the scientific method". Probably not. Is it closer to the scientific method than the "I read something or believe something that means that won't work, but can't be bothered to tell whether it's the same thing" approach favored by some folks? Hell yeah.
Btw, that attitude is why every new self-help author or guru has to come up with new names for every damn thing: the old names get worn out by people who conclude they already "know" what that thing is, because their brother told them something about something like that once and it sounded kind of like something else they tried that didn't work.
Yet century-old techniques work fine, if you actually know how to do them, and you actually DO them. But surprisingly few people ever actually try, let alone try with all their might, in the "shut up and do the impossible" sense.
I'm allergic to self-experimentation. I find that I'm not a very good judge on my own reactions. Furthermore, self-experimentation is probably the worst way to go about setting up a true model of the world.
I am unable to make enough sense of what you say to try it. It is not written in a language I can read.
And that's not a criticism I have a problem with. Hell, if you actually tried something and it didn't work, and you gave me enough information to be able to tell what you did and what result you got instead , that would be excellent criticism, in my book.
Helpful criticism is helpful, and always welcomed, at least by me.
Why shouldn't you?
I don't understand. Why should I have a problem with Eliezer's criticism, or any considered criticism or honest opinion? It is only ignorant criticism and anti-applause lights that I have a problem with.
Well, that's ambiguity in interpretation of "having a problem with something". I (mis)interpreted your statement to mean "this kind of criticism doesn't bother me", that is you are not going to change anything in yourself in response, which would be unhealthy, whereas you seem to have intended it to say "this kind of criticism doesn't offend me".
So basically are you saying Eliezer, gjm and others are falling for the fallacy fallacy ?
When I read this, I get the same feeling as before, when you wrote about changing your ways in order to introduce your techniques to this forum. The feeling is that when you talk of rigor, you see it as a mere custom, something socially required, and quite amusing, really, since all that rigor can't be true, anyway. After all, it's only possible to make attempts at being precise, so who are you kidding. Plus, truth is irrelevant. And here we are, the LessWrong crowd, all for the image, none for the substance, bad for efficiency.
I wouldn't say that of everybody on LessWrong, but there is certainly a vocal contingent of that stripe. That contingent unfortunately also suffers from the use of cognitive models that, to me, are as primitive as the medieval four-humors model.
So when they push my "ignorance and superstition" buttons in the same posts where they're demanding properly validated rituals and papers for things they could verify for themselves in ten minutes by simple self-experimentation, it's rather difficult to take them seriously as "rationalists". (Especially when they go on to condemn theists for suffering from the same delusions as they are, just externally directed.)
I totally don't mind engaging with people who want to learn something and are willing to actually look at experience, instead of just talking about it and telling themselves they already know what works or what is likely to work, without actually trying it. The other people, I can't do a damn thing for.
If your interest is in "science", I can't help you. I'm not a scientist, and I'm not trying to increase the body of knowledge of science. Science is a movement; I'm interested in individuals. And individual rationalists ought to be able to figure things out for themselves, without needing the stamp of authority.
I also have no interest in being an authority -- the only authority that counts in any field is your own results.
This is why I hope that the next P. J. Eby starts out by first reading the OBLW sequences, and only then begins his explorations into akrasia and willpower.
You cannot verify anything by self-experimentation to nearly the same strength as by "properly validated rituals and papers". The control group is not there as impressive ritual. It is there because self-experimentation is genuinely unreliable.
I agree with Seth Roberts that self-experimentation can provide a suggestive source of anecdotal evidence in advance of doing the studies. It can tell you which studies to do. But in this case it would appear that formal studies were done and failed to back up the claims previously supported by self-experimentation. This is very, very bad. And it is also very common - the gold standard shows that introspection is not systematically trustworthy.
I'm a bit confused as to your goal, Eliezer.
Are you trying to find a fully general solution to the akraisia problem, applicable to any human currently alive… or do you want to know how you can overcome akrasia? The first is going to be a fair bit harder than the second, and you probably don't have time to do that and save the world.
If you shoot a little lower on this one and just try to find something that works for you I think your argument will change… quite a lot.
If you think that's the case, you didn't read the whole Wikipedia page on that, or the cite I gave to a 2001 paper that independently re-creates a portion of NLP's model of emotional physiology. I've seen more than one other peer-reviewed paper in the past that's recreated some portion of "NLP, Volume I", as in, a new experimental result that supports a portion of the NLP model.
Hell, hyperbolic discounting using the visual representation system was explained by NLP submodalities research two decades ago, for crying out loud. And the somatic marker hypothesis is at the very core of NLP. Affective asynchrony? See discussions of "incongruence" and "anchor collapsing" in NLP vI, which demonstrate and explain the existence of duality of affect.
IOW, none of the real research validation of NLP has the letters "N-L-P" on it .
Unreliable for what purpose? I would think that for any individual's purpose, self-experimentation is the ONLY standard that counts... it's of no value to me if a medicine is statistically proven to work 99% of the time, if it doesn't work for ME.
Unreliable for getting true explanations. Self-experimentation is generally too poorly controlled to give unconfounded data about what really caused a result. (Also, typically sample size is too small to justify generalizability.)
The way I'd put it for this stuff is that experiments help communicate why someone would try a technique, they help people distinguish signal from noise, because there are a ton of people out there saying X works for me.
This sounds like being uninterested in the chances of winning a lottery, since the only thing that matters is whether the lottery will be won by ME, and it costs only a buck to try (perform a self-experiment).
And yet, this sort of thinking produces people who get better results in life, generally. Successful people know they benefit from learning to do one more useful thing than the other guy, so it doesn't matter if they try fifty things and 49 of them don't work, whether those fifty things are in the same book or different books, because the payoff of something that works is (generally speaking) forever.
Success in learning, IOW, is a black-swan strategy: mostly you lose, and occasionally you win big. But I don't see anybody arguing that black swan strategies are mathematically equivalent to playing the lottery.
IMO, the rational strategy is to try things that might work better, knowing that they might fail, yet trying to your utmost to take them seriously and make them work. Hell, I even read "Dianetics" once, or tried to. I got a third of the way through that huge tome before I concluded that it was just a giant hypnotic induction via boredom. (Things I read later about Scientology's use of the book seem to actually support this hypothesis.)
This became infeasible with the invention of printing press. There is too much stuff out there, for any given person to learn. Or to ever see all the titles of the stuff that exists. Or the names of the fields for which it's written. There is too much science, and even more nonsense. You can't just tell "read everything". It's physically impossible.
P.S. See this disclaimer, on second thought I connotationally disagree with this comment.
What happened to "Shut up and do the impossible"? ;-)
More seriously, what difference does it make? The winning attitude is not that you have to read everything, it's that if you find one useful thing every now and then that improves your status quo, you already win.
Also, when it comes to self-help, you're in luck -- the number of actually different methods that exist is fairly small, but they are infinitely repeated over and over again in different books, using different language.
My personal sorting tool of choice is looking for specificity of language: techniques that are described in as much sensory-oriented, "near" language as possible, with a minimum of abstraction. I also don't bother evaluating things that don't make claims that would offer an improvement over anything else I've tried, and I have a preference for reading authors who've offered insightful models and useful techniques in the past.
Lately, I've gotten over my snobbish tendency to avoid authors who write things I know or suspect aren't true (e.g. stupid quantum mechanics interpretations); I've realized that it just doesn't have as much to do with whether they will actually have something useful to say, as I used to think it did.
PJ, is there a survey / summary / list of these methods online? Could you please link, or, if there's no such survey, summarize the methods briefly?
You keep using that phrase. I do not think it means what you think it does.
Viva randomness! At least it's better than stupidity. And is about as effective as reversed stupidity. Which is not intelligence.
You should know better what you need, what's good for you, than a random number generator. And you should work on your field of study being better than a procedure for crafting another random option for such a random choice. I wonder how long it'll take to stumble on success if you use a hypothetical "buy a random popular book" order option on Amazon.
P.S. See this disclaimer, on second thought I connotationally disagree with this comment.
Okay. Another take. Is this really true? How long would it take for a new-commer to walk through every available option? How much would it cost? What is the chance he should expect before starting the whole endeavor that any of the available options will help? For the last question, the lottery analogy fits perfectly, no "works only for ME" excuse.
The plural of anecdote is not data. Many people will tell you how they were cured by faith healers or other quacks, and, indeed, they had problems that went away after being "treated" by the quack. Does that make the quacks effective or give credibility to their theories about the human body?
The same applies to methods of affecting the human brain. As a non-expert, from the outside I can't tell the difference between NLP, Freudian psychotherapy, and whatever hocus-pocus Scientology says helps people. All have elaborate theories to explain their alleged benefits, and all have had people who swear it works.
To quote Wikipedia:
Until I do see some acceptance among the academic community, I remain unconvinced that NLP is anything more than a self-reinforcing collection of hypotheses, speculation, and metaphors. It could very well be otherwise, but I can't know that it isn't!
Few of your comments here seem to me to describe things that are obviously checkable in ten minutes by simple self-experimentation. (Even ignoring the severe unreliability of self-experimentation, since doubtless there are at least some instances in which self-experimentation can provide substantial evidence.) Perhaps they are so checkable with the help of extra information that you've declined to provide. Perhaps I've just not read the right comments. Perhaps I've read the right comments and forgotten them. Would you care to clarify?
Mostly, I've offered questions that people could ask themselves in relation to specific procrastination scenarios, that would give them an insight into the process of how they're doing it. IIRC, two people have reported back with positive hits; one of the two also had a second scenario, for which my first question did not produce a result, but it's not clear yet what the answer to my second question was. (I gave both questions up front, along with the sequence to use them in, and criteria for determining whether an answer was "near" or "far", along with instructions to reject the "far" answers. One respondent gave a "far" answer, so I asked them to repeat.)
I've also linked to a video offering a simple motivational technique based on my model; a few people have posted positive comments here, and I've also gotten a number of private emails from users here via the feedback form on my site, expressing gratitude for its usefulness to them. The video is just about 10 minutes long.
In another comment, I described a simple NLP submodalities exercise that could be tried in a few minutes, albeit with the disclaimer that some people find it hard to consciously observe or manipulate submodalities directly. (The technique in my video is a bit more indirect, and designed to avoid conscious interference in the less-conscious aspects of the process.)
I've referenced various books on other techniques I've used, and I believe I even mentioned that Byron Katie's site at thework.org includes a free 20-page excerpt from Loving What Is that provides instructions for a testable technique that operates on the same fundamental basis as my models.
I'm really not sure what the heck else people want. Even if you claim, as Eliezer does, that he can't understand my writing, it's not like I haven't referenced plenty of other people's writing, and even my spoken language (in the video) as alternative options.
I also find your writing difficult. If you'll accept a recommendation, I think your readership here might get more from shorter comments in which more work has gone into each word.
So, I watched the video (some time ago, when you posted about it) and gave it one trial. The technique wasn't effective for me on the task I tried it on. The particular failure mode was one you mentioned in the video, and if you are correct about the generality with which it makes the technique not work then I would expect the technique to be generally ineffective for the things I'd benefit from motivational help with.
Your suggestions about identifying the causes of procrastination: I haven't tried that yet, and it sounds interesting; I notice that when someone did try it and got results that didn't perfectly match your theory your immediate response was not "oh, that's interesting; perhaps my theory needs some tweaking" but "I don't believe you". Can you see how this might make people skeptical?
Referencing books is only helpful in so far as (1) it's not necessary to read the whole of a lengthy book to extract the small piece of information you've been asked for, (2) the book is clearly credible, and (3) the book is actually available (e.g., in lots of libraries, or inexpensive, or online). To those who are skeptical about the whole self-help business, #2 is a pretty difficult criterion to meet.
Indeed. It is supposed to be a free sample, after all. The work I charge for is fixing those things that make it not work. The things that make motivation not work are much, much more diverse than the things that make it actually work.
My response was, "you didn't follow directions", actually. Unless you're talking about the first part where the only information given was, "it didn't work". If you've ever done software tech support, you already know that "it didn't work" is not a well-formed answer. (Similarly, the later answer given was also not well-formed, by the criteria I laid out in advance.)
Failure to meet entry criterion for a technique does not constitute failure of the technique or the model: if you build a plane without an engine, and it doesn't take off, this does not represent a failure of aerodynamics. Indeed, aerodynamics predicts that failure mode, and so did I.
The response I got was not unexpected; it's common for people to have trouble at first, especially on things they don't want to look too closely at. I've had people spend up to 30 minutes in the "talk around the problem" failure mode before they could actually look at what they were thinking. The other most common failure mode is that somebody does see or hear something, but rejects it as nonsensical or irrelevant, then reports that they didn't get anything.
Third most common failure mode is lack of body awareness or physical suppression, but I know he doesn't have that as a general problem because his first response indicated awareness. His first response also indicated he is capable of perceiving responses, so that pretty much narrows it down to avoidance or assumption of irrelevance . If it's neither, then it might be relevant to a model update, especially if it's a repeatable result.
(At this point, however, he's going to have to repeat the asking of the second question, to test that, though, because these responses don't stick in long-term memory; in a sense, they are long-term memory.)
I think this (not the fact that it's a free sample, but the fact that apparently it's a feature, not a bug, if it doesn't work well for many people) makes it rather unuseful as a try-it-yourself demonstration of how good your models and techniques are.
There was no such first part; even jimrandomh's initial response had more information than that in it. And after he gave more information your reply was still "I don't believe you" rather than "you didn't follow directions". Interested parties can check the thread for themselves.
No, to be sure. But once you hedge your description of your technique and what it's supposed to achieve with so many qualifications -- once you say, in so many words, that you expect it not to work when tried -- how can it possibly be reasonable for you to use it as an example of how you've supplied us with empirically testable evidence for what you say?
Saying "You can check my ideas by trying this technique -- but of course it's quite likely not to work" is just like saying "You can check my belief in God by praying to him for a miracle -- but of course he works in mysterious ways and often says no."
The point of the exercise is that it's targeted to work for as many people as possible for a fairly narrow range of tasks, so as to give a sample of what it's like when it works.
Even chronic procrastinators can achieve success with the technique, as long as they don't use it on the thing they're procrastinating on -- it only works if you don't distract yourself with other thoughts, and if you're stressed about something, you're probably going to distract yourself with other thoughts.
Most people, however, don't seem to have any significant stressors about cleaning their desk. Also, it's not a difficult thing to visualize in its completed form.
Btw, just as a datapoint, what did you try it on, and what failure mode did you encounter? I am, ironically, MORE interested in failure reports than successes; the video continually gets rave reviews, but as much as I enjoy them, I can't learn anything new from another success report!
I just rechecked myself; here are the relevant portions. Jim said:
I took this statement as a literal description of what happened, i.e., jim thought about "it" -- whatever "it" was -- got no physical response, and had thoughts about the details of the task. THEN (2nd step) he was unable to begin working on it.
"Unable to begin working on it" is the part I referred to as not well-formed; this does not contain any description of how he arrived at that conclusion. It is the equivalent of "it doesn't work" in tech support.
The unspecified "it" is also potentially relevant; I don't know if he refers there to the task itself, or one of the questions I said to ask about the task; and this is an important distinction. I've also noticed that some people can "think about their task" and not get a response because they are not thinking about actually starting on the task... and Jim's statements would be consistent with a sequence of thinking about the idea of the task, followed by preparing to actually perform the task... at which point an undescribed response is occurring, whereby he is then "unable to" perform the task.
I commented on the conflict between these two statements:
Meaning: as far as I can tell, those statements are not talking about the same thing. I.e., one is a referent to some sort of pre-task preparation unrelated to the problem, and the other is actually about beginning it.
In other words: all the information was in the first sentence, but the second one is where the problem actually is. So I then asked Jim to direct his attention to that part of his thought process, and get more specific:
He then replied with two more not well-formed statements; instead of describing his thoughts or experiences, he replied with abstract, "far" explanations about the subject matter, instead of his direct response to the subject matter, i.e.:
and:
Neither of these utterances describes a concrete experience; they are verbalizations of precisely the kind I described in the "how to know if you're making shit up" comment beforehand. They are far, not near thinking, and my techniques only use far thinking to ask questions, and determine what questions to ask. The answers sought, however, are exclusively "near".
Thus, when someone replies with a "far" answer, I know that they have not actually answered my question or followed instructions - they are not using the part of their brain that will produce the desired result.
Notice, by the way, that at no time did I say I did not believe him. I took him quite literally at his word, to the extent that he gave me words that map to some sort of experience.
I tried it on the same example you proposed: desk-clearing. My desk is a mess; I would quite like it to be less of a mess; clearing it is never a high enough priority to make it happen. But I don't react to the thought of a clear desk with the "Mmmmmm..." response that you say is necessary for the technique to work.
As for your discussion with Jim: you did not at any point tell him that he didn't do what you'd told him to, or say anything that implied that; you did say that you think his statements contradict one another (implication: at least one of them is false; implication: you do not believe him). And then when he claimed that what stopped him was apathy and down-prioritizing by "the attention-allocating part of my brain" you told him that that wasn't really an answer, and your justification for that was that his brain doesn't really work in the way he said (implication: what he said was false; aliter, you didn't believe him).
So although you didn't use the words "I don't believe him", you did tell him that what he said couldn't be correct.
Incidentally, I find your usage of the word "incompatible" as described here so bizarre that it's hard not to see it as a rationalization aimed at avoiding admitting that you told jimrandomh he'd contradicted himself when in fact all he'd done was to say two things that couldn't both be true if your model of his mind is correct. However, I'll take your word for it that you really meant what you say you meant, and suggest that when you're using a word in so nonstandard a way you might do well to say so at the time.
Can you link to these things? Your comments? Here? There's an LW search box.
How To Tell If You're Making Shit Up
NLP Submodalities Experiment
Motivation technique video
Edit to add:
"How To Tell If You're Making Shit Up" seems useful. Do you see why this would seem useful to me while "NLP Submodalities" doesn't?
For the same reason that yours and Robin's writing on biases is more useful than the source material, I imagine. That is, it's been predigested. It probably also doesn't hurt that I have to teach "how to tell if you're making shit up" to every single client of mine, so I have some practice at doing so! (Albeit mostly in real-time interaction.)
FYI, NLP volume I represents the more detailed "brain software" model from which that summary was derived, which I recommended to you because you said you couldn't follow my writing.
You can also see why I was excited when Robin started posting about near/far stuff on OB -- it fit very nicely into the work I was already doing, and into the NLP presupposition that "conscious verbal responses are to be treated as unsubstantiated rumor unless confirmed by unconscious nonverbal response" -- i.e., don't trust what somebody says about their behavior, because that's not the system that runs the behavior.
The Near/far distinction mainly added an evolutionary explanation that was not a part of NLP, and gave a better why for not trusting the verbal explanation. Near/far in a literal sense, as in "people respond differently based on distance in space/time/abstraction level of visualization", has been part of the NLP models for over 20 years now. But once again, the mainstream experiments are just now being done, presumably by people who've never heard of NLP, or who assume it's crackpottery.
deleted