All of localdeity's Comments + Replies

It's entirely possible for a highly intelligent person to follow a strategy that is completely fucking idiotic.  Common, in fact.

How different is that from them writing a book about congestion pricing and you reading it? In both scenarios you are basically consuming the outputs of this author's mind.

Well, one difference is that, if you have a question about something the book hasn't specifically addressed in your reading thus far ("How does your theory apply to hotels used by the eclipse-watching crowds in 2017?"), in the first scenario you can just ask the author, but in the second case, after reading the relevant section of the book (and perhaps checking neighboring sections, the ... (read more)

2Adam Zerner
That makes sense. Although I think the larger point I was making still stands: that in reading the book you're primarily consuming someone else's thoughts, just like you would be if the author sat there on the bench lecturing you (it'd be different if it were more of a two-way conversation; I should have clarified that in the post). I suppose "primarily" isn't true for all readers, for all books. Perhaps some readers go slowly enough where they actually spend more of their time contemplating than they do reading, but I get the sense that that is pretty rare.

And in Childhoods of exceptional people, the author finds that immersion in boredom was a universal experience:

But this immersion in boredom is also a universal in the biographies of exceptional people. A substantial fraction were completely kept apart from other children, either because their guardians decided so or because they were bedridden with various illnesses during childhood (like Descartes). A spicy hypothesis raised by this is that socializing too much with children is simply not good for your intellectual development. (I’m not going to test tha

... (read more)
2Adam Zerner
That makes sense. I didn't mean to imply that such an extreme degree of isolation is a net positive. I don't think it is.

What is solitude?

I have thoughts about "loneliness", a related concept:

  • There are many kinds of social interaction that you can want, need, or benefit from.  (If we need to distinguish a "want" from a "need", I'd say a "need" is something that causes more negative effects than simple frustration if it's not satisfied.)
    • Interaction types that probably register as a "need" to at least some people: seeing human faces; having fun with friends; discussing your problems with someone; intellectual stimulation; sexual and romantic activities; probably more.
      • I'm
... (read more)
2Adam Zerner
Hm. On the one hand, I agree that there are distinct things at play here and share the instinct that it'd be appropriate to have different words for these different things. But on the other hand, I'm not sure if the different words should fall under the umbrella of solitude, like "romantic solitude" and "seeing human faces solitude". I dunno, maybe it should. After all, it seems that in different conceptualizations of solitude, it's about being isolated from something (others' minds, others' physical presence). Ultimately, I'm trusting Newport here. I think highly of him and know that he's read a lot of relevant literature. At the same time, I still wouldn't argue too confidently that his preferred definition is the most useful one.

Just because the average person disapproves of a protest tactic doesn't mean that the tactic doesn’t work. Roger Hallam's "Designing the Revolution" series outlines the thought process underlying disruptive actions like the infamous soup-throwing protests.  Reasonable people may disagree (I disagree with quite a few things he says), but if you don't know the arguments, any objection is going to miss the point.  To be clear, PauseAI does not endorse or engage in disruptive civil disobedience, but I discuss it here to illustrate some broa

... (read more)
1WillPetillo
Thanks for the link!  It's important to distinguish here between: (1) support for the movement,  (2) support for the cause, and (3) active support for the movement (i.e. attracting other activists to show up at future demonstrations) Most of the paper focuses on 1, and also on activist's beliefs about the impact of their actions.  I am more interested in 2 and 3.  To be fair, the paper gives some evidence for detrimental impacts on 2 in the Trump example.  It's not clear, however, whether the nature of the cause matters here.  Support for Trump is highly polarized and entangled with culture, whereas global warming (Hallam's cause) and AI risk (PauseAI's) have relatively broad but frustratingly lukewarm public support.  There are also many other factors when looking past short-term onlooker sentiment to the larger question of affecting social change, which the paper readily admits in the Discussion section.  I'd list these points, but they largely overlap with the points I made in my post...though it was interesting to see how much was speculative.  More research is needed. In any case, I bring up the extreme case to illustrate that the issue is far more nuanced than "regular people get squeamish--net negative!"  This is actually somewhat irrelevant to PauseAI in particular, because most of our actions are around public education and lobbying, and even the protests are legal and non-disruptive.  I've been in two myself and have seen nothing but positive sentiment from onlookers (with the exception of the occasional "good luck with that!" snark).  The hard part with all of these is getting people to show up.  (This last paragraph is not a rebuttal to anything you have said, it's a reminder of context)

I would also comment that, if the environment was so chaotic that roughly everything important to life could not be modeled—if general-purpose modeling ability was basically useless—then life would not have evolved that ability, and "intelligent life" probably wouldn't exist.

You have a point, although I don't think having a genuine feeling of despair is a hugely important variable.  As the story goes:

[Method actor Dustin] Hoffman had a grueling scene coming up, where his character hadn’t slept in three days, and Hoffman told [Sir Laurence] Olivier that to prepare for the scene, he too hadn’t slept for 72 hours.

“My dear boy,” replied Olivier, “why don’t you try acting?”

But more generally, even if you want to teach some kind of set of skills and resilience for dealing with things like "sitting still for hours a day", "doing... (read more)

Is this a case for or against formal education? Either way, it is wise.

If one does accept the premise that feigning enthusiasm is a useful skill, that's still not a good justification of formal education as it exists: it certainly doesn't take 12 years of grade school to teach that skill.

4TsviBT
No, the depth of the despair does mostly strictly+monotonically increase, so you do continually unlock new levels of the skill.

Epistemic status: this is an attempt to steelman the case for the death penalty

...

I do not believe in vengeance or justice. I do however believe in fixing problems. And it's clear the only way to fix this problem is to put such people in positions where they cannot do anyone any harm.

Some people have complained that, when their opponents "steelman" their position, in practice it can mean they steelman a particular argument that is not their main argument.  This struck me as a remarkably explicit and self-aware example of that.

I don't know what the sol... (read more)

4Viliam
Steelmanning is not the same as passing the ideological Turing test. ITT is successful when your opponent agrees with you, or when your opponent cannot distinguish you from their actual allies. Steelman is successful if you, or your allies, find something useful in the ideas of your opponent. Whether your opponent approved of the result, or not. In ITT, your opponent is the judge (of how much it passes). With a steelman, you are the judge (of whether you have extracted something useful). When steelmanning, you cherry-pick the good parts, and discard the rest. When passing an ITT, you need to pass all checks. For example, an ITT of a religion is... speaking like a true believer. A steelman of religion is e.g. saying that we don't really know how the universe came to existence, and that there are some social benefits of religion.
4Richard_Kennaway
Steelmanning is writing retcon fanfiction of your interlocutor’s arguments. As such it necessarily adds, omits, or changes elements of the source material, in ways that the other person need not accept as a valid statement of their views.

I personally recommend that all parents donate to the Localdeity Enrichment Fund, an important yet frequently overlooked cause area.

Whoever wrote that article is confused, since in the table in the section labeled "Analogy vs Simile: The Differences" they have several entries the wrong way around (compared to the two paragraphs preceding it).

It seems to me that you could use the same comparison for either an analogy or a simile.  An analogy would usually be in the present tense, "X is like Y", and followed by more explanation of the concept the analogy is meant to illustrate.  A simile would more frequently be in the past tense as part of a narrative, and more frequently use ... (read more)

The term "privilege" is bad here; prefer "advantage".  "Privilege"—privi-lege, private law—implies that there's an authority deciding to grant it to some people and not others, which would be unjust (since most things that affect intelligence, such as genetics and childhood nutrition, happen long before a person does anything to "deserve" it more than others), which in turn encourages people to get angry and suspicious, and encourages the advantaged to feel embarrassed or even guilty by association when they've done nothing wrong.  Calling it "pr... (read more)

Other reasons:

Biases towards claiming agreement with one’s own beliefs

If the institution is widely trusted, respected, high status, etc., as well as powerful, then if Alice convinces you that the institution supports her beliefs, then you might be inclined to give more credence to Alice's beliefs.  That would serve Alice's political agenda.

Weaker biases towards claiming disagreement with one’s own beliefs

If the institution is widely hated—for example al-Qaeda, the CIA, the KGB—or considered low status, crazy, and so on, then if Alice convinces you tha... (read more)

In Judaism, you're not supposed to marry a non-Jew unless they convert to Judaism (a lengthy process from what I've heard), so I suspect the families on both sides of the deal are usually equally religious.

In any case, googling for "grief and genetic closeness study" yields this:

A Twin Loss Survey was completed by MZ and same-sex DZ twins following loss of a cotwin and nontwin relatives. Twin survivors (N = 612; MZ = 506; DZ, n = 106) included twins whose age at loss was 15 years or older. Participation age was M = 47.66 years (SD = 15.31). Hamilton's incl

... (read more)
2Kaj_Sotala
Twins grieving more strongly for deceased co-twins seems to me explained by twins having a more closely coupled history than non-twin relatives. MZ twins grieving more strongly than DZ twins seems to me explained by MZ having larger similarity in personality so bonding more strongly due to that.
2ChristianKl
From Gemini Pro 2.0:

Everything you say seems straightforwardly correct or a logical guess.  I'd add:

  • I expect identical twin sisters do feel closer to their nieces than non-identical sisters feel to their nieces.  There would probably be a higher degree of discovering that this person just happens to have traits like your own.
  • Even from a fully logical "optimize my inclusive genetic fitness" perspective, there is value, all else being equal, in most parent-child relationships that isn't in most aunt-niece relationships.  Because you raised this child, she probabl
... (read more)
9PaulBecon
Unless the identical twin sisters marry identical brother twins, the preferential love for their own husband would distinguish their feelings for their own children

I think there's at least decent truth to it.  One study:

This study examines gift giving at Israeli weddings. In accordance with kin selection theory, we hypothesized that wedding guests possessing greater genetic relatedness to the newlyweds would offer greater sums of money as wedding gifts. We also hypothesized that family members stemming from the maternal side (where the genetic lineage has higher kinship certainty) would offer the newlyweds more money than those stemming from the paternal side. Data on the monetary gift sums of the wedding guests

... (read more)
6Kaj_Sotala
30 weddings in one particular culture doesn't sound like a particularly representative sample. I would expect that in formal situations like weddings, social norms and expectations would determine gift-giving as strongly if not more strongly than genuine liking. And "closer relatives should give more generous gifts than more distant ones" sounds like a pretty natural social norm. With regard to those other studies, I don't think you can conclude anything from just a relatedness-grief correlation. As you note yourself, there's also a relatedness-closeness correlation, so we should expect a relatedness-grief correlation even in worlds with no genetic effect. There's also a cultural mechanism where you are expected to feel grief when people related to you die. And none of these studies establish a mechanism for how the effect is supposed to work. There are some simple and straightforward mechanisms for establishing closeness with close relatives - e.g. "you grow to care about your parents who you have known for as long you can remember", "you grow to care about children that you personally gave birth to", "you grow to care about people you spend a lot of time with", etc..  But in the case of a cousin who you might never have met, by what mechanism is evolution going to get you to care about them? Before the invention of DNA testing, the only evidence for them being related to you was someone claiming that they are your cousin. And if that was enough to win over someone's trust, we'd expect there to be a lot more con schemes that tried to establish that the con artist was the mark'a long-lost cousin (or even better, sibling).
4ChristianKl
Taking one study about how much wedding gifts come from each side in one specific culture of Israeli weddings, seems very bad reasoning. Depending of the economics of marriage, wedding gifts differ from culture to culture.  In Judaism, religion passes primarily through the maternal lineage by cultural custom, so there are a lot of other reasons besides kinship certainty. 

I think the hemisphere stuff is quite literal.  I think it's general knowledge that the right eye feeds into the left side of the brain, and vice versa (Actually, looking it up, it is the case that the left is controlled by the right and vice versa, but I see some claims that the information feeds into both sides, in a nearly balanced manner[1]; but I don't know if Ziz knows that); and Ziz's whole "unihemispheric sleep" thing tells you to keep one eye closed and distract the other eye so that eventually one hemisphere falls asleep.

  1. ^

    Claude sez: "When n

... (read more)
1Milan W
This ""unihemispheric sleep" thing seems like it came from crazy and is an excellent way to produce even more crazy. A tale as old as time: small group of people produce some interesting ideas and all is mostly fine until they either take too many drugs or get the bright idea of intentionally messing up their sleep. This starts a self reinforcing loop of drugs / messed up sleep causing crazy causing drugs / messed up sleep causing even more crazy.

It's not about the eyes, it's about the part of the visual field.

The image from the right half of the visual field (left part of each retina) feeds into the left hemisphere and the image from the left half of the visual field (right part of each retina) feeds into the right hemisphere.

Since in humans each eye observes both sides of the visual field, you need to have ~50% of each eye's fibers (each corresponding to something like a pixel) to go to each hemisphere.

In vertebrates where the overlap in visual fields of each eye is minimal (e.g. horses, rabbits)... (read more)

BTW, on Ziz's obituary someone wrote:

Like Jesus, he will arise from the dead.

not sure if sincere or trolling...

The date on that comment is Jan 30 2025.  Methinks 90% likelihood it's causally downstream from the recent murders and that the poster knows Ziz was never dead.

If you would use genetic studies to guide clinical trial representation for a drug to combat heart disease you would look at the genes associated with heart disease and see that mutations in those genes are evenly distributed in your clinical trial representation. You would not focus on the race with which people self-identify. 

I asked Claude a few questions.  I'll just give snippets of the answers:

  1. Are there scenarios where doctors recommend different doses of a medication, or other variations in a medical plan, based on a patient's race?
    1. Yes. &nb
... (read more)
6ChristianKl
For background, I'm coming from Germany, where our liberals protest when they ask asylum seekers for their ethnic identity to find out whether they are discriminated in their homeland for their ethnic identity, because we believe treating people different based on ethnic identity is wrong.  Obese people show different effects to all sorts of clinical interventions compared to people that are underweight. Yet, the FDA makes no attempt to have a representative sample of obese people and underweight people in their clinical trials. When Big Pharma companies recruit patients for clinical trials, they don't try to a representative population when it comes to weight. In many cases they have a hypothesis that their drug will be more effective if the trial population is based to have less preconditions and then they recruit a clinical trial population that's biased by design.  Clinical trials have trial populations that are sized to find clinically significant effects in the total trial population. If you have a clinical trial that sized to find an effect in the general population but only have that effect on Native Americans or on Black people, you are unlikely to find a statistical significant effect if you have Native Americans and Black people at their normal representation of the population.  The way doses for antidepressants and antipsychotics are chosen in clinical practice is often that you start with the lowest dose and change the dose for a single patient till the dose is good for the patient.  It's also worth noting that genetic differences between different populations in Africa itself are higher than genetic differences between Whites and Asians. When people like Elizabeth Warren identify as Native Americans even when they are genetically mostly White, it's not very safe that you have a good idea of someone genetics from their racial self-identification. There's a reason why self-studies don't ask whether someone is gay but asks for whether they belong to t

@Friendly Monkey , I'm replying to your reaction:

There are people who require multiple methods of persuasion before they act in the way you want.  One category is decisionmakers for an organization, who have actually been persuaded by intimidation, but they can't just say that, because they would look weak and possibly as though they're defecting against the organization or its aims, so they need to sell it as some high-minded decision they've come to of their own accord.  Or it could be the reverse: decisionmakers who are persuaded by your ideol... (read more)

"A random walk, in retrospect, looks like like directional movement at a speed of .

The average distance from the starting point is close to  after n random steps (in 1 dimension).  But I'd characterize that as a speed of .  Or you could say "... looks like a directional movement of distance ".

4lsusr
You are correct. I have fixed the math. Thank you!

I expect that's about not trusting the foreigners who did the clinical trials (I have heard this)[1], and not so much about expecting that Americans are biologically different from foreigners.

  1. ^

    Specifically, someone with some knowledge told me that the FDA knows that there are some countries where the trials are completely untrustworthy.  And that there was a political decision where they said, "If we disallow trials from some countries but not others, there will be much complaining and we'll probably be called racists", and solved the issue by disallo

... (read more)

Let's see what the base rate for murder is.  After some googling... Since the "clearance rate" for homicides is 50% (as of a recent year), even if we know there were N murders, it's hard to say how many distinct murderers there were.  But some source says it's a small percentage of murderers who kill again, so let's just assume that N murders = N murderers.  Both "taking the homicide rate (7.5 per 100k per year in 2022) and multiplying by a lifetime (we'll say 80 years)", and "googling for the percentage of deaths that are caused by homicide... (read more)

eukaryote139

That's definitely a good point and model vis-a-vis "this group/ideology is targeting these people specifically".

I would also point out that specifically rejecting demographically-vulnerable people is likely to push more of them towards this ideology - though even if that effect weren't in play, it would still be shitty to tarnish a broad group of generally fine community members by common demographic.

I'd point out that the magnitude of the "exploitation" is the magnitude of the incentive for market players to find the better solution.  If Bob is the one guy for whom making him wear a tuxedo isn't worthwhile, and if it's close to worthwhile—e.g. him wearing it produces $4000 of value to the company over the time he wears it—then that's $1000 being left on the table.  If there are 100 employees being "exploited" like Bob, for whom making them wear tuxedos is extremely wasteful—say it produces only $100 of value for them to wear tuxedos—then the... (read more)

3Viliam
The problem with words like "better" and "left on the table" is that better globally is not necessarily better for me, and leaving value on the table is not a problem for me if it all comes from your part. Option A: I get 5 units of utility, you get 5 units of utility. Option B: I get 6 units of utility, you get 3 units of utility, 1 unit of utility gets burned. Option A is clearly better for me out of these two. It seems like there should be an option C: 6.5 units of utility for me, 3.5 units of utility for you. But maybe there isn't . Maybe it actually costs me 1 unit of utility to keep a gun pointing at your head. And without the gun, you might turn the table and demand 5 units of utility or bust. In theory, if you were a robot, you could commit to act as if the gun is pointed at your head, even if there is none, and then we could split the money I saved by not having to buy the gun. In practice, humans don't seem to work that way sufficiently reliably.

Some related scenarios are discussed in my post here, e.g. when popularity ≈ beauty + substance, and if popularity and beauty are readily apparent then you can estimate substance.

1StartAtTheEnd
That's basically the exact same idea I came up with! Your link says popularity ≈ beauty + substance, that's no different than my example of "success of a species = quality of offspring + quantity of offspring". I just generalized to a higher number of dimensions, such that for a space of N dimensions, the success of a person is the area spanned. So it's like these stat circles but n-dimensional where n is the number of traits of the person in question. I don't know if traits are best judged when multiplied or added together, but one could play around with either idea. I'm not sure my insights say anything that you haven't already, but what I wanted to share is that you might be able to improve yourself by observing unsuccessful people and copying their trait in the dimension where you're lacking (this was voted 'wrong' above but I'm not sure why). And that if you want success, mimicking the actions of somebody who is ugly should be more effective, and this is rather unintuitive and amusing. I also think it would be an advantage for an attractive person to experience what it's like not to be attractive for a while, getting used to this, and then becoming attractive again. Since he would have to make up for a deficit (he's forced to improve himself) and then when the advantage comes back, he'd be further than if he never had the period of being unattractive. And as is often the case with intelligent people, I never really had to study in school, but this made me unable to develop proper study habits. If I learned how below-average people made it through university, this would likely help me more than even observing the best performing student in my class. A related insight is that if you want a good solution, you have to solve a worse problem. Want a good jacket for the cold weather? Find out what brands they use on Greenland, those are good, they have to be. Want to get rid of a headache? Don't Google "headache reliefs", instead, find out what people with migrain

There had been a study comparing the effects of moderate protests vs extreme protests, in hypothetical situations (the study participants would e.g. read a fictional article describing the activities of a protest group), and concluded that "Extreme Protest Actions Reduce Popular Support for Social Movements":

How do protest actions impact public support for social movements? Here we test the claim that extreme protest actions—protest behaviors perceived to be harmful to others, highly disruptive, or both—typically reduce support for social movements. Across

... (read more)

Smart people are often too arrogant and proud, and know too much.

I thought that might be the case.  If you looked at GPT-3 or 3.5, then, the higher the quality of your own work, the less helpful (and, potentially, the more destructive and disruptive) it is to substitute in the LLM's work; so higher IQ in these early years of LLMs may correlate with dismissing them and having little experience using them.

But this is a temporary effect.  Those who initially dismissed LLMs will eventually come round; and, among younger people, especially as LLMs get... (read more)

Using LLMs is an intellectual skill.  I would be astonished if IQ was not pretty helpful for that.

For editing adults, it is a good point that lots of them might find a personality tweak very useful, and e.g. if it gave them a big bump in motivation, that would likely be worth more than, say, 5-10 IQ points.  An adult is in a good position to tell what's the delta between their current personality and what might be ideal for their situation.

Deliberately tweaking personality does raise some "dual use" issues.  Is there a set of genes that make... (read more)

gwern*154

Using LLMs is an intellectual skill. I would be astonished if IQ was not pretty helpful for that.

I don't think it is all that helpful, adjusting for the tasks that people do, after years of watching people use LLMs. Smart people are often too arrogant and proud, and know too much. "It's just a pile of matrix multiplications and a very complicated if function and therefore can't do anything" is the sort of thing only a smart person can convince themselves, where a dumb person thinking "I ask the little man in the magic box my questions and I get the rig... (read more)

On a quick skim, an element that seems to be missing is that having emotions which cause you to behave 'irrationally' can in fact be beneficial from a rational perspective.

For example, if everyone knows that, when someone does you a favor, you'll feel obligated to find some way to repay them, and when someone injures you, you'll feel driven to inflict vengeance upon them even at great cost to yourself—if everyone knows this about you, then they'll be more likely to do you favors and less likely to injure you, and your expected payoffs are probably higher t... (read more)

3Larry Lee
Thanks for the feedback @localdeity. I agree that my article could be read as implying that emotions are inherently irrational and that, from an evolutionary perspective, emotions have underlying logics (for instance anger likely exists to ensure that we enforce our social boundaries against transgression). This reading does not reflect my views however. My scheme follows decision theory by assuming that we can assign an "objective" utility value to each action/option. This utility value should encompass everything - including whatever benefits may be reflected in the logics underlying emotions. Thus, there shouldn't be any benefit that emotions provide that is not included in these utility values. There are times when our emotions are aligned with those actions that maximize expected utility, but this is not guaranteed.  Whenever an emotion goads us to act in line with utility maximization we can call that emotion "rational." When the emotion spurs us to act in a way that conflicts with our best interest (all things considered), we can call that emotion "irrational". My goal in this article was not to argue that emotions are fundamentally irrational. Emotions operate according to their own internal rules. These rules are more akin to pattern-response than to the sorts of calculations prescribed by decision theory. This article tries to integrate these effects into decision theory to create a model of human behavior that is qualitatively more accurate. 

What is categorized as "peer pressure" here?  Explicit threats to report you to authorities if you don't conform?  I'm guessing not.  But how about implicit threats?  What if you've heard (or read in the news) stories about people who don't conform—in ways moderately but not hugely more extreme than you—having their careers ruined?  In any situation that you could call "peer pressure", I imagine there's always at least the possibility of some level of social exclusion.

The defining questions for that aspect would appear to be "Do yo... (read more)

2Gunnar_Zarncke
We call it  "peer pressure" when it is constraining the individual (or at least some of them) without providing perceived mutual value. It is the same mechanism that leads to people collaborating for the common good. The interesting question is which forces or which environments lead to a negative sum game.

The thing that comes to mind, when I think of "formidable master of rationality", is a highly experienced engineer trying to debug problems, especially high-urgency problems that the normal customer support teams haven't been able to handle.  You have a fresh phenomenon, which the creators of the existing product apparently didn't anticipate (or if they did, they didn't think it worth adding functionality to handle it), which casts doubt on existing diagnostic systems.  You have priors on which tools are likely to still work, priors on which unde... (read more)

One argument I've encountered is that sentient creatures are precisely those creatures that we can form cooperative agreements with.  (Counter-argument: one might think that e.g. the relationship with a pet is also a cooperative one [perhaps more obviously if you train them to do something important, and you feed them], while also thinking that pets aren't sentient.)

Another is that some people's approach to the Prisoner's Dilemma is to decide "Anyone who's sufficiently similar to me can be expected to make the same choice as me, and it's best for all of us if we cooperate, so I'll cooperate when encountering them"; and some of them may figure that sentience alone is sufficient similarity.

3Sinclair Chen
we completely dominate dogs. society treat them well because enough humans love dogs. I do think that cooperation between people is the origin of religion, and its moral rulesets which create tiny little societies that can hunt stags. 
2Nathan Helm-Burger
We need better, more specific terms to break up the horrible mishmash of correlated-but-not-truly-identical ideas bound up in words like consciousness and sentient. At the very least, let's distinguish sentient vs sapient. All animals all sentient, only smart ones are sapient (maybe only humans depending on how strict your definition). https://english.stackexchange.com/questions/594810/is-there-a-word-meaning-both-sentient-and-sapient Some other terms needing disambiguation... Current LLMs are very knowledgeable, somewhat but not very intelligent, somewhat creative, and lack coherence. They have some self-awareness but it seems to lack some aspects that animals have around "feeling self state", but some researchers are working on adding these aspects to experimental architectures. What a mess our words based on observing ourselves make of trying to divide reality at the joints when we try to analyze non-human entities like animals and AI!

So, the arithmetic and geometric mean agree when the inputs are equal, and, the more unequal they are, the lower the geometric mean is.

I note that the subtests have ceilings, which puts a limit on how much any one can skew the result.  Like, if you have 10 subtests, and the max score is something like 150, then presumably each test has a max score of 15 points.  If we imagine someone gets five 7s and five 13s (a moderately unbalanced set of abilities), then the geometric mean is 9.54, while the arithmetic mean is 10.  So, even if someone wer... (read more)

2tailcalled
If you consider the "true ability" to be the exponential of the subtest scores, then the extent to which the problem I mention applies depends on the base of the exponential. In the limiting case where the base goes to infinity, only the highest ability matter, whereas in the limiting case where the base goes to 1, you end up with something basically linear. As for whether it's a crux, approximately nobody has thought about this deeply enough that they would recognize it, but I think it's pretty foundational for a lot of disagreements about IQ.

1. IQ scores do not measure even close to all cognitive abilities and realistically could never do that.

Well, the original statement was "sums together cognitive abilities" and didn't use the word "all", and I, at least, saw no reason to assume it.  If you're going to say something along the lines of "Well, I've tried to have reasonable discussions with these people, but they have these insane views", that seems like a good time to be careful about how you represent those views.

2. Many of the abilities that IQ scores weight highly are practically unim

... (read more)
7Viliam
The correlations are the important part. A popular competitor to IQ is the theory of multiple intelligences. It sounds very nice and plausible, the only problem is that the actual data do not support the theory. When you measure them, most of the intelligences correlate strongly with each other, and the ones that correlate less are the ones that stretch the meaning of "intelligence" a bit too far (things like "dancing intelligence"). Another problem is that no one agrees on the standard list of those multiple intelligences (different lists of various lengths have been proposed), because all those lists are a result of armchair reasoning. The proper way to do that would be to collect lots of data first, and then do factor analysis and see what you get as a result. But when you actually do that, what you get is... IQ.
7Steven Byrnes
(Fun tangent, not directly addressing this argument thread.) There’s a trio of great posts from 2015 by @JonahS : The Truth About Mathematical Ability ; Innate Mathematical Ability ; Is Scott Alexander bad at math? which (among other things) argues that you can be “good at math” along the dimension(s) of noticing patterns very quickly, AND/OR you can be “good at math” along the dimension(s) of an “aesthetic” sense for concepts being right and sensible. (My summary, not his.) The “aesthetics” is sorta a loss function that provides a guidestar for developing good deep novel understanding—but that process may take a very long time. He offers Scott Alexander, and himself, and Alexander Grothendieck as examples of people with lopsided profiles—stronger on “aesthetics” than they are on “fast pattern-recognition”. I found it a thought-provoking hypothesis. I wish JonahS had written more.
2tailcalled
The analogy that I'm objecting to is, if you looked at e.g. the total for a ledger or a budget, it is an index that sums together expenses in a much more straightforward way. For instance if there is a large expense, the total is large. Meanwhile, IQ scores are more like the geometric mean of the entries on such an entity. The geometric mean tells you whether the individual items tend to be large or small, which gives you broad-hitting information that distinguishes e.g. people who live in high-income countries from people who live in low-income countries, or large organizations from individual people; but it won't inform you if someone got hit by a giant medical bill or if they managed to hack themselves to an ultra-cheap living space. These pretty much necessarily have to be low-rank mediators (like in the g model) rather than diverse aggregates (like in the sum model). (Well, a complication in this analogy is that a ledger can vary not just in the magnitude of the transfers but also qualitatively in the kinds of transfers that are made, whereas IQ tests fix the variables, making it more analogous to a standardized budget form (e.g. for tax or loan purposes) broken down by stuff like "living space rent", "food", "healthcare", etc..)

thinks of IQ as an index that sums together cognitive abilities

Is this part not technically true?  IQ tests tend to have a bunch of subtests intended to measure different cognitive abilities, and you add up—or average, which is adding up and dividing by a constant—your scores on each subtest.  For example (bold added):

The current version of the test, the WAIS-IV, which was released in 2008, is composed of 10 core subtests and five supplemental subtests, with the 10 core subtests yielding scaled scores that sum to derive the Full Scale IQ.

4tailcalled
That's part of the problem, often the bad middle ground looks superficially plausible, so it's very sticky and hard to get rid of, because it's not exactly that people get told the wrong things but rather that they spontaneously develop the wrong ideas. The three basic issues with this viewpoint are: 1. IQ test batteries do not measure even close to all cognitive abilities and realistically could never do that. 2. Many of the abilities that IQ scores weight highly are practically unimportant. 3. Differential-psychology tests are in practice more like log scales than like linear scales, so "sums" are more like products than like actual suns; even if you are absurdly good at one thing, you're going to have a hard time competing with someone in IQ if they are moderately better at many things.
localdeity5314

Interesting.  The natural approach is to imagine that you just have a 3-sided die with 2, 4, 6 on the sides, and if you do that, then I compute A = 12 and B = 6[1].  But, as the top Reddit comment's edit points out, the difference between that problem and the one you posed is that your version heavily weights the probability towards short sequences—that weighting being 1/2^n for a sequence of length n.  (Note that the numbers I got, A=12 and B=6, are so much higher than the A≈2.7 and B=3 you get.)  It's an interesting selection effect.

T... (read more)

2Anders Lindström
  Yes! This kind of kills the "paradox". Its approaching an apples and oranges comparison. Surviving sequences with n=100 rolls (for illustrative purposes) [6, 6] [6, 6] [2, 6, 6] [6, 6] [2, 6, 6] [6, 6] Estimate for A: 2.333 [6, 6] [4, 4, 6, 2, 2, 6] [6, 6] [6, 2, 4, 4, 6] [6, 4, 6] [4, 4, 6, 4, 6] [6, 6] [6, 6] Estimate for B: 3.375 if you rephrase A: The probability that you roll a fair die until you roll two 6s in a row, given that all rolls were even. B: The probability that you roll a fair die until you roll two non-consecutive 6s (not necessarily in a row), given that all rolls were even. This changes the code to: A_estimate = num_sequences_without_odds/n B_estimate = num_sequences_without_odds/n And the result (n=100000)  Estimate for A: 0.045 Estimate for B: 0.062 I guess this is what most people where thinking when reading the problem, i.e., its a bigger chance of getting two non consecutive 6s. But by the wording (see above) of the "paradox" it gives more rolls on average for the surviving sequences, but you on the other hand have more surviving sequences hence higher probability.

I can also come up with a story where obviously it's cheaper and more effective to disable all of the nuclear weapons than it is to take over the world, so why would the AI do the second thing?

Erm... For preventing nuclear war on the scale of decades... I don't know what you have in mind for how it would disable all the nukes, but a one-off breaking of all the firing mechanisms isn't going to work.  They could just repair/replace that once they discovered the problem.  You could imagine some more drastic thing like blowing up the conventional exp... (read more)

3Tahp
I don't think you're being creative enough about solving the problem cheaply, but I also don't think this particular detail is relevant to my main point. Now you've made me think more about the problem, here's me making a few more steps toward trying to resolve my confusion: The idea with instrumental convergence is that smart things with goals predictably go hard with things like gathering resources and increasing odds of survival before the goal is complete which are relevant to any goal. As a directionally-correct example for why this could be lethal, humans are smart enough to do gain-of-function research on viruses and design algorithms that predict protein folding. I see no reason to think something smarter could not (with some in-lab experimentation) design a virus that kills all humans simultaneously at a predetermined time, and if you can do that without affecting any of your other goals more than you think humans might interfere your goals, then sure, you kill all the humans because it's easy and you might as well. You can imagine somehow making an AI that cares about humans enough not to straight up kill all of them, but if humans are a survival threat, we should expect it to find some other creative way to contain us, and this is not a design constraint you should feel good about. In particular, if you are an algorithm which is willing to kill all humans, it is likely that humans do not want you to run, and so letting humans live is bad for your own survival if you somehow get made before the humans notice you are willing to kill them all. This is not a good sign for humans' odds of being able to get more than one try to get AI right if most things are concerned with their own survival, even if that concern is only implicit in having any goal whatsoever. Importantly, none of this requires humans to make a coding error. It only requires a thing with goals and intelligence, and the only apparent way to get around it is to have the smart thing implicitly

why most perfect algorithms that recreate a strawberry on the molecular level destroy the planet as well.

Phrased like this, the answer that comes to mind is "Well, this requires at least a few decades' worth of advances in materials science and nanotechnology and such, plus a lot of expensive equipment that doesn't exist today, and e.g. if you want this to happen with high probability, you need to be sure that civilization isn't wrecked by nuclear war or other threats in upcoming decades, so if you come up with a way of taking over the world that has higher certainty than leaving humanity to its own devices, then that becomes the best plan."  Classic instrumental convergence, in other words.

1Tahp
Oops, I meant cellular, and not molecular. I'm going to edit that. I can come up with a story in which AI takes over the world. I can also come up with a story where obviously it's cheaper and more effective to disable all of the nuclear weapons than it is to take over the world, so why would the AI do the second thing? I see a path where instrumental convergence leads anything going hard enough to want to put all of the atoms on the most predictable path it can dictate. I think the thing that I don't get is what principle it is that makes anything useful go that hard. Something like (for example, I haven't actually thought this through) "it is hard to create something with enough agency/creativity to design and implement experiments toward a purpose without also having it notice and try to fix things in the world which are suboptimal to the purpose."

The political version of the question isn't functionally the same as the skin cream version, because the former isn't a randomized intervention—cities that decided to add gun control laws seem likely to have other crime-related events and law changes at the same time, which could produce a spurious result in either direction.  So it's quite reasonable to say "My opinion is determined by my priors and the evidence didn't appreciably affect my position."

localdeity3110

90% awful idea: "Genetic diversity" in computer programs for resistance to large-scale cyberattacks.

The problem: Once someone has figured out the right security hole in Tesla's software (and, say, broken into a server used to deliver software updates), they can use this to install their malicious code into all 5 million Teslas in the field (or maybe just one model, so perhaps 1 million cars), and probably make them all crash simultaneously and cause a catastrophe.

The solution: There will probably come a point where we can go through the codebase and pick r... (read more)

4davekasten
yup, as @sanxiyn says, this already exists.  Their example is, AIUI, a high-end research one; an actually-on-your-laptop-right-now, but admittedly more narrow example is address space layout randomization.   
sanxiyn3512

This is a good idea and it already works, it is just that AI is wholly unnecessary. Have a look at 2018 post Protecting Applications with Automated Software Diversity.

To me, that will lead to an environment where people think that they are engaging with criticism without having to really engage with the criticism that actually matters. 

This is a possible outcome, especially if the above tactic were the only tactic to be employed.  That tactic helps reduce ignorance of the "other side" on the issues that get the steelmanning discussion, and hopefully also pushes away low-curiosity tribalistic partisans while retaining members who value deepening understanding and intellectual integrity.  There are lots of ... (read more)

4ChristianKl
If I say that other psychiatrists at the conference are engaging in an ethical lapse when they charge late fees to poor people then I'm engaging in an uncomfortable interpersonal conflict. It's about personal incentives that actually matter a lot to the day-to-day practice of psychiatry.  While the psychiatrists are certainly aware of them charging poor people, they are likely thinking about it normally as business as usual instead of considering it as an ethical issue.  If we take Scott's example of psychiatrists talking about racism being a problem in psychiatry I don't think the problem is that that racism is unimportant. The problem is rather that you can get points by virtue signaling talking about the problem and find common ground around the virtue signaling if you are willing to burn a few scapegoats while talking about the issues of charging poor people late fees is divisive.  Washington DC is one of the most liberal places in the US with people who are good at virtue signaling and pretending they care about "solving systematic racism" yet, they passed a bill to require college degrees for childcare services. If you apply the textbook definition of systematic racism, requiring college degrees for childcare services is about creating a system that prevents poor Black people to look after children.  Systematic racism that prevents poor Black people from offering childcare services is bad but the people in Washington DC are good at rationalising. The whole discourse about racism is of a nature where people score their points by virtue signaling about how they care about fighting racism. They practice steelmanning racism all the time and steelmanning the concept of systematic racism and yet they pass systematic racist laws because they don't like poor Black people looking after their children.  If you tell White people in Washington DC who are already steelmanning systematic racism to the best of their ability that they should steelman it more because they

I want to register high appreciation of Elizabeth for her efforts and intentions described here. <3

The remainder of this post is speculations about solutions.  "If one were to try to fix the problem", or perhaps "If one were to try to preempt this problem in a fresh community".  I'm agnostic about whether one should try.

Notes on the general problem:

  • I suspect lots of our kind of people are not enthusiastic about kicking people out.  I think several people have commented, on some cases of seriously bad actors, that it took way too long to a
... (read more)
9ChristianKl
It's worth noting that Jacy was sort-of kicked out (see https://nonprofitchroniclesdotcom.wordpress.com/2019/04/02/the-peculiar-metoo-story-of-animal-activist-jacy-reese/ )
3ChristianKl
To me, that will lead to an environment where people think that they are engaging with criticism without having to really engage with the criticism that actually matters.  From Scott's Criticism Of Criticism Of Criticism: If you frame the criticism as having to be about the mission of psychiatry, it's easy for people to see "Is it ethical to charge poor patients three-digit fees for no-shows?" as off-topic.  In an organization like GiveWell people who criticize GiveWell's mission in such a way, are unlikely to talk about the ways, in which GiveWell favors raising more donations over being more truthseeking, that Ben Hoffman described. 

Issues in transcript labeling (I'm curious how much of it was done by machine):

  • After 00:07:55, a line is unattributed to either speaker; looks like it should be Timothy.
  • 00:09:43 is attributed to Timothy but I think must be Elizabeth.
  • Then the next line is unattributed (should be Timothy).
  • After 00:14:00, unattributed (should be Timothy).
  • After 00:23:38, unattributed (should be Timothy)
  • After 00:32:34, unattributed (probably Elizabeth)
2Elizabeth
Sorry, I missed this too. The first-pass transcript was indeed done by AI. I went over it probably dozens of times, but I guess not enough.
1Timothy Telleen-Lawton
[I’m coming back to the comments in this post now and feeling grateful for all the engagement with our podcast. Apologies for being very low engagement on LW for the last decade.] I just (re-?)noticed that I never addressed your curiosity about machine editing. Yes, we made heavy use of an automatic transcript/audio editing tool. We can get you the name of it if desired, though I recommend reaching out to me offline if I don’t reply here sufficiently quickly to any follow-up questions you or anyone might have. Your curiosity helps me realize that I think we should consider flagging all use of automatic tools even if they don’t make use of AI @Elizabeth (I’m not even sure if this tool does or doesn’t 😬). One of Elizabeth and my ongoing curiosities is what the best format is for optimizing clarity/trnasparency, completeness of context, and ease of engagement (among other things). My current best guess is that we will consider publishing at least two versions of each podcast, at least once we have sufficient editorial capacity: 1. “Complete” unedited audio with automatically generated transcript—ideally with some cursory review for accuracy including (in my favorite world that would require more talent than I currently have) a community-transcript-editting process so that people like you could make these changes (and add more footnotes) directly as on Wikipedia. (“Complete” is in quotes ‘only’ because it feels impossible to include all context in EVN/my conversations unless we record all of our interactions, which is rough because she’s become one of my top 5 friends these days). 2. Easy edit for fast publishing without all the tricky bits that feel like they need more context to publish in a non confusing way. 3. Pithy version to optimize productive engagement (without sacrificing integrity, obvi). Ideally makes heavy use of footnotes and links that link everywhere including to the relevant portion of the full transcript. If we had this I imagine it should b
1Timothy Telleen-Lawton
Awesome, thank you! I'm not sure if we're going to correct this; it's a pain in the butt to fix, especially in the YouTube version, and Elizabeth (who has been doing all the editing herself) is sick right now.

Grammatically, the most obvious interpretation is a universal quantification (i.e. "All men are taller than all women"), which I think is a major reason why such statements so often lead to objections of "But here's an exception!"  Maybe you can tell the audience that they should figure out when to mentally insert "... on average" or "tend to be".  Though there are also circumstances where one might validly believe that the speaker really means all.  I think it's best to put such qualified language into your statements from the start.

2Benaya Koren
Here I mostly agree Here I don't, for the same reason that I don't ask about "water in the refrigerator outside eggplant cells". Because pragmatics are for better or worse part of the language.

Are you not familiar with the term "vacuously true"?  I find this very surprising.  People who study math tend to make jokes with it.

The idea is that, if we were to render a statement like "Colorless green ideas sleep furiously" into formal logic, we'd probably take it to mean the universal statement "For all X such that X is a colorless green idea, X sleeps furiously".  A universal statement is logically equivalent to "There don't exist any counterexamples", i.e. "There does not exist X such that X is a colorless green idea and X does not s... (read more)

2Jim Pivarski
This sounds like it's using Russell's theory of descriptions, in that you're replacing "Colorless green ideas do Y" with "For all X such that X is a colorless green idea, X does Y." Not everyone agrees that this is a correct interpretation, in part because it seems that statements like "Dragons are attacking Paris" should be false. I think it would be reasonable to say that "colorless green ideas" is not just a set of objects in which there are no existing members, but meaningless (for two reasons: "colorless" and "green" conflict, and ideas can't be colored, anyway). I think that was Chomsky's intention—not to write a false sentence, but a meaningless one.
2cubefox
I don't think so. "Smoking causes cancer" doesn't express a universal (or existential) quantification either. Or "Canadians are polite", "Men are taller than women" etc.

to the point where you can't really eliminate the context-dependence and vagueness via taboo (because the new words you use will still be somewhat context-dependent and vague)

You don't need to "eliminate" the vagueness, just reduce it enough that it isn't affecting any important decisions.  (And context-dependence isn't necessarily a problem if you establish the context with your interlocutor.)  I think this is generally achievable, and have cited the Eggplant essay on this.  And if it is generally achievable, then:

Richard is arguing against

... (read more)

Richard is arguing against foundational pictures which assume these problems away, and in favor of foundational pictures which recognize them.

I think you should handle the problems separately.  In which case, when reasoning about truth, you should indeed assume away communication difficulties.  If our communication technology was so bad that 30% of our words got dropped from every message, the solution would not be to change our concept of meanings; the solution would be to get better at error correction, ideally at a lower level, but if necessar

... (read more)

Presumably anything which is above 50% eggplant is rounded to 100%, and anything below is rounded to 0%.

No, it's more like what you encounter in digital circuitry.  Anything above 90% eggplant is rounded to 100%, anything below 10% eggplant is rounded to 0%, and anything between 10% and 90% is unexpected, out of spec, and triggers a "Wait, what?" and the sort of rethinking I've outlined above, which should dissolve the question of "Is it really eggplant?" in favor of "Is it food my roommate is likely to eat?" or whatever new question my underlying pur... (read more)

2cubefox
Note that in the example we never asked the question "Is it really an eggplant?" in the first place, so this isn't a question for us to dissolve. The question was rather how to update our original belief, or whether to update it at all (leave it unchanged). You are essentially arguing that Bayesian updating only works for beliefs whose vagueness (fuzzy truth value) is >90% or <10%. That Bayesian updating isn't applicable for cases between 90% and 10%. So if we have a case with 80% or 20% vagueness, we can't use the conditionalization rule at all. This "restricted rounding" solution seems reasonable enough to me, but less than satisfying. First, why not place the boundaries differently? Like at 80%/20%? 70%/30%? 95%/5%? Heck, why not 50%/50%? It's not clear where, and based on which principles, to draw the line between using rounding and not using conditionalization. Second, we are arguably throwing information away when we have a case of vagueness between the boundaries and refrain from doing Bayesian updating. There should be an updating solution which works for all degrees of vagueness so long as we can't justify specific rounding boundaries of 50%/50%. This solution assumes we can only use probability estimates when they are a) relevant to practical decisions, and b) that cases between 90% and 10% vagueness are never decision relevant. Even if we assume b) is true, a) poses a significant restriction. It makes Bayesian probability theory a slave of decision theory. Whenever beliefs aren't decision relevant and have a vagueness between the boundaries, we wouldn't be allowed to use any updating. E.g. when we are just passively observing evidence, as it happens in science, without having any instrumental intention with our observations other than updating our beliefs. But arguably it's decision theory that relies on probability theory, not the other way round. E.g. in Savage's or Jeffrey's decision theories, which both use subjective probabilities as input in order

The edges of perhaps most real-world concepts are vague, but there are lots of central cases where the item clearly fits into the concept, on the dimensions that matter.  Probably 99% of the time, when my roommate goes and buys a fruit or vegetable, I am not confounded by it not belonging to a known species, or by it being half rotten or having its insides replaced or being several fruits stitched together.  The eggplant may be unusually large, or wet, or dusty, or bruised, perhaps more than I realized an eggplant could be.  But, for many pu... (read more)

2cubefox
It appears you are appealing to rounding: Most concepts are vague, but we should round the partial containment relation to a binary one. Presumably anything which is above 50% eggplant is rounded to 100%, and anything below is rounded to 0%. And you appear to be saying in 99% of cases, the vagueness isn't close to 50% anyway, but closer to 99% or 1%. That may be the case of eggplants, or many nouns (though not all), but certainly not for many adjectives, like "large" or "wet" or "dusty". (Or "red", "rational", "risky" etc.)

I don't think you mentioned "nootropic drugs" (unless "signaling molecules" is meant to cover that, though it seems more specific).  I don't think there's anything known to give a significant enhancement beyond alertness, but in a list of speculative technologies I think it belongs.

[This comment is no longer endorsed by its author]Reply
5Mateusz Bagiński
mentioned in the FAQ

I would be surprised if grocery stores sold edge cases... But perhaps it was a farmer's market or something, perhaps a seller who often liked to sell weird things, perhaps grew hybridized plants.  I'll take the case where it's a fresh vegetable/fruit/whatever thing that looks kind of eggplant-ish.

Anyway, that would generally be determined by: Why do I care whether he bought an eggplant?  If I just want to make sure he has food, then that thing looks like it counts and that's good enough for me.  If I was going to make a recipe that called fo... (read more)

2cubefox
The problem is that almost all concepts are vague, including "vague" and "exact". And things often fit a concept to, like, 90% or 10%. Instead of being a clear 50% edge case. If none of these cases allows for the application of Bayesian updating, because we "lose interest" in the question of how to update, then conditionalization isn't applicable to the real world.
Load More