All of MichaelDickens's Comments + Replies

Claude 3.5 Sonnet is a mid-sized model that cost a few $10M's to train

I don't get this, if frontier(ish) models cost $10M–$100M, why is Nvidia's projected revenue more like $1T–$10T? Is the market projecting 100,000x growth in spending on frontier models within the next few years? I would have guessed more like 100x–1000x growth but at least one of my numbers must be wrong. (Or maybe they're all wrong by ~1 OOM?)

This is actually a crazy big effect size? Preventing ~10–50% of a cold for taking a few pills a day seems like a great deal to me.

5Drake Thomas
I agree, zinc lozenges seem like they're probably really worthwhile (even in the milder-benefit worlds)! My less-ecstatic tone is only relative to the promise of older lesswrong posts that suggested it could basically solve all viral respiratory infections, but maybe I should have made the "but actually though, buy some zinc lozenges" takeaway more explicit. 

Don't push the frontier of regulations. Obviously this is basically saying that Anthropic should stop making money and therefore stop existing. The more nuanced version is that for Anthropic to justify its existence, each time it pushes the frontier of capabilities should be earned by substantial progress on the other three points.

I think I have a stronger position on this than you do. I don't think Anthropic should push the frontier of capabilities, even given the tradeoff it faces.

If their argument is "we know arms races are bad, but we have to accele... (read more)

4JustinShovelain
I agree. Anthropic's marginal contribution to safety (compared to what we would have in a world without Anthropic) probably doesn't offset Anthropic's contribution to the AI race. I think there are more worlds where Anthropic is contributing to the race in a negative fashion than there are worlds where Anthropic's marginal safety improvement over OpenAI/DeepMind-ish orgs is critical for securing a good future with AGI (weighing things according to the impact sizes and probabilities).

If lysine is your problem but you don't want to eat beans, you can also buy lysine supplements.

1Gordon Seidoh Worley
If someone has gone so far as to buy supplements, they have already done far more to engineer their nutrition than the vegans who I've known who struggle with nutrition.

I primarily use a weird ergonomic keyboard (the Kinesis Advantage 2) with custom key bindings. But my laptop keyboard has normal key bindings, so my "normal keyboard" muscle memory still works.

On Linux Mint with Cinnamon, you can do this in system settings by going to Keyboard -> Layouts -> Options -> Caps Lock behavior. (You can also put that line in a shell script and set the script to run at startup.)

I use a Kinesis Advantage keyboard with the keys rebound to look like this (apologies for my poor graphic design skills):

https://i.imgur.com/Mv9FI7a.png

  • Caps Lock is rebound to Backspace and Backspace is rebound to Shift.
  • Right Shift is rebound to Ctrl + Alt + Super, which I use as a command prefix for window manager commands.
  • "custom macro" uses the keyboard's built-in macro feature to send a sequence of four keypresses (Alt-G Ctrl-`), which I use as a prefix for some Emacs commands.
  • By default, the keyboard has two backslash (\) keys. I use the OS keybo
... (read more)
  1. There were two different clauses, one about malaria and the other about chickens. "Helping people is really important" clearly applies to the malaria clause, and there's a modified version of the statement ("helping animals is really important") that applies to the chickens clause. I think writing it that way was an acceptable compromise to simplify the language and it's pretty obvious to me what it was supposed to mean.
  2. "We should help more rather than less, with no bounds/limitations" is not a necessary claim. It's only necessary to claim "we should help more rather than less if we are currently helping at an extremely low level".
2Said Achmiz
A strange objection—since if you are correct and this is what was meant, then it strengthens my point. If thinking that helping people is really important AND that we should help more rather than less doesn’t suffice to conclude that we should give to chicken-related charities, then still less does merely one of those two premises suffice. (And “helping animals is really important” is, of course, quite far from an uncontroversial claim.) No, this does not suffice. It would only suffice if giving to chicken-related charities were the first (or close to the first) charity that we’d wish to give to, if we were increasing our helping from an extremely low level to a higher one. Otherwise, if we believe, for instance, that helping a little is better than none, and helping a reasonable and moderate amount is better than helping a little, but helping a very large amount is worse (or even just no better) than the preceding, then this threshold may easily be reached long before we get anywhere near chickens (or any other specific cause). In order to guarantee the “we should give to chicken-related charities” conclusion, the “helping more is better than helping less” principle must be unbounded and unlimited.

MIRI's communications strategy update published in May explained what they were planning on working on. I emailed them a month or so ago and they said they are continuing to work on the things in that blog post. They are the sorts of things that can take longer than a year so I'm not surprised that they haven't released anything substantial in the way of comms this year.

1Roko
This?

That's only true if a single GPU (or small number of GPUs) is sufficient to build a superintelligence, right? I expect it to take many years to go from "it's possible to build superintelligence with a huge multi-billion-dollar project" and "it's possible to build superintelligence on a few consumer GPUs". (Unless of course someone does build a superintelligence which then figures out how to make GPUs many orders of magnitude cheaper, but at that point it's moot.)

2Nathan Helm-Burger
Sadly, no. It doesn't take superintelligence to be deadly. Even current open-weight LLMs, like Llama 3 70B, know quite a lot about genetic engineering. The combination of a clever and malicious human, and an LLM able to offer help and advice is sufficient. Furthermore, there is the consideration of "seed AI" which is competent enough to improve and not plateau. If you have a competent human helping it and getting it unstuck, then the bar is even lower. My prediction is that the bar for "seed AI" is lower than the bar for AGI.

I don't think controlling compute would be qualitatively harder than controlling, say, pseudoephedrine.

(I think it would be harder, but not qualitatively harder—the same sorts of strategies would work.)

2Nathan Helm-Burger
I agree that some amount of control is possible. But if we are in a scenario in the future where the offense-defense balance of bioweapons remains similar to how it is today, then a single dose of pseudoephedrine going unregulated by the government and getting turned into methamphetamine could result in the majority of humanity being wiped out. Pseudoephedrine is regulated, yes, but not so strongly that literally none slips past the enforcement. With the stakes so high, a mostly effective enforcement scheme doesn't cut it.

Also, I don't feel that this article adequately addressed the downside of SA that it accelerates an arms race. SA is only favored when alignment is easy with high probability and you're confident that you will win the arms race, and you're confident that it's better for you to win than for the other guy[1], and you're talking about a specific kind of alignment where an "aligned" AI doesn't necessarily behave ethically, it just does what its creator intends.

[1] How likely is a US-controlled (or, more accurately, Sam Altman/Dario Amodei/Mark Zuckerberg-contr... (read more)

4Sammy Martin
We do discuss this in the article and tried to convey that it is a very significant downside of SA. All 3 plans have enormous downsides though, so a plan posing massive risks is not disqualifying. The key is understanding when these risks might be worth taking given the alternatives. * CD might be too weak if TAI is offense-dominant, regardless of regulations or cooperative partnerships, and result in misuse or misalignment catastrophe * If GM fails it might blow any chance of producing protective TAI and hand over the lead to the most reckless actors. * SA might directly provoke a world war or produce unaligned AGI ahead of schedule. SA is favored when alignment is easy or moderately difficult (e.g. at the level where interpretability probes, scalable oversight etc. help) with high probability, and you expect to win the arms race. But it doesn't require you to be the 'best'. The key isn't whether US control is better than Chinese control, but whether centralized development under any actor is preferable to widespread proliferation of TAI capabilities to potentially malicious actors Regarding whether the US (remember on SA there's assumed to be extensive government oversight) is better than the CCP: I think the answer is yes and I talk a bit more about why here. I don't consider US AI control being better than Chinese AI control to be the most important argument in favor of SA, however. That fact alone doesn't remotely justify SA: you also need easy/moderate alignment and you need good evidence than an arms race is likely unavoidable regardless of what we recommend.

Cooperative Development (CD) is favored when alignment is easy and timelines are longer. [...]

Strategic Advantage (SA) is more favored when alignment is easy but timelines are short (under 5 years)

I somewhat disagree with this. CD is favored when alignment is easy with extremely high probability. A moratorium is better given even a modest probability that alignment is hard, because the downside to misalignment is so much larger than the downside to a moratorium.[1] The same goes for SA—it's only favored when you are extremely confident about alignment +... (read more)

2Sammy Martin
Let me clarify an important point: The strategy preferences outlined in the paper are conditional statements - they describe what strategy is optimal given certainty about timeline and alignment difficulty scenarios. When we account for uncertainty and the asymmetric downside risks - where misalignment could be catastrophic - the calculation changes significantly. However, it's not true that GM's only downside is that it might delay the benefits of TAI. Misalignment (or catastrophic misuse) has a much larger downside than a successful moratorium. That is true, but trying to do a moratorium, losing your lead, and then someone else developing catastrophically misaligned AI when you could have developed a defense against it if you'd adopted CD or SA has just as large a downside. And GM has a lower chance of being adopted than CD or SA, so the downside to pushing for a moratorium is not necessarily lower. Since a half-successful moratorium is the worst of all worlds (assuming that alignment is feasible) because you lose out on your chances of developing defenses against unaligned or misused AGI, it's not always true that the moratorium plan has fewer downsides than the others. However, I agree with your core point - if we were to model this with full probability distributions over timeline and alignment difficulty, GM would likely be favored more heavily than our conditional analysis suggests, especially if we place significant probability on short timelines or hard alignment
4Nathan Helm-Burger
I think moratorium is basically intractable short of a totalitarian world government cracking down on all personal computers. Unless you mean just a moratorium on large training runs, in which case I think it buys a minor delay at best, and comes with counterproductive pressures on researchers to focus heavily on diverse small-scale algorithmic efficiency experiments.

Also, I don't feel that this article adequately addressed the downside of SA that it accelerates an arms race. SA is only favored when alignment is easy with high probability and you're confident that you will win the arms race, and you're confident that it's better for you to win than for the other guy[1], and you're talking about a specific kind of alignment where an "aligned" AI doesn't necessarily behave ethically, it just does what its creator intends.

[1] How likely is a US-controlled (or, more accurately, Sam Altman/Dario Amodei/Mark Zuckerberg-contr... (read more)

  1. I was asking a descriptive question here, not a normative one. Guilt by association, even if weak, is a very commonly used form of argument, and so I would expect it to be in used in this case.

I intended my answer to be descriptive. EAs generally avoid making weak arguments (or at least I like to think we do).

I will attempt to answer a few of these.

  1. Why has EV made many moves in the direction of decentralizing EA, rather than in the direction of centralizing it?

Power within EA is currently highly centralized. It seems very likely that the correct amount of centralization is less than the current amount.

  1. Why, as an organization aiming to ensure the health of a community that is majority male and includes many people of color, does the CEA Community Health team consist of seven white women, no men, and no people of color?

This sounds like a rhetorical qu... (read more)

3Viliam
Working hard together with similarly minded people seems great. Never taking a break, and isolating yourself from the world, is not. People working at startups usually get at least free weekends, and often have a partner at home who is not a member of the startup. If you never take a break, I suspect that you are optimizing for appearing to work hard, rather than for actually being productive.
2Mateusz Bagiński
Not necessarily guilt-by-association, but maybe rather pointing out that the two arguments/conspiracy theories share a similar flawed structure, so if you discredit one, you should discredit the other. Still, I'm also unsure how much structure they share, and even if they did, I don't think this would be discursively effective because I don't think most people care that much about (that kind of) consistency (happy to be updated in the direction of most people caring about it).
2Peter Berggren
Thanks for giving some answers here to these questions; it was really helpful to have them laid out like this. 1. In hindsight, I was probably talking more about moves towards decentralization of leadership, rather than decentralization of funding. I agree that greater decentralization of funding is a good thing, but it seems to me like, within the organizations funded by a given funder, decentralization of leadership is likely useless (if leadership decisions are still being made by informal networks between orgs rather than formal ones), or it may lead to a lack of clarity and direction.  3. I understand the dynamics that may cause the overrepresentation of women. However, that still doesn't completely explain why there is an overrepresentation of white women, even when compared to racial demographics within EA at large. Additionally, this also doesn't explain why the overrepresentation of women here isn't seen as a problem on CEA's part, if even just from an optics perspective. 4. Makes sense, but I'm still concerned that, say, if CEA had an anti-Stalinism team, they'd be reluctant to ever say "Stalinism isn't a problem in EA." 5. Again, this was a question that was badly worded on my end. I was referring more specifically to organizations within AI safety, more than EA at large. I know that AMF, GiveDirectly, The Humane League, etc. fundraise outside EA. 6. I was asking a descriptive question here, not a normative one. Guilt by association, even if weak, is a very commonly used form of argument, and so I would expect it to be in used in this case. 7. That makes sense. That was one of my hypotheses (hence my phrase "at least upon initial examination"), and I guess in hindsight it's probably the best one. 10. Starting an AI capabilities company that does AI safety as a side project generally hasn't gone well, and yet people keep doing it. The fact that something hasn't gone well in the past doesn't seem to me to be a sufficient explanation for why people do

Thank you for writing about your experiences! I really like reading these posts.

How big an issue do you think the time constraints were? For example, how much better a job could you have done if all the recommenders got twice as much time? And what would it take to set things up so the recommenders could have twice as much time?

4Zvi
I think twice as much time actually spent would have improved decisions substantially, but is tough - everyone is very busy these days, so it would require both a longer working window, and also probably higher compensation for recommenders. At minimum, it would allow a lot more investigations especially of non-connected outsider proposals.

Do you think a 3-state dark mode selector is better than a 1-state (where "auto" is the only state)? My website is 1-state, on the assumption that auto will work for almost everyone and it lets me skip the UI clutter of having a lighting toggle that most people won't use.

Also, I don't know if the site has been updated but it looks to me like turntrout.com's two modes aren't dark and light, they're auto and light. When I set Firefox's appearance to dark or auto, turntrout.com's dark mode appears dark, but when I set Firefox to light, turntrout.com appears l... (read more)

2TurnTrout
IIRC my site checks (in descending priority): 1. localStorage to see if they've already told my site a light/dark preference; 2. whether the user's browser indicates a global light/dark preference (this is the "auto"); 3. if there's no preference, the site defaults to light. The idea is "I'll try doing the right thing (auto), and if the user doesn't like it they can change it and I'll listen to that choice." Possibly it will still be counterintuitive to many folks, as Said quoted in a sibling comment.

Do you think a 3-state dark mode selector is better than a 1-state (where “auto” is the only state)? My website is 1-state, on the assumption that auto will work for almost everyone and it lets me skip the UI clutter of having a lighting toggle that most people won’t use.

Gwern discusses this on his “Design Graveyard” page:

Auto-dark mode: a good idea but “readers are why we can’t have nice things”.

OSes/browsers have defined a ‘global dark mode’ toggle the reader can set if they want dark mode everywhere, and this is available to a web page; if you are

... (read more)

OP did the work to collect these emails and put them into a post. When people do work for you, you shouldn't punish them by giving them even more work.

I've only read a little bit of Martin Gardner, but he might be the Matt Levine of recreational math.

Many newspapers have a (well-earned) reputation for not technically lying.

Thank you, this information was useful for a project I'm working on.

I don't think I understand what "learn to be visibly weird" means, and how it differs from not following social conventions because you fail to understand them correctly.

I was recently looking into donating to CLTR and I'm curious why you are excited about it? My sense was that little of its work was directly relevant to x-risk (for example this report on disinformation is essentially useless for preventing x-risk AFAICT), and the relevant work seemed to be not good or possibly counterproductive. For example their report on "a pro-innovation approach to regulating AI" seemed bad to me on two counts:

  1. There is a genuine tradeoff between accelerating AI-driven innovation and decreasing x-risk. So to the extent that this repo
... (read more)
3Zach Stein-Perlman
I don't know. I'm not directly familiar with CLTR's work — my excitement about them is deference-based. (Same for Horizon and TFS, mostly. I inside-view endorse the others I mention.)

My perspective is that I'm much more optimistic about policy than about technical research, and I don't really feel qualified to evaluate policy work, and LTFF makes almost no grants on policy. I looked around and I couldn't find any grantmakers who focus on AI policy. And even if they existed, I don't know that I could trust them (like I don't think Open Phil is trustworthy on AI policy and I kind of buy Habryka's arguments that their policy grants are net negative).

I'm in the process of looking through a bunch of AI policy orgs myself. I don't think I ca... (read more)

if you think the polling error in 2024 remains unpredictable / the underlying distribution is unbiased

Is there a good reason to think that if polls have recently under-reported Republican votes?

6Rafael Harth
I think the burden of proof goes the other way? Like, the default wisdom for polling is that each polling error[1] is another sample from a distribution centered around 0. It's not very surprising that it output a R bias twice in a row (even if we ignore the midterms and assume it was properly twice in a row). It's only two samples! That happens all the time. If you want a positive argument: pollsters will have attempted to correct mistakes, and if they knew that there would be an R/D bias this time, they'd adjust in the opposite way, hence the error must be unpredictable. ---------------------------------------- 1. That is, for a smart polling average; individual polls have predictable bias. ↩︎

I don't know how familiar you are with regular expressions but you could do this with a two-pass regular expression search and replace: (I used Emacs regex format, your preferred editor might use a different format. notably, in Emacs [ is a literal bracket but ( is a literal parenthesis, for some reason)

  1. replace "^(https://.? )([[.?]] )*" with "\1"
  2. replace "[[(.*?)]]" with "\1"

This first deletes any tags that occur right after a hyperlink at the beginning of a line, then removes the brackets from any remaining tags.

RE Shapley values, I was persuaded by this comment that they're less useful than counterfactual value in at least some practical situations.

(2) have "truesight", i. e. a literally superhuman ability to suss out the interlocutor's character

Why do you believe this?

6Thane Ruthenis
See e. g. this and this, and it's of course wholly unsurprising, since it's literally what the base models are trained to do.

If your goal is to influence journalists to write better headlines, then it matters whether the journalist has the ability to take responsibility over headlines.

If your goal is to stop journalists from misrepresenting you, then it doesn't actually matter whether the journalist has the ability to take responsibility, all that matters is whether they do take responsibility.

4ChristianKl
If the journalist accurately represents my position in the text of the article I would already see that as a win in most of the media interviews (I have given a bunch but it was a decade ago).

Often, you write something short that ends up being valuable. That doesn't mean you should despair about your longer and harder work being less valuable. Like if you could spend 40 hours a week writing quick 5-hour posts that are as well-received as the one you wrote, that would be amazing, but I don't think anyone can do that because the circumstances have to line up just right, and you can't count on that happening. So you have to spend most of your time doing harder and predictably-less-impactful work.

(I just left some feedback for the mapping discussion post on the post itself.)

Some feedback:

  • IMO this project was a good use of your time ex ante.[1] Unclear if it will end up being actually useful but I think it's good that you made it.
  • "A new process for mapping discussions" is kind of a boring title and IMO does not accurately reflect the content. It's mapping beliefs more so than discussions. Titles are hard but my first idea for a title would be "I made a website that shows a graph of what public figures believe about SB 1047"
  • I didn't much care about the current content because it's basically saying things I already knew (lik
... (read more)
2Nathan Young
Thanks, appreciated. 

This is a good and important point. I don't have a strong opinion on whether you're right, but one counterpoint: AI companies are already well-incentivized to figure out how to control AI, because (as Wei Dai said) controllable AI is more economically useful. It makes more sense for nonprofits / independent researchers to do work that AI companies wouldn't do otherwise.

If Open Phil is unwilling to fund some/most of the best orgs, that makes earning to give look more compelling.

(There are some other big funders in AI safety like Jaan Tallinn, but I think all of them combined still have <10% as much money as Open Phil.)

I should add that I don't want to dissuade people from criticizing me if I'm wrong. I don't always handle criticism well, but it's worth the cost to have accurate beliefs about important subjects. I knew I was gonna be anxious about this post but I accepted the cost because I thought there was a ~25% chance that it would be valuable to post.

A few people (i.e. habryka or previously Benquo or Jessicata) make it their thing to bring up concerns frequently.

My impression is that those people are paying a social cost for how willing they are to bring up perceived concerns, and I have a lot of respect for them because of that.

2Noosphere89
As someone who has disagreed quite a bit with Habryka in the past, endorsed. They are absolutely trying to solve a frankly pretty difficult problem, where there's a lot of selection for more conflict than is optimal, and also selection for being more paranoid than is optimal, because they have to figure out if a company or person in the AI space is being shady or outright a liar, which unfortunately has a reasonable probability, but there's also a reasonable probability of them being honest but them failing to communicate well. I agree with Raemon that you can't have your conflict theory detectors set to 0 in the AI space.

Thanks for the reply. When I wrote "Many people would have more useful things to say about this than I do", you were one of the people I was thinking of.

AI Impacts wants to think about AI sentience and OP cannot fund orgs that do that kind of work

Related to this, I think GW/OP has always been too unwilling to fund weird causes, but it's generally gotten better over time: originally recommending US charities over global poverty b/c global poverty was too weird, taking years to remove their recommendations for US charities that were ~100x less effective ... (read more)

I've been avoiding LW for the last 3 days because I was anxious that people were gonna be mad at me for this post. I thought there was a pretty good chance I was wrong, and I don't like accusing people/orgs of bad behavior. But I thought I should post it anyway because I believed there was some chance lots of people agreed with me but were too afraid of social repercussions to bring it up (like I almost was).

1MichaelDickens
I should add that I don't want to dissuade people from criticizing me if I'm wrong. I don't always handle criticism well, but it's worth the cost to have accurate beliefs about important subjects. I knew I was gonna be anxious about this post but I accepted the cost because I thought there was a ~25% chance that it would be valuable to post.

What are the norms here? Can I just copy/paste this exact text and put it into a top-level post? I got the sense that a top-level post should be more well thought out than this but I don't actually have anything else useful to say. I would be happy to co-author a post if someone else thinks they can flesh it out.

Edit: Didn't realize you were replying to Habryka, not me. That makes more sense.

I get the sense that we can't trust Open Philanthropy to do a good job on AI safety, and this is a big problem. Many people would have more useful things to say about this than I do, but I still feel that I should say something.

My sense comes from:

  • Open Phil is reluctant to do anything to stop the companies that are doing very bad things to accelerate the likely extinction of humanity, and is reluctant to fund anyone who's trying to do anything about it.
  • People at Open Phil have connections with people at Anthropic, a company that's accelerating AGI and h
... (read more)
2MichaelDickens
I've been avoiding LW for the last 3 days because I was anxious that people were gonna be mad at me for this post. I thought there was a pretty good chance I was wrong, and I don't like accusing people/orgs of bad behavior. But I thought I should post it anyway because I believed there was some chance lots of people agreed with me but were too afraid of social repercussions to bring it up (like I almost was).

"Open Phil higher-ups being friends with Anthropic higher-ups" is an understatement. An Open Philanthropy cofounder (Holden Karnofsky) is married to an Anthropic cofounder (Daniela Amodei). It's a big deal!

And I agree with Bryan Caplan's recent take that friendships are often a bigger conflict of interest than money, so Open Phil higher-ups being friends with Anthropic higher-ups is troubling.

No kidding. From https://www.openphilanthropy.org/grants/openai-general-support/:

OpenAI researchers Dario Amodei and Paul Christiano are both technical advisors to Open Philanthropy and live in the same house as Holden. In addition, Holden is engaged to Dario’s sister Daniela.

Wish OpenPhil and EAs in general were more willing to reflect/talk publicly about their mistake... (read more)

Maybe make a post on the EA forum?

I want to add the gear of "even if it actually turns out that OpenPhil was making the right judgment calls the whole time in hindsight, the fact that it's hard from the outside to know that has some kind of weird Epistemic Murkiness effects that are confusing to navigate, at the very least kinda suck, and maybe are Quite Bad." 

I've been trying to articulate the costs of this sort of thing lately and having trouble putting it into words, and maybe it'll turn out this problem was less of a big deal than it currently feels like to me. But, something like... (read more)

Epistemic status: Speculating about adversarial and somewhat deceptive PR optimization, which is inherently very hard and somewhat paranoia inducing. I am quite confident of the broad trends here, but it's definitely more likely that I am getting things wrong here than in other domains where evidence is more straightforward to interpret, and people are less likely to shape their behavior in ways that includes plausible deniability and defensibility.

I agree with this, but I actually think the issues with Open Phil are substantially broader. As a concrete ex... (read more)

Reply7733

As a frequent oatmeal-eater, I have a few miscellaneous comments:

  • You mentioned adding fruit paste, fruit syrup, and fruit pulp to oatmeal, but I'm surprised you didn't mention what I consider the best option: whole fruit. I usually use blueberries but sometimes I mix it up with blackberries or sliced bananas.
  • I buy one-minute oats. You don't actually need them to cook them for a minute, you can just pour boiling water onto them and they'll soften up by the time they're cool enough to eat.
  • I wouldn't eat oats for the protein, they have more than rice but
... (read more)

Relatedly, I see a lot of people use mediocre AI art when they could just as easily use good stock photos. You can get free, watermarkless stock photos at https://pixabay.com/.

The mnemonic I've heard is "red and yellow, poisonous fellow; red and black, friend of Jack"

I was reading some scientific papers and I encountered what looks like fallacious reasoning but I'm not quite sure what's wrong with it (if anything). It does like this:

Alice formulates hypothesis H and publishes an experiment that moderately supports H (p < 0.05 but > 0.01).

Bob does a similar experiment that contradicts H.

People look at the differences in Alice's and Bob's studies and formulate a new hypothesis H': "H is true under certain conditions (as in Alice's experiment), and false under other conditions (as in Bob's experiment)". They look at... (read more)

7JBlack
Yes, it's definitely fishy. It's using the experimental evidence to privilege H' (a strictly more complex hypothesis than H), and then using the same experimental evidence to support H'. That's double-counting. The more possibly relevant differences between the experiments, the worse this is. There are usually a lot of potentially relevant differences, which causes exponential explosion in the hypothesis space from which H' is privileged. What's worse, Alice's experiment gave only weak evidence for H against some non-H hypotheses. Since you mention p-value, I expect that it's only comparing against one other hypothesis. That would make it weak evidence for H even if p < 0.0001 - but it couldn't even manage that. Are there no other hypotheses of comparable or lesser complexity than H' matching the evidence as well or better? Did those formulating H' even think for five minutes about whether there were or not?
4jbkjr
It sounds to me like a problem of not reasoning according to Occam's razor and "overfitting" a model to the available data. Ceteris paribus, H' isn't more "fishy" than any other hypothesis, but H' is a significantly more complex hypothesis than H or ¬H: instead of asserting H or ¬H, it asserts (A=>H) & (B=>¬H), so it should have been commensurately de-weighted in the prior distribution according to its complexity. The fact that Alice's study supports H and Bob's contradicts it does, in fact, increase the weight given to H' in the posterior relative to its weight in the prior; it's just that H' is prima facie less likely, according to Occam. Given all the evidence, the ratio of likelihoods P(H'|E)/P(H|E)=P(E|H')P(H')/(P(E|H)P(H)). We know P(E|H') > P(E|H) (and P(E|H') > P(E|¬H)), since the results of Alice's and Bob's studies together are more likely given H', but P(H') < P(H) (and P(H') < P(¬H)) according to the complexity prior. Whether H' is more likely than H (or ¬H, respectively) is ultimately up to whether P(E|H')/P(E|H) (or P(E|H')/P(E|¬H)) is larger or smaller than P(H')/P(H) (or P(H')/P(¬H)). I think it ends up feeling fishy because the people formulating H' just used more features (the circumstances of the experiments) in a more complex model to account for the as-of-yet observed data after having observed said data, so it ends up seeming like in selecting H' as a hypothesis, they're according it more weight than it deserves according to the complexity prior.

Suppose an ideology says you're not allowed to question idea X.

I think there are two different kinds of "not questioning": there's unquestioningly accepting an idea as true, and there's refusing to question and remaining agnostic. The latter position is reasonable in the sense that if you refuse to investigate an issue, you shouldn't have any strong beliefs about it. And I think the load-bearingness is only a major issue if you refuse to question X while also accepting that X is true.

There's an argument for cooperating with any agent in a class of quasi-rational actors, although I don't know how exactly to define that class. Basically, if you predict that the other agent will reason in the same way as you, then you should cooperate.

(This reminds me of Kant's argument for the basis of morality—all rational beings should reason identically, so the true morality must be something that all rational beings can arrive at independently. I don't think his argument quite works, but I believe there's a similar argument for cooperating on the prisoner's dilemma that does work.)

If I want to write to my representative to oppose this amendment, who do I write to? As I understand, the bill passed the Senate but must still pass Assembly. Is the Senate responsible for re-approving amendments, or does that happen in Assembly?

Also, should I write to a representative who's most likely to be on the fence, or am I only allowed to write to the representative of my district?

2Linch
You are definitely allowed to write to anyone! Free speech! In theory your rep should be more responsive to their own districts however. 

5 minute super intense cardio, as a replacement for long, low intensity cardio. It is easier to motivate oneself to do 5 minutes of Your-Heart-Might-Explode cardio than two hours of jogging or something. In fact it takes very little motivation, if you trick yourself into doing it right after waking up, when your brain is on autopilot anyway, and unable to resist routine.

Interesting, I had the complete opposite experience. I previously had the idea that exercise should be short and really hard, and I couldn't stick with it. Then I learned that it's bette... (read more)

What's the deal with mold? Is it ok to eat moldy food if you cut off the moldy bit?

I read some articles that quoted mold researchers who said things like (paraphrasing) "if one of your strawberries gets mold on it, you have to throw away all your strawberries because they might be contaminated."

I don't get the logic of that. If you leave fruit out for long enough, it almost always starts growing visible mold. So any fruit at any given time is pretty likely to already have mold on it, even if it's not visible yet. So by that logic, you should never eat frui... (read more)

2Morpheus
Heuristics I heard: cutting away moldy bits is ok for solid food (like cheese, carrot). Don't eat moldy bread, because of mycotoxins (googeling this I don't know why people mention bread in particular here). Gpt-4 gave me the same heuristics.
1cubefox
Low confidence: Given that our ancestors had to deal with mold for millions of years, I would expect that animals are quite well adapted to its toxicity. This is different from (evolutionary speaking) new potentially toxic substances, like e.g. transfats or microplastics.

I don't understand how not citing a source is considered acceptable practice. It seems antithetical to standard journalistic ethics.

3Viliam
citing is good for journalistic ethics, but linking is bad for search engine optimization -- at least this is what many websites seem to believe. the idea is that a link to an external source provides PageRank to that source that you could have provided to a different page on your website instead. if anyone in the future tries to find X, as a journalist, you want them to find your article about X, not X itself. journalism is a profit-making business, not charity.
3Garrett Baker
Is it? That’s definitely what my English teacher wanted me to believe, but since every newspaper does it, all the time (except when someone Tweets something) I don’t see how it could be against journalistic ethics. Indeed, I think there’s a strong undercurrent in most mainstream newspapers that “the people” are not smart enough to evaluate primary sources directly, and need journalists & communicators to ensure they arrive at the correct conclusions.

we have found Mr Altman highly forthcoming

He was caught lying about the non-disparagement agreements, but I guess lying to the public is fine as long as you don't lie to the board?

Taylor's and Summers' comments here are pretty disappointing—it seems that they have no issue with, and maybe even endorse, Sam's now-publicly-verified bad behavior.

we have found Mr Altman highly forthcoming

That's exactly the line that made my heart sink.

I find it a weird thing to choose to say/emphasize.

The issue under discussion isn't whether Altman hid things from the new board; it's whether he hid things to the old board a long while ago.

Of course he's going to seem forthcoming towards the new board at first. So, the new board having the impression that he was forthcoming towards them? This isn't information that helps us much in assessing whether to side with Altman vs the old board. That makes me think: why repo... (read more)

Load More