Is there a term for this?

Seems like a subtype of Bulverism; not aware of a more specific term.

I also find it hard to call out this type of behaviur when it happens, even when I can tell exactly what is going on.

Assuming you have a LWer-typical level of atypicality, you could say "I literally do/believe [outlandish but politically-neutral activity/opinion], there's no way closed-mindedness is my problem." (If it were me, I'd use donating to Shrimp Welfare; apparently most people think that's strange, for some reason.)

What should I try to do this year?

abstractapplic

I find myself, for the first time in a while, with enough energy and stability to attempt nontrivial projects outside my dayjob. Regarding the next ~10 months, I’ve narrowed my options to two general approaches; as expected beneficiaries of both, I’d like the LessWrong hivemind’s help choosing between them.

The first option is making more D&D.Sci Scenarios, running them on a more consistent schedule, crossposting them to more platforms, and getting more adventurous about their form and content. The second is creating Epistemic Roguelikes, a new^[1] genre of rationalist videogame about deducing and applying the newly-randomized ruleset each run.

Prima facie, prioritizing D&D.Sci this year (and leaving more speculative aspirations to be done next year... (read 235 more words →)

Replying toshortest goddamn bayes guide ever

abstractapplic5d

shortest goddamn bayes guide ever

Premise: The average rationalist cannot use Bayes theorem. No, I will be stronger and more specific: when I ask a room full of people at a LessWrong meetup whether they can write the formula for Bayes Theorem on a piece of paper, less than half of them can.

Nitpick: I think that second sentence is actually weaker-and-more-specific. I'm very confident I can do a Bayesian update right, I not infrequently use Bayes in my dayjob, I once made a videogame about Bayesian reasoning . . . and if you asked me to write the formula without checking Wikipedia I'd have to spend a minute or two deriving it from first principles. I suspect at least a few people are in a similar boat.

Replying toDeeper Reviews for the top 15 (of the 2024 Review)

abstractapplic23d

Deeper Reviews for the top 15 (of the 2024 Review)

Typo in title, unless I'm misunderstanding something: 15 != 50.

Replying toD&D.Sci April 2021 Evaluation and Ruleset

abstractapplic23d

D&D.Sci April 2021 Evaluation and Ruleset

That's pretty fair and useful criticism. If I was making this again I'd have Water Elementals roll +2d6 or +3d4 instead of +1d12; the uniform distribution is in retrospect suspiciously unnatural.

(I do however plead the mitigating factors that foam swords are cheap and "foam swords plus the stuff that ends up mattering" is one of the 'correct' solutions.)

((Also, I'm happy you played one of my older scenarios, and that you took the time to share your thoughts. Feedback is always greatly appreciated.))

abstractapplic25dQuick Take

I tried ChatGPT(-5.2-Thinking) on the original D&D.Sci challenge (which is tough, but not tricky) and it got almost a perfect answer, one point shy of the optimal.

I also tried ChatGPT on the second D&D.Sci challenge (which is tricky, but not tough), and it completely failed (albeit in a sensible and conservative manner). Repeated prompts of "You're missing something, please continue with the challenge" didn't help.

Replying toThe Ones who Feed their Children

abstractapplic1mo

The Ones who Feed their Children

This was incredibly good, though it seemed borderline incoherent on first reading.

abstractapplic1mo

Why do you think METR hasn't built more tasks then, if it's easy?

I have no idea, I just don't think the "actually making the tasks" part can be the limiting factor.

I take it you have a negative opinion of them?

Yes; I also have a positive opinion of them, and various neutral opinions of them.

(My position could be summed up as "the concept of time horizons was really good & important, and their work is net positive, but it could use much stronger methodological underpinning and is currently being leaned on too heavily by too many people"; I'm given to understand that's also their position on themselves.)

abstractapplic1mo

. . . I realize the start of this post reads like a weird brag but imo it really isn't. "Hey failed-wannabe-gamedev, I need a bunch of puzzles and it's ok if they're not very fun and it's ok if there's no UI and it's actively preferable if they're ridiculously complicated and time-consuming and spreadsheet-requiring and reminiscent-of-someone's-dayjob, we're paying a couple grand apiece" is a pitch I imagine a lot of people would be willing and able to jump at, many much moreso than me.

abstractapplic1mo

Apparently it's hard

No? I contributed a ~20hr task to them and it was pretty easy actually? I've been making benchmark-shaped things on and off for the past five years, for free, as a hobby?

(Most of the effort my end was getting it METR's required format, recruiting & managing my playtester, and contemplating whether I was complicit in intellectual fraud^[1]; if they'd made those things easier or handled them themselves I'd have made more; IIRC the actual "make a ~20hr task" part took me <20hrs.)

--It becomes standard practice for any benchmark-maker to include a human baseline for each task in the benchmark, or at least a statistically significant sample.
--They also include information about

abstractapplic1moReview for 2024 Review

The Mysterious Trump Buyers on Polymarket

This was very important. I don't think the specific example is timeless or general enough to belong in a Best Of 2024 collection, but the fact that prediction markets can behave this way - not just in theory, but in practice, in such a way that it potentially alters the course of history - is a big deal, and worth recording.

(Something the OP doesn't mention is the way this effect recurses. The people who shifted probability on prediction markets thereby shifted probabilities in real life, and thus ended up with more money.)

Multidimensional voting is good. It's nice to be able to distinguish "you're wrong" from "shut up" (or "you're right" from "well said"); and it's nice to be part of a community which makes & encourages that distinction.

•••

What's the meta on communicating "hey random pseudonymous internet acquaintance / stranger, just fyi I've stumbled on something linking you to your irl identity, here's how I did it so you can ensure no-one else makes that connection"? I can never figure out how to not feel/seem/be creepy when doing this, but knowing and not telling seems like a dick move.

I'm fanatically in favor of creating new ways to test (& thereby develop) rationality in general and scientific capabilities in particular. However, any such resources would necessarily be dual-use, providing AI developers evals (or eval paradigms) which could help accelerate AI development along the axes on which it's currently most lacking. This seems like an obviously insane thing to worry about but I can't figure out why; soliciting other opinions.

To improve Rationality, create Situations

abstractapplic

3mo

Introduction

Basis: The rationalist project was built on the idea of overcoming bias, using the last half-century of psych findings to eliminate errors and thereby become stronger.
Problem: The psych literature is wrong (mostly, probably, plausibly; we don't even know).
Problem: Even when it's right, it might not apply to an individual: maybe people in general are overconfident, but that doesn't mean you specifically are on the specific topic in question.
Problem: Some biases are subtly loadbearing, or cancel out other biases.
But: We do keep getting important things right, especially purely-epistemically^[1]. We called crypto, called covid, and called AI. (If I'd made investment decisions over the last decade based solely on What LW Is On About

... (read 807 more words →)

If model predictions are consistently too low/high:

If using gradient descent, might be a bad starting point

To diagnose: Try running it with number of training rounds and/or learning rate set to/near zero, and seeing if it predicts an unsuitable value for everything.

To fix: Set the starting point to the average outcome in the training set.

If using gradient descent, might be numerical instability

To diagnose: Watch how individual and aggregate predictions change from round to round. If they flicker back and forth (with unornamented gradient descent) or swing back and forth like a pendulum (with momentum), it’s instability.

To fix: Lower learning rate; possibly increase number of rounds to compensate.

Might be distribution shift

To diagnose: See if... (read 551 more words →)

You’re probably overestimating how well you understand Dunning-Kruger

abstractapplic

4mo

Content Note: Trolling

I

The popular conception of Dunning-Kruger is something along the lines of “some people are too dumb to know they’re dumb, and end up thinking they’re smarter than smart people”. This version is popularized in endless articles and videos, as well as in graphs like the one below.

Usually I'd credit the creator of this graph but it seems rude to do that when I'm ragging on them

Except that’s wrong.

II

The canonical Dunning-Kruger graph looks like this:

Notice that all the dots are in the right order: being bad at something doesn’t make you think you’re good at it, and at worst damages your ability to notice exactly how incompetent you are. The actual... (read 936 more words →)

232

•••

D&D.Sci: Serial Healers [Evaluation & Ruleset]

abstractapplic

5mo

This is a followup to the D&D.Sci post I made on the 6th; if you haven’t already read it, you should do so now before spoiling yourself.

Here is the web interactive I built to let you evaluate your solution; below is an explanation of the rules used to generate the dataset (my full generation code is available here, in case you’re curious about details I omitted). You’ll probably want to test your answer before reading any further.

Who Dunnit?

In rough order of ascending difficulty:

Nettie Silver

Nettie heals Smokesickness; all Smokesickness healing happens when she’s in the area. (She’s been caught multiple times, but she has friends in high places who scupper all such investigations.)

Zancro

Zancro... (read 980 more words →)

•••

D&D.Sci: Serial Healers

abstractapplic

5mo

This is a D&D.Sci scenario: a puzzle where players are given a dataset to analyze and an objective to pursue using information from that dataset.

Thank you to aphyer for the original concept. (NB: The concept was general enough that aphyer can play this one without an unfair advantage.)

Intended Difficulty: ~2/5, on average

The people of Calderia have a problem: magic-users have been surreptitiously and nonconsensually healing^[1] citizens’ minor illnesses and injuries, without charging market rate. In the name of justice, their leaders have tasked you with hunting down these vile rightdoers, so they can be forcibly compensated for their time and effort.

(You hadn’t dared to hope that the problem they wanted help with was... (read 270 more words →)

•••

There's an upside of conventional education which no-one on any side of any debate ever seems to bring up, but which was a major benefit (possibly the major benefit) of my post-primary studies. Namely: it lets students discover what they have a natural aptitude for (or lack thereof) relative to a representative peer group. The most valuable things I learned in my Engineering courses at university were:

.I'm pretty mediocre at Engineering, especially sub-subjects which aren't strictly Structural and/or Mechanical.

.In particular, I'm significantly worse than the average would-be Engineer at working with electronics, so I should change my plans to specialize in that field.

.Conversely, I'm significantly better than the average would-be Engineer at... (read more)

D&D.Sci: The Choosing Ones [Answerkey and Ruleset]

abstractapplic

9mo

This is a followup to the D&D.Sci post I made nine days ago; if you haven’t already read it, you should do so now before spoiling yourself.

Below is an explanation of the rules used to generate the dataset (my full generation code is available here, in case you’re curious about details I omitted).

Ruleset

Skills and Competence

The success of a hero is decided by six hidden variables: Int, Wis, Cha, Str, Con, and Dex^[1]. These are used to produce Competence via the formula

Competence = 8*Int + 9*Wis + 10*Cha + 10*min(Int,Wis,Cha) + 11*max(Str,Con,Dex) + 11*Str+12*Con + 13*Dex^[2]

Competence, Luck, and Success

The Challenge of saving a world is given by

Challenge = 1d20 + 1d40 + 1d60

... (read 853 more words →)

D&D.Sci: The Choosing Ones

abstractapplic

9mo

This is a D&D.Sci scenario: a puzzle where players are given a dataset to analyze and an objective to pursue using information from that dataset.

Thank you to Juan Vasquez for playtesting.

Intended Difficulty: ~3.5/5

The fairy in your bedroom explains that she is a champion of Fate, tasked with whisking mortals into mysterious realms of wonderment and (mild) peril; there, they forge friendships with mythical creatures, do battle with ancient evils, and return to their mundane lives having gained the confidence that comes with having saved a world^[1]. But there’s an unusually large, important world experiencing an unusually non-mild amount of peril - she’d even go so far as to call it moderate peril!... (read 290 more words →)

Notes on the Long Tasks METR paper, from a HCAST task contributor

abstractapplic

9mo

I contributed one (1) task to HCAST, which was used in METR’s Long Tasks paper. This gave me some thoughts I feel moved to share.

ETA: I've made some substantial changes thanks to responses by the original authors.

Regarding Baselines and Estimates

METR’s tasks have two sources for how long they take humans: most of those used in the paper were Baselined using playtesters under persistent scrutiny, and some were Estimated by METR.

I don’t quite trust the Baselines. Baseliners were allowed/incentivized to drop tasks they weren’t making progress with, and were – mostly, effectively, there’s some nuance here I’m ignoring – cut off at the eight-hour mark; Baseline times were found by averaging time taken... (read 595 more words →)

115

I recently watched (the 1997 movie version of) Twelve Angry Men, and found it fascinating from a Bayesian / confusion-noticing perspective.

My (spoilery) notes (cw death, suspicion, violence etc):

The existence of other knives of the same kind as the murder weapon is almost perfectly useless as evidence. The fact that the knife used was identical to the one the accused owned, and was used to kill so close to when the defendant's knife (supposedly) went missing, is still too much of a coincidence to ignore. The only way it would realistically be a different knife is if someone was actively trying to frame the defendant, and arranged for his knife to be lost

... (read 849 more words →)

Equations Mean Things

abstractapplic

11mo

I asserted that this forum could do with more 101-level and/or mathematical and/or falsifiable posts, and people agreed with me, so here is one. People confident in highschool math mostly won’t get much out of most of this, but students browsing this site between lectures might.

The Sine Rule

Say you have a triangle with side lengths a, b, c and internal angles A, B, C. You know a, know A, know b, and want to know B. You could apply the Sine Rule. Or you could apply common sense: “A triangle has the same area as itself”. ^[1]

The area of a triangle is half the base times the height. If you treat a as... (read 706 more words →)

. . . Is there a way a random punter could kick in, say, $100k towards Elon's bid? Either they end up spending $100k on shares valued at somewhere between $100k and $150k; or, more likely, they make the seizure of OpenAI $100k harder at no cost to themselves.

I once saw an advert claiming that a pregnancy test was “over 99% accurate”. This inspired me to invent an only-slightly-worse pregnancy test, which is over 98% accurate. My invention is a rock with “NOT PREGNANT” scrawled on it: when applied to a randomly selected human being, it is right more than 98% of the time. It is also cheap, non-invasive, endlessly reusable, perfectly consistent, immediately effective and impossible to apply incorrectly; this massive improvement in cost and convenience is obviously worth the ~1% decrease in accuracy.

Some Theses on Motivational and Directional Feedback

abstractapplic

Note: Probably reinventing the wheel here. Heavily skewed to my areas of interest. Your results may vary.

Feedback can be decomposed into motivational ("Keep doing what you're doing!") and directional ("Here's what you should do differently . . .").
1. There's also material support ("I like your work, please accept this job offer / some money / my hand in marriage / etc."), which isn't really a type of feedback; I mention it because it can come packaged with the other two or be mistaken for them. (Consider a famous director whose movie gets a rave review from a popular critic: they might not derive any more encouragement from this than their fans already give

... (read 915 more words →)

Enemy HP: 72/104

Fractionalist cast Reduce-4!

It succeeded!

Enemy HP: 18/26

abstractapplic

D&D.Sci Scenario Index

The Summoned Heroine's Prediction Markets Keep Providing Financial Services To The Demon King!

Three Subtle Examples of Data Leakage

Notes on the Long Tasks METR paper, from a HCAST task contributor

abstractapplic

abstractapplic

What should I try to do this year?

To improve Rationality, create Situations

You’re probably overestimating how well you understand Dunning-Kruger

D&D.Sci: Serial Healers [Evaluation & Ruleset]

D&D.Sci: Serial Healers

D&D.Sci: The Choosing Ones [Answerkey and Ruleset]

D&D.Sci: The Choosing Ones

D&D.Sci

abstractapplic

D&D.Sci Scenario Index

The Summoned Heroine's Prediction Markets Keep Providing Financial Services To The Demon King!

Three Subtle Examples of Data Leakage

Notes on the Long Tasks METR paper, from a HCAST task contributor

abstractapplic

abstractapplic

What should I try to do this year?

To improve Rationality, create Situations

You’re probably overestimating how well you understand Dunning-Kruger

D&D.Sci: Serial Healers [Evaluation & Ruleset]

D&D.Sci: Serial Healers

D&D.Sci: The Choosing Ones [Answerkey and Ruleset]

D&D.Sci: The Choosing Ones

D&D.Sci

Introduction

If model predictions are consistently too low/high:

I

II

Who Dunnit?

Nettie Silver

Zancro

Ruleset

Skills and Competence

Competence, Luck, and Success

Regarding Baselines and Estimates

The Sine Rule