All of junk heap homotopy's Comments + Replies

Answer by junk heap homotopy110

Set theory is the prototypical example I usually hear about. From Wikipedia:

Mathematical topics typically emerge and evolve through interactions among many researchers. Set theory, however, was founded by a single paper in 1874 by Georg Cantor: "On a Property of the Collection of All Real Algebraic Numbers".

It'd be cool if a second group also worked towards "rationality skill assessment."

This was my project at last year's Epistea, but I sort of had to pause it to work full-time on my interp upskilling experiment.

I only got as far as implementing ~85% of an app to facilitate this (as described here), but maybe a quick chat about this would still be valuable?

I wouldn’t say that. Signalling the way you seem to have used it implies deception on their part, but each of these instances could just be a skill issue on their end, an inability to construct the right causal graph with sufficient resolution.

For what it’s worth whatever this pattern is pointing at also applies to how wrongly most of us got the AI box problem, i.e., that some humans by default would just let the damn thing out without needing to be persuaded.

2Viliam
How would one even distinguish between those who don't actually care about solving the problem and only want to signal that they care, and those who care but are too stupid to realize that intent is not magic? I believe that both do exist in the real world. I would probably start charitably assuming stupidity, and try to explain. If the explanations keep failing mysteriously, I would gradually update towards not wanting to actually achieve the declared goal.

There seems to two major counter-claims to your project:

  • Feedback loops can't work for nebulous domains, so this whole thing is misguided.
  • Transfer learning is impossible and you can't get better at rationality by grinding LeetCode equivalents.

(There's also the third major counter-claim that this can't work for alignment research, but I assume that's actually irrelevant since your main point seems to be about rationality training.)

My take is that these two claims stem from inappropriately applying an outcome-oriented mindset to a process-oriented proble... (read more)

There's this tension between what I know from the literature (i.e. transfer learning is basically impossible) and my lived experience that I and a handful of the people I know in real life whom I have examined in depth are able to quickly apply e.g. thermodynamics concepts to designing software systems, or how consuming political fiction has increased my capacity to model equilibrium strategies in social situations. Hell, this entire website was built on the back of HPMoR, which is an explicit attempt to teach rationality by reading about it.

The point othe... (read more)

Answer by junk heap homotopy50

The part in That Alien Message and the Beisutsukai shorts that are about independently regenerating known science from scratch. Or zetetic explanations, whichever feels more representative of the idea cluster.

In particular, how does one go about making observations that are useful for building models? How does one select the initial axioms in the first place?

I don't know. It seems to me that we have to make the graph of progress in alignment vs capabilities meet somewhere and part of that would probably involve really thinking about which parts of which bottlenecks are really blockers vs just epiphenomena that tag along but can be optimised away. For instance, in your statement:

If research would be bad for other people to know about, you should mainly just not do it

Then maybe doing research but not having the wrong people know about it is the right intervention, rather than just straight-up not doing it at all?

Yup, that's definitely on the roadmap. Sometimes you need facts to advance a particular skill (e.g., if you're figuring out scaling laws, you have got to know a bunch of important numbers) or even just to use the highly specialised language of your field, and there's no better way to do that than to use SRS.

We're probably going to offer Anki import some time in the future just so we can take advantage of the massive amount of material other people have already made, but also I can't promise one-to-one parity since I'd like to aim more towards gwern's progr... (read more)

Have you scaled up the moderation accordingly? I have noticed fewer comments that are on the level of what gets posted on r/slatestarcodex these days but I'm not sure if it's just a selection effect.

4Raemon
For the past couple weeks we've been checking each new comment and rejecting ones we think fail the criteria in the comment warning. (This ends up being something like 9 comments a day. For comparison, there's like 2-3 comments from newish users each day that actually seem to meet the bar) We'll hopefully have the "rejected" section of the site up soon so that people can evaluate whether they think we're making reasonable calls here. (To be clear, when I say "not reduced the number of posts", I mean "reduced the number of people submitting posts", not "posts that get accepted." We've also been rejecting a few new posts a day)

Congratulations! I don't think I would have navigated the past three years as well as I did without your posts.

Is this particular to LessWrong or is there some browser shenanigans going on in being able to render this?

1Florence Hinder
It's just an embed link that is setup to work with LessWrong and the EA forum. 

Person-affecting view

I want to see if I’m cut out to do nontrivial independent alignment research, and I want to get an answer as quickly as possible. The best way to do that is to waste everyone’s time publicly lay down my maps and hope that someone out there will feel white-hot intransigent rage at someone being wrong on the internet and correct me.

Why alignment research?

That is, why not work on anything else like biorisk? Or some other generic longtermist cause?

The God of Power Laws has decreed that most human endeavors be Pareto-distributed. In a hi... (read more)

2the gears to ascension
I always parsed it as wordcel=insufficient training data to form connected mental representation, shape rotator=sufficient training data to form connected mental representation. I would not call PIBBS an unconnected-words group.

Have you read What Should A Professional Mathematician Know? The relevant bits are in the last two sections.

Anecdotally speaking, being forced to take care of a puppy made me significantly more empathic towards animals in general. Whereas I subconsciously saw pet owners as being 'hijacked' (even taking into account a rather animal-obsessed childhood), it was only after fully bonding with my puppy that I was able to empathise with suffering animals on a gut level (again, I used to be able to, then I was desensitised for years, then today I'm back to full-on animal empathy mode).

All this happened over the course of 4-5 days. It was actually quite scary to see my values change so abruptly, to the extent that visiting r/petloss literally makes me want to vomit due to how heavy it makes my heart.

Now I'm curious. Does studying history make you update in a similar way? I feel that these times are not especially insane compared to the rest of history, though the scale of the problems might be bigger.

Wei Dai102

Now I’m curious. Does studying history make you update in a similar way?

History is not one of my main interests, but I would guess yes, which is why I said "Actually, I probably shouldn’t have been so optimistic even before the recent events..."

I feel that these times are not especially insane compared to the rest of history, though the scale of the problems might be bigger.

Agreed. I think I was under the impression that western civilization managed to fix a lot of the especially bad epistemic pathologies in a somewhat stable way, and was unpleasantly surprised when that turned out not to be the case.

Hi there. I signed up for the event but didn’t receive any URL. Can someone help me out?

What do you think are the odds of a successful vaccine being developed within 3 months? 6 months? A year? Before we achieve herd immunity?

How long do you think will the peak last? How does your position (and best-policy recommendations) change as this variable changes?

Definitely in 5-10 years. Hopefully with accommodations for foreignfolk as well?

Not sure if this is the proper place to say this but your first link is broken.

http://www.yudkowsky.net/virtues/ -> http://www.yudkowsky.net/rational/virtues/