All of algon33's Comments + Replies

I am kind of suprised you didn't reference causal inference here to just gesture at the task in which we "figure out which variables are directly relevant - i.e. which variables mediate the influence of everything else". Are you pointing to a different sort of idea/do you not feel causal inference is adequate for describing this task?

Also, scenario 1 and 2 seem fairly close to the "linear" and "non-linear" models of innovation Jason Crawford described in his talk "The Non-Linear Model of Innovation." To be honest, I prefered his description of the models. ... (read more)

Hymn to breaking strain or Hope Eyrie.

Here's a visualisation that goes along with Euclid's elements

This was one of many from an article on "The Empirical MetaMathematics of Euclid and Beyond". It is a long essay on the overarching structure of Euclid's elements and verifies some claims made about Euclid's Elements e.g. the proofs were ordered in nearly the most parsimon... (read more)

Definetely not a subject, but I'd say that the visualisation of Wolfram's theory of everything is excellent. Of course there are problems with his theory of everything, like the fact that he hasn't actually proved his claims that it generates GR field equations or replicates QM. Or shown that his theory evades the critical objection Scott Aaronson raised. but as a visualisation:

1. It is aesthetically pleasing
2. Compactly contains the basic ideas of his T.o.E. 
3. Ties the basic concepts together to see how they could generate a theory of physics

So I'd still rec... (read more)

I am glad you put the quotation marks around "morality as taxes" since what my mind jumped to upon verbalising the title was what you described in the last part of your post: something you'd be glad to evade where possible. In retrospect, its clear that the quotation marks were meant to point to another approach and not the one your thought experiment is meant to represent. Still, I think "Wholehearted choices vs morality as taxes" would be a little clearer as a title. 

My personal reasons:

1. I assumed the question was about the first few decades after "first contact".
2. A large chunk of my probability mass is on first contact being unintentional, and something neither side can do much about. Or perhaps one "side" is unaware of it. Like if we receive some message directed to no one in particular, or recording the remnants of some extreme cosmic event that seems mighty unatural. 
3. It feels like we're near certain to have created an AGI by then. I am unsure enough about the long term time scales of AGI improvement, and their l

You should alter questions to make it clear "we" is meant to be humans or whatever we makes that succeeds us.

Also, perhaps a queston on whether "first contact" will be us detecting them without their being aware of it.

Minor quibble which I hope isn't breaking a norm: BetFair did seem to pay out last week, or at least some of the bets on who would win the presidency were settled on 07/11/20. 

Do you expect we'll be n the midst of a third wave before the vaccine begins to be doled out? Or just beginning to enter one? 

Thanks for the post.

The post about Sweden's unusual situation you linked to has updated. The author claims that the reduced death rate is mostly due to younger people getting near all of the covid cases, which is supported by recent data (the figure shows total number of changes between July and 03 Nov). Why that is the case is another issue. 

Edit: As always, thanks for the post. 

What about Thorium? A back of the envelope calculation suggests thorium reactors could supply us with energy 100-500 years. I got this from a few sources. First used the figure of the 170 GW days produced per metric tonne of fuel (Fort St Vrain HTR) and the availability of fuel (500-2500 ktonnes according to Wikipedia) to estimate 10-50 years out of Thorium reactors if we keep using 15TW of energy. And that's not even accounting for breeding reactors, which can produce their own fuel. So if we do go with the theoretical maximum, then we should multiply thi... (read more)

Thanks for the reply. Feelings of helplessness sounds about right, and I think you may be right about giving your self the feeling that you are being supported. Only, people with severe chronic pain often suffer from anxiety and depression as well. It seems like it would be a hard battle getting their brains to recognise those aforementioned feelings. 

How does this apply for physically painful trauma? I understand that the broader process should work, but I'm curious if you could guess what frame would be the most helpful for such trauma.

Somewhat urgent: can anyone recommend a good therapist or psychiatrist for anxiety/depression in the UK? Virtual sessions are probably required. Private is fine. Also, they shouldn't be someone biased towards rationalist types. The person I'm thinking of has nearly no knowledge of these ideas.

Other recommendations that seem relevant welcome.

I still disagree. You can use Fermat's last theorem rigorously without understanding why it works. Same for the four colour theorem. And which mathematics understand why we can classify finite simple groups the way we do? I'd bet fewer than a percent do. Little wonder, if the proof's 3 volumes long! My point is that there are many theorems a mathematician will use without rigorously knowing why it works. Oh sure, you can tell them a rough story outlining the ideas. But could the prove it themselves? Probably not, without a deep understanding of the area. Y... (read more)

Note that I didn't say it's not an aesthetic preference. I just don't think likely to be false --> ugly, though I agree learnings its likely to be false-->uglier than before.

No, to understand why the transfer principle works requires a fair amount of knowledge of mathematical logic. It doesn't follow that you can't perform rigorous proofs once you've accepted it. Or am I missing something here?

Because of the shift in culture in mathematics, wherein the old proofs were considered unrigorous. Analysis ala Weirstrauss put the old statements on firmer footing, everyone migrated there, and infinitesimals were left to langiush until a transfer principle  was proven to give them a rigorous founding. But by that time, standard analysis had born such great fruits that it was deeply intertwined with modern mathematics. And of course, there's been a trend against the infinitary and against the incomputable in the past century.

So there's both instituti... (read more)

Stuart, by "$P_{rt}$$\left(R|D_{1;j}\right)$ is complex" are you referring to their using $R=R(.,E[{\Theta }_{\ast }^R|D_{1;j}\left]\right)$ as the estimated reward function?

Also, what did you think of their arguement that their agents have no incentive to manipulate their beliefs because they evaluate future trajectories based of their current beliefs about how likely they are? Does that suffice to implement eq. 1) from your motivated value selection paper?

Not really? The axioms (for hyperreals) aren't much different to that of the reals. Yes, its true that you need some strange constructions to justify that the algebra works as you'd expect. But many proofs in analysis become intuitive with this setup, and surely aid pedagogy. Admittedly, you need to learn standard analysis anyway since the tools are used in so many more areas. But I'd hardly call it ugly. 

Recall that memories are pathway dependant i.e. you can remember an "idea" when given verbal cues but not visual ones. Or given cues in the form of "complete this sentence " and "complete this totally different sentence expressing the same concept". If you memorise a sentence and can recall it any relevant context, I'd say you've basically learnt it. But just putting it into SRS on its own won't do that. Like, that's why supermemo has such a long list of rules and heuristics on how to use SRS effectively.

Thanks. Two questions:

Do the staff and faculty have a similair diversity of opinions?

Is messaging chai-info@berkeley.edu in orde to contact your peers the right procedure here?

Here's the re-written version, and thanks for the feedback.

Having an Anki deck is kind of useless in my view. When you encounter a new idea, you need to integrate it with the rest of your thoughts. For a technique, you must integrate it with your unconscious. But often, there's a tendency to just go "oh, that's useful" and do nothing with it. Putting it into space repitition software to view later won't accomplish anything since you're basically memorising the teacher's password. Now suppose you take the idea, th... (read more)

Active IRD doesn't have anything to do with corrigibility, I guess my mind just switched off when I was writing that. Anyway, how diverse are CHAI's views on corrigibility? Could you tell me who I should talk to? Because I've already read all the published stuff on it if I'm understanding you rightly and I want to make sure that all the perspectives no this topic are covered.

Hey Rohin, I'm writing a review on everything that' been written on corrigibility so far. Do the "the off switch game", "Active Inverse Reward Design" "should robots be obedient", "incorrigibility in CIRL" as well as your reply in the Newsletter represent CHAI's current views on the subject? If not, which papers contain them?

IIRC, this also shows a discontinuous flip at the bottom followed by slower change.

Maybe edit the post so you include this? I know I was wondering about this too.

Before or after what? If it is a passage in a book, or an article you wrote, I agree that's enough. But what about a nebulous concept you struggled to put into words? Or an idea which seemed to have suprising links to other thoughts, which you didn't pursue at the time. If you write all this stuff down explicitly, then fine. If not, and you're writing style is like mine, then it seems better to link to other cards and leave it to your future self to figure it out.

Plus, links provide the system extra information with which it can auto-suggest other relevant ideas that you weren't even aware you were considering.

I started writing a blog post in response, but that seems a bit much for a comment. Suffice to say, I agree that anti-spaced repetition is a good idea. However, it throws away the context of the notes you made, as well as showing it to you after your mind has totally forgotten about it. And as I wrote, those seem to be major factors in the value of the Zettlekasten method!

Yeah, I had some ideas concerning how to keep track of Zettlekasten as well as the right way to display graphs. Reinforcing the network is definetely a worthwhile idea. The entire point is to suggest good links, but also give you the freedom to traverse your graph. RE the hyperlinks: I agree about the worry of biases. But more than that, it seems the network should not automate link suggestion without leaving the option to create links yourself. As you say, the worth of the Zettlekasten method is largely in instilling virtuous mental hanits. What you suggested seems like it could instil laziness in the user.

Do you do this in a piecemeal way, or do you assign a few days to re-organising your thoughts when you learn some important new principle?

Epistemic status: unsure

I have a hypothesis about why Zettlekasten provide diminishing returns over time. A corrolary is that others should find even less value in your Zettles. Which ties into some of your points, and shows what is missing from the Zibbaldone. Plus there are some suggestions on how to correct the flaw.

One of the key benefits to the Zettlekasten is that the way you link cards reflects your psyche's understanding of the ideas. Of course other note-taking systems have this advantage. But this isn't baked into them like it is with Z... (read more)

How does this relate to the whole "no-self" thing? Is the character becoming aware of the player there?

If there is a mistake deep in the belief of someone

Are they not ideal Bayesians? Also, do they update based off other people's priors? It could be intresting to make them all ultra-finitists.

Mimemis land is confusing from the outside. I'm not sure how they could avoid stumbling upon "correct" forms of manipulating beliefs, if they persist for long enough and there are large enough stochastic shocks to the communities beliefs. If they also copid succesful people in the past, I feel like this would be even more likely. Unless they happ... (read more)

Dutch custom prevents me from recommending my own recent paper in any case

This phrase and its implications are perfect examples of problems in corrigibility. Was that intentional? If so, bravo. Your paper looks interesting, but I think I'll read the blog post first. I want a break from reading heavy papers. I wonder if the researchers would be OK with my drawing on their blog posts in the review. Would you mind?

Thanks for recommending "Reward tampering", it is much appreciated. I'll get on it after synthesising what I've read so far. Otherwise, I don't think I'll learn much.

Hey, thanks for writing all of that. My current goal is to do an up to date literature review on corrigibility, so that was a most helpful comment. I'll definitely look over your blog, since some of these papers are quite dense. Out of the paper's you recommended, is there one that stands out? Bear in mind that I've read Stewart and MIRI's papers already.

Based off what you've said in the comments, I'm guessing you'd say the various forms of corrigibility are natural abstractions. Would you say we can use the strategy you outline here to get "corrigibility by default"?

Regarding iterations, the common objection is that we're introducing optimisation pressure. So we should expect the usual alignment issues anyway. Under your theory, is this not an issue because of the sparsity of natural abstractions near human values?

This came out of the discussion you had with John Maxwell, right? Does he think this is a good presentation of his proposal?

How do we know that the unsupervised learner won't have learnt a large number of other embeddings closer to the proxy? If it has, then why should we expect human values to do well?

Some rough thoughts on the data type issue. Depending on what types the unsupervised learner provides the supervised, it may not be able to reach the proxy type by virtue of issues with NN learning processes.

Recall that tata types can be viewed as hom... (read more)

Alright, here's the link for Friday: meet.google.com/qxw-zpsi-oqn

Thanks for replying.

Hangouts I suppose. It just works. Would next weekend be OK for you?

Edit: I've scheduled a meeting for 12pm UK time on Saturday. Tell me if that works for you.

meet.google.com/kdf-xavk-nnh

Sometimes the cluster in the map a preference is pointing at involves another preference. Which provides a natural resolution mechanism. What happens when there's two preferences, I'm unsure. I suppose it depends on how your map changes. In which case, I think you should focus on how to make purity coherent you should start off with some "simple" map and various "simple" changes in the map. To make purity coherent relative to your map is both computationally hard, and empathetically hard.

Side-note: It would be interesting to ... (read more)