= 27d567bc2d48d9f40e9ccc20ea370b15)
Vladimir_Nesov comments on BOOK DRAFT: 'Ethics and Superintelligence' (part 1, revised) - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (38)
I wouldn't describe TDT as "an extension of CDT". Also, the AI being self-modifying is probably not the central issue actually addressed by TDT (as opposed to what possibly motivated development of the ideas).
"An extension of CDT" is how Yudkowsky (2010) presents TDT. The paper also says one of its central goals is to handle self-modifying AI.
A page number, rather than a reminder of the publication year, would be a more helpful parenthetical there.
The first point is made in paragraph #1:
"Timeless decision theory (TDT) is an extension of causal decision networks that compactly represents uncertainty about correlated computational processes and represents the decision-maker as such a process."
The second point is made in paragraph #2:
"I show that an evidential or causal decision-maker capable of self-modifying actions, given a choice between remaining an evidential or causal decision-maker and modifying itself to imitate a timeless decision-maker, will choose to imitate a timeless decisionmaker on a large class of problems."
http://singinst.org/upload/TDT-v01o.pdf
After a fashion, since causal networks are not exactly CDT, modeling correlated computations with causal networks makes them less "causal" (i.e. related to physical causality), and the paper doesn't achieve clear specification of how to do that (it's an open problem, but I can say that any nontrivial causal network relating computations may need to be revised in face of new logical evidence, which makes the decision procedure that itself works with resolution of logical uncertainty brittle).
That CDT/EDT agents with self-modification would become more TDT-like is somewhat different from saying that TDT "suits the needs of a self-modifying AI". TDT is a more sane theory, and to the extent CDT/EDT agents would prefer to be more effective, they'd prefer to adopt TDT's greater sanity. But TDT is not a fixed point, suiting the needs of a self-modifying AI is a tall order that probably can't be met to any reasonable extent, since that would mean establishing some rules that AI itself would not have much opportunity to rebuild. Perhaps laws of physics or logic can quality, with appropriate framing.
(I agree with your description more now than when I posted the comment, my perception of the paper having clouded the memory of its wording.)
Fair enough. Thanks for this. I've clarified the wording in my copy of this intro.
I'm familiar with the link. I suggested adding the page number because that affords significant ease to finding the passage you claim is present, even given the pdf file.
Edit: To avoid this passive-aggressive beating around the bush: the part you've posted so far gives off the vibe of bibliography padding. You've done a great service to LW in your articles reminding everyone of the importance of building off the work of others. But if there's a specific claim the author makes that you're relying on, it helps establish relevance and make things easy on the reader if you are more specific than just citing the whole work
Pages 2 and 3.
Re: your edit.
That is not standard practice when the point I'm citing the work for is in the abstract of the paper. Moreover, it's just not true that it's always standard practice to cite the page number of the specific work. I can point you to thousands of examples. That said, for books it is indeed helpful to cite page numbers.
Separate issues going on here: yes, it is common not to cite the specific page number, perhaps because one's referring to the work "as a whole". But my criticism of your paper having a vibe of bib-padding was not because of you saying "Yudkowsky (2010)" here, or any particular time in your excerpt, but rather, because taken as a whole, the excerpt looks that way. For example, there is much more citation than explanation of what the cited works are specifically there to substantiate.
Certainly, doing that a few times is fine, but here it looks more like what would result from someone trying to game the "bibliography length metric".
Independently of that, your comment on this thread was supposedly to help someone find where a paper makes a claim that others didn't remember the paper making. Regardless of the standards for academic papers, it is general etiquette that would suggest you give the page number in such a reply. And that shouldn't be hard if you know the paper well enough to be using it for that claim.
So while "Yudkowsky (2010)" might be fine for the paper, when someone questions whether it makes a specific claim, you should do a little more than just give the naked citation a second time within the same thread -- then you should give the page number.
I just disagree with all of this. The "explanation" stuff comes later; this is just the intro. If you read major works in Anglophone philosophy, you'll see that what I have here is very much the same style. You can even compare to, for example, the extended abstract submitted by Salamon & Bostrom for an edited volume on the technological singularity.
And no, you don't usually give the page number when the claims you're saying a cited work covers are in the abstract of the paper. The hope is that someone will bother to at least read the abstract of the cited paper before bothering the original author with questions about "Which page number is that on?"
"Um... page one?"
I would ask you to reconsider. As SilasBarta says, "Yudkowsky (2010)" is fine in the paper, but you used it in the comments here in response to someone's question in this forum.
You seem to assume that the only way someone could have asked that question is if they hadn't read even the abstract. But it is easy for me to imagine someone who read the whole paper, or some significant fraction, and just have missed or forgotten the claim that you attribute to Yudkowsky <embarrassed cough>. In which case, saying "It's on page one" would be helpful.
In fact, having read that significant fraction, I would be moderately surprised to hear Yudkowsky characterize TDT as an extension of CDT. He gave me the strong impression of offering an alternative to CDT, one which gets right answers where CDT is wrong. To me, calling TDT "an extension of CDT" implies that it applies to a wider range of problems than CDT, while agreeing with CDT where CDT gives a well-defined answer.
But this is a correct characterization of what TDT does. It extends applicability of CDT from action-determined to decision-determined problems.
Again, separate issues here that I think you're blurring (of I'm just being unclear): I'm not criticizing you for lacking page numbers in the paper excerpt, but for piling on whole-work citations without clarifying what specific insight it adds. I will have to retract that in light of viewing this as an introduction, since you say your paper will cover that later.
WRT page numbers, my criticism was that when someone says, "hey, that claim about [citation of X which I have already read] doesn't sound right", then you should give a more helpful answer than "oh, that's in [citation of X]" -- you should point to a more specific passage.
And yes, it would have indeed been helpful if you had said "see the abstract", because that would have told the questioner (and the onlooker, me, who was wondering the same thing) what you are basing that claim on. In this matter, for the reasons Vladimir gave, TDT isn't best regarded as an extension of CDT. So a reply that you just got it from the abstract would show that (as turned out to be the case) your claim was based on a summary rather than on a specific analysis of the mechanics of TDT and its relationship to other decision theories.
Edit: I apologize for any abrasiveness in my comments here. I sensed a kind of stubbornness and condescension in your replies and overreacted to that.
Luke quoted the source and gave a link to it in digital form. If you want to find the context, open it up in Acrobat and copy the quote into the search bar.
Warning: my comment here is based on dim memories and vague understandings. That said...
Are you sure that being an extension of causal decision networks is the same as being an extension of causal decision theory? I took "causal decision networks" to be referring to the Judea Pearl-inspired stuff. I thought that Eliezer said somewhere that CDT just assumes causal dependencies wherever it wants, while Eliezer derives them with causal networks.