What are you trying to achieve with these questions?
If you just think the questions are entertaining to think about, you might find reading the Sequences worthwhile, as they and the discussions around them explore many of these questions at some length.
If you're trying to explore something more targeted, I've failed to decipher what it might be.
It may even preserve its own values to continue achieving the goal
This is backwards. An intelligent system selects goals that, if achieved, optimize the world according to its values.
I change it so I feel that way ALL THE TIME.
The local jargon for this is "wireheading." And, sure, if you don't value anything as much or more than you value pleasure (or whatever it is you get from watching Seinfeld, then you might choose to wirehead given the option.
But, again, it matters whether you value something in the first place. If something else is important to you besides pleasure, then you won't necessarily trade everything else for more pleasure.
The local consensus is that what humans value is complex; that there is no single thing (such as pleasure) to which human values reduce. Personally, I'm inclined to agree with that. But, well, you tell me: do you value anything other than pleasure?
It's a bit more than entertaining, since I plan to upload as soon as it's a mature technology. My concern is that I will degenerate into a wirehead, given the complete freedom to do so. Human values may be complex, but as far as I can tell there's no guarantee that the values of an uploaded mind will remain complex.
Will be going over the Sequences. Thanks buddy. :)
A few notes about the site mechanics
A few notes about the community
If English is not your first language, don't let that make you afraid to post or comment. You can get English help on Discussion- or Main-level posts by sending a PM to one of the following users (use the "send message" link on the upper right of their user page). Either put the text of the post in the PM, or just say that you'd like English help and you'll get a response with an email address.
* Normal_Anomaly
* Randaly
* shokwave
* Barry Cotter
A note for theists: you will find the Less Wrong community to be predominantly atheist, though not completely so, and most of us are genuinely respectful of religious people who keep the usual community norms. It's worth saying that we might think religion is off-topic in some places where you think it's on-topic, so be thoughtful about where and how you start explicitly talking about it; some of us are happy to talk about religion, some of us aren't interested. Bear in mind that many of us really, truly have given full consideration to theistic claims and found them to be false, so starting with the most common arguments is pretty likely just to annoy people. Anyhow, it's absolutely OK to mention that you're religious in your welcome post and to invite a discussion there.
A list of some posts that are pretty awesome
I recommend the major sequences to everybody, but I realize how daunting they look at first. So for purposes of immediate gratification, the following posts are particularly interesting/illuminating/provocative and don't require any previous reading:
More suggestions are welcome! Or just check out the top-rated posts from the history of Less Wrong. Most posts at +50 or more are well worth your time.
Welcome to Less Wrong, and we look forward to hearing from you throughout the site!
Note from orthonormal: MBlume and other contributors wrote the original version of this welcome post, and I've edited it a fair bit. If there's anything I should add or update on this post (especially broken links), please send me a private message—I may not notice a comment on the post. Finally, once this gets past 500 comments, anyone is welcome to copy and edit this intro to start the next welcome thread.