All of lavalamp's Comments + Replies

An extremely low prior distribution of life is an early great filter.

Done. Thank you for running these.

Check out the previous discussion Luke linked to: http://lesswrong.com/lw/c45/almost_every_moral_theory_can_be_represented_by_a/

It seems there's some question about whether you can phrase deontological rules consequentially-- to make this more formal that needs to be settled. My first thought is that the formal version of this would say something along the lines of "you can achieve an outcome that differs by only X%, with a translation function that takes rules and spits out a utility function, which is only polynomially larger." It's not clear t... (read more)

(Sorry for slow response. Super busy IRL.)

If a consequentialism talks about murder being bad, they mean that it's bad if anybody does it.

Not necessarily. I'm not saying it makes much sense, but it's possible to construct a utility function that values agent X not having performed action Y, but doesn't care if agent Z performs the same action.

It is technically true that all of these ethical systems are equivalent, but saying which ethical system you use nonetheless carries a lot of meaning.

a) After reading Luke's link below, I'm still not certain if... (read more)

If indeed the frameworks are isomorphic, then actually this is just another case humans allowing their judgment to be affected by an issue's framing. Which demonstrates only that there is a bug in human brains.

I think so. I know they're commonly implemented without that feedback loop, but I don't see why that would be a necessary "feature".

Which is why I said "in the limit". But I think, if it is true that one can make reasonably close approximations in any framework, that's enough for the point to hold.

Hm, thanks.

Are you saying that some consequentialist systems don't even have deontological approximations?

It seems like you can have rules of the form "Don't torture... unless by doing the torture you can prevent an even worse thing" provides a checklist to compare badness ...so I'm not convinced?

How does it change the numbers if you condition on the fact that Alcor has already been around for 40 years?

Absolutely, granted. I guess I just found this post to be an extremely convoluted way to make the point of "if you maximize the wrong thing, you'll get something that you don't want, and the more effectively you achieve the wrong goal, the more you diverge from the right goal." I don't see that the existence of "marketing worlds" makes maximizing the wrong thing more dangerous than it already was.

Additionally, I'm kinda horrified about the class of fixes (of which the proposal is a member) which involve doing the wrong thing less effect... (read more)

90% agree, one other thing you may not know: both dropbox and google drive have options to automatically upload photos from your phone, and you don't have to sync your desktop with them. So it's not clear that they merely double the needed space.

I think your expanded point #6 fails to consider alternative pressures for hard drive & flash memory. Consider places like dropbox; they represent a huge demand for cheap storage. People probably (?) won't want huge(er) drives in their home computers going forward, but they are quite likely to want cloud storage if it comes down another order of magnitude in price. Just because people don't necessarily directly consume hard drives doesn't mean there isn't a large demand.

Consider also that many people have high MP digital cameras, still and video. Those files add up quickly.

It sounds like, "the better you do maximizing your utility function, the more likely you are to get a bad result," which can't be true with the ordinary meanings of all those words. The only ways I can see for this to be true is if you aren't actually maximizing your utility function, or your true utility function is not the same as the one you're maximizing. But then you're just plain old maximizing the wrong thing.

Hypocrisy is only a vice for people with correct views. Consistently doing the Wrong Thing is not praiseworthy.

Unfortunately, it's much easier to demonstrate inconsistency than incorrectness.

Ah, thank you for the explanation. I have complained about the proposed method in another comment. :)

http://lesswrong.com/lw/jao/siren_worlds_and_the_perils_of_overoptimised/aso6

The IC correspond roughly with what we want to value, but differs from it in subtle ways, enough that optimising for one could be disastrous for the other. If we didn't optimise, this wouldn't be a problem. Suppose we defined an acceptable world as one that we would judge "yeah, that's pretty cool" or even "yeah, that's really great". Then assume we selected randomly among the acceptable worlds. This would probably result in a world of positive value: siren worlds and marketing worlds are rare, because they fulfil very specific criteri

TL;DR: Worlds which meet our specified criteria but fail to meet some unspecified but vital criteria outnumber (vastly?) worlds that meet both our specified and unspecified criteria.

Is that an accurate recap? If so, I think there's two things that need to be proven:

1. There will with high probability be important unspecified criteria in any given predicate.

2. The nature of the unspecified criteria is such that it is unfulfilled in a large majority of worlds which fulfill the specified criteria.

(1) is commonly accepted here (rightly so, IMO). But (2) seems... (read more)

Possibly of interest: Help Teach 1000 Kids That Death is Wrong. http://www.indiegogo.com/projects/help-teach-1000-kids-that-death-is-wrong

(have not actually looked in detail, have no opinion yet)

I think you're getting downvoted for your TL;DR, which is extremely difficult to parse. May I suggest:

TL;DR: Treating "computers running minds" as discrete objects might cause a paradox in probability calculations that involve self-location.

I dunno. I'd be pretty happy with a system that produced reasonable output when staffed with idiots, because that seems like a certainty. I actually think that's probably why democracy seems to be better than monarchies-- it has a much lower requirement for smarts/benevolence. "Without suffering" may be a high bar, but the universe is allowed to give us problems like that! (And I don't think that democracy is even close to a complete solution.)

EDIT: Also, perhaps the entirety of the system should be to make sure that an "utter genius with leet rationality skillz" is in the top position? I'd be very happy with a system that caused that even when staffed by morons.

I think we just mean different things by "human level"-- I wouldn't consider "human level" thought running at 1/5th the speed of a human or slower to actually be "human level". You wouldn't really be able to have a conversation with such a thing.

And as Gurkenglas points out, the human brain is massively parallel-- more cores instead of faster cores is actually desirable for this problem.

Ah, I see. Yeah, 1 bit in input bitstream != 1 bit of bayesian evidence.

I think you mean BB(1000) bits of evidence?

1) Yes, brains have lots of computational power, but you've already accounted for that when you said "human-level AI" in your claim. A human level AI will, with high probability, run at 2x human speed in 18 months, due to Moore's law, even if we can't find any optimizations. This speedup by itself is probably sufficient to get a (slow-moving) intelligence explosion.

2) It's not read access that makes a major difference, it's write access. Biological humans probably will never have write access to biological brains. Simulated brains or AGIs probabl... (read more)

I see, thanks!

You can't "count every possible program equally".

I did know this and should have phrased my sentence hypothetically. :)

The only programs allowed in the Solomonoff distribution are ones that don't have any extended versions that produce the same output observed so far.

Did not know that! It seems like that would leave some probability mass unassigned, how do you rebalance? Even if you succeed, it seems likely that (for large enough outputs) there'll be lots of programs that have epsilon difference--that are basically the same, for all practical purposes.

I have been thinking that the universal prior is tautological. Given a program, there are an infinite number of longer programs which perform the same computation (or an indistinguishable variation) but only a finite number of shorter programs having this characteristic. If you count every possible program equally, you'll find that each short program represents a host of longer programs. However, now that I write this down, I'm no longer sure about it. Can someone say why/if it's wrong?

I'd like to agree with you, but how do I know you're not a concern troll?

I believe if you read my previous comments, you'll see that they all are attempts to do exactly this. I will bow out of this conversation now.

(Meta: you're tripping my troll sensers. I'm sorry if it's unintentional on your part. I'm just not getting the sense that you're trying to understand me. Or it's the case that the two of us just really cannot communicate in this forum. Either way, it's time to call it quits.)

EDIT: Your response to this has caused my P(you're trolling me) to rise from ~60% to ~95%.

Ah, you're disagreeing with the model and phrasing it as "if that model were true, no one would sell you btc, but people are willing to sell, therefore that model is false." Do I understand?

If so, I do not agree that "if that model were true, no one would sell you btc" is a valid inference.

You have mentioned the Mathematical Universe hypothesis several times, and Tegmark's is a name very much associated with it, ...

Right, I know what the MUH is, I know who Tegmark is, I just don't recognize terms that are a combination of his name and (im)materialism. Please taboo your terms! I don't know what they mean to you!

If anything, let's call my position "non-distinctionalism"-- I maintain that there's no other coherent models for the word "exist" than the one I mentioned earlier, and people who use the word "exist"

What Vaniver said. Also, emperically, you can look at the current price/order book on an exchange and see that people are in fact willing to sell you these things. If my holdings represented a life altering sum of money it would be time to take less risk and I would be one of those people.

The computer doesn't hold cash (clearly), it has account # and password of a bank account (or private key to bitcoin addresses if it's a particularly risky computer). The two thin computers therefore only have half as much money to bet. (Or they'll overdraw their account.)

Bitcoin is plenty liquid right now unless you're throwing around amounts > $1 mil or so.

Thanks for editing-- I'm still puzzled.

I also don't know what "Tegmarkian immaterialism" is and I'm not arguing for or against it. I do not know what "immaterialism" is and I'm also not arguing for or against that. (Meta: stop giving either sides of my arguments names without first giving the names definitions!)

If anything, let's call my position "non-distinctionalism"-- I maintain that there's no other coherent models for the word "exist" than the one I mentioned earlier, and people who use the word "exist"... (read more)

I think we are having very significant communication difficulties. I am very puzzled that you think that I think that I'm arguing against MUH. I think something like MUH is likely true. I do not know what "Tegmarkian materialism" is and I'm not defending or attacking it. I also cannot make sense of some of your sentences, there seems to be some sort of editing problem.

So far, none of this tells us what immateriality is. But then it isn't easy to say what matter is either.

Yeah. There's supposedly two mysterious substances. My claim is that I can't see a reason to claim they're separate, and this thought experiment is (possibly) a demonstration that they're in fact same thing. Then we still have one mysterious substance, and I'm not claiming to make that any less mysterious with this argument.

There are more than two options. If you had evidence of a bitstring corresponding to a billions of years of biological develop

What would "immaterial existence" even mean?

I don't know exactly, but if "material existence" means something, so does "immaterial existence".

Hm. I don't think "material existence", if it's a thing, has a unique opposite.

I guess I'd define exists-in-the-real-world as equivalent to a theoretical predicate function that takes a model of a thing, and a digitized copy of our real world, and searches the world for patterns that are functionally isomorphic (given the physics of the world) to the model, and returns tr... (read more)

What would "immaterial existence" even mean?

I think my claim is that the above argument shows that whatever that might be, it's equivalent to epistemological objectivism.

Specifically, to believe that they're separate, given the scenario where you simulate universes until you find a conscious mind and then construct a replica in your own universe, you have to believe both of the following at the same time:

(1) Mind X didn't have real memories/experiences until you simulated it in the "real" world (i.e., yours), and (2) proof of mind X's r... (read more)

Perhaps this isn't obvious, but note that fully validating nodes do not need to store the entire block chain history, currently, just the set of unspent transaction outputs, and there are proposals to eliminate the need for validators to store even that. If it's just a gentleman's agreement then it would be doable but wouldn't really have any teeth against a motivated attacker.

That's a good point. To make this work, it'd probably make the most sense to treat the pre-published hash the same as unspent outputs. It can't be free to make these or you could ... (read more)

Infinitely is a bit of an overstatement, especially if there's a fee to store a hash. I agree it might still be prudent to have a time limit, though. Miners can forget the hash once it's been referenced by a transaction.

The ability to pay to store a hash in the blockchain could be interesting for other reasons, like proving knowledge at a particular point in the past. There's some hacky ways to do it now that involve sending tiny amounts of BTC to a number of addresses, or I suppose you could also hash your data as if it were a pubkey and send 1e-8 btc to that hash-- but that's destructive.

Thanks for the explanation, it seems like I'm not wildly misreading wikipedia. :)

It seems like the more qubits are required for this attack, the more likely we are to have a long warning time to prepare ourselves. The other attack of just cracking the pubkey when a transaction comes through and trying to beat the transaction, seems vastly more likely to be an actual problem.

Do you have any idea how I'd go about estimating the number of qubits required to implement just the SHA256(SHA256(...)) steps required by mining?

They don't.

Suppose you publish hash HT1 of a transaction T1 spending address A, and then several blocks later when you publish T1 itself, someone hacks your pubkey and publishes transaction T2 also spending address A. Miners would hypothetically prefer T1 to T2, because there's proof that T1 was made earlier.

In the case where someone had even earlier published hash HT0 of transaction T0 also spending address A, but never bothers to publish T0 (perhaps because their steal bot--which was watching for spends from A--crashed), well, they're out of luck, becau... (read more)

I always forget about the RIPEMD-160 step. The bitcoin wiki claims that it's strictly more profitable to mine than to search for collisions, but it seems to me that that's a function of block reward vs. value of address you're trying to hack, so I don't know if I believe that. https://en.bitcoin.it/wiki/Technical_background_of_Bitcoin_addresses

It's unclear to me how you would actually implement this in a quantum computer; do you have to essentially build a set of quantum gates that implement RIPEMD-160(SHA256(pubkey))? Does this imply you need enough qubit... (read more)

I think you misunderstood me-- the transaction could still be rejected when you try to get it included in a subsequent block if it's not valid. The hash of the transaction is just to prove that the transaction is the first spend from the given address; the transaction doesn't/can't get checked when the hash is included in the blockchain. Miners wouldn't be able to do it for free-- the protocol addition would be that you pay (from a quantum-safe address) to include a hash of a transaction into the blockchain. You publish the actual transaction some number o... (read more)

Me three-- I thought I was the only one, where are we all hiding? :)

That's true, but there's some precedent for picking a block that everyone agrees upon (that exists before quantum computers) and changing the rules at that block to prevent someone from rewriting the blockchain. A lot depends on how much warning we have.

It looks like making a cryptocoin that can survive quantum computers might be a high value activity.

I think a gigahertz quantum computer would have the hashing power of the current bitcoin network.

My math agrees with you. Looks like I was underestimating the effect of quantum computers.

Difficulty is currently growing at 30-40% per month. That won't last forever, obviously, but we can expect it to keep going up for a while, at least. (https://blockchain.info/charts/hash-rate) Still, it looks like you'd need an unrealistic amount of ASICs to match the output of just 1000 quantum computers.

Given that, there'll probably be a large financial incentive to m... (read more)

This is not completely true-- since only hashes of the public key are posted until funds are spent from an address

So what? Then the attacker waits till someone spents his funds and double spends them and gives the double spending transaction a high processing fee.

This can be fixed by a protocol addition, which can be implemented as long as there's warning. (First publish a hash of your transaction. Once that's been included in a block, publish the transaction. Namecoin already does something like this to prevent that exact attack.)