Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Comment author: SilentCal 21 April 2015 05:17:11PM *  6 points [-]

But you can also write the opposite parable:

Crusoe has finished planting his most fertile fields and is preparing to plant his remaining seed in the less fertile adjoining land when the stranger arrives.

S: I have found some excellent fields, too far from your home here to be of any use to you, but which will serve me well; and I need only some seed. I see you have some excess, which will be much more productive in my fields than in yours; might I borrow it, and easily repay after harvest?

RC: How much seed will you repay me, for each I lend you now?

S: One seed for each; my religion would forbid any other arrangement.

RC: The land I intend to plant them in is not so lush, but even so I expect to reap many times the seed I plant. Where is my self-interest in placing it where it will yield less?

Moral: some goods decay, but others can compound themselves--or, more often, create other goods that can create other goods in a long chain that ends with more of the first good.

The role of money is to make such long chains implicit; a positive real interest rate reflects that such chains are possible for most goods, and any exceptions will increase in price faster than interest accumulates.

Comment author: Pentashagon 28 April 2015 04:53:34AM 1 point [-]

In your story RC incurs no opportunity cost for planting seed in and tending a less efficient field. There should be an interest rate as a function for lending the last Nth percent of the seed based on the opportunity cost of planting and harvesting the less efficient field, which at some point crosses 0 and becomes negative. The interest rate drops even more quickly once his next expected yield will be more than he can eat or store for more than a single planting season.

If RC is currently in the situation where his desired interest rate is still positive for his last unplanted seed then his capital is constrained and he should instead ask for investment seed from S, for which he would be willing to pay interest.

In order not to starve RC should aim to grow sufficient grain such that his probability of having too little is lower than some risk threshold. In most cases this will leave him with excess seeds at every harvest (beyond the extra seeds required to avoid starvation risk) which he can lend. Depending on his assessment of the loan risk, he may even be able to save himself time and trouble by growing less grain to produce fewer seeds with the expectation that his loan will be repaid, which would allow him to realize a profit on an otherwise zero-interest loan.

Likewise, money below a certain threshold should compound; beyond that threshold it represents unacceptable opportunity costs for exercising its power as if it had been used to purchase excess goods. Investing/loaning those purchased goods should be the basis for money's value beyond the threshold.

Comment author: Nanashi 15 April 2015 11:45:06AM 2 points [-]

Thank you by the way for actually including an example of such an attack. The discussion between ChristianKI and myself covered about 10 different subjects so I wasn't exactly sure what type of attack you were describing.

You are correct, in such an attack it would not be a question of trusting OpenPGP. It's a general question of trusting software. These vulnerabilities are common to any software that someone might choose to download.

In this case, I would argue that a transparent, sandboxed programming language like javascript is probably one of the safer pieces of "software" someone can download. Especially because browsers basically treat all javascript like it could be malicious.

Comment author: Pentashagon 18 April 2015 03:47:16AM 2 points [-]

In this case, I would argue that a transparent, sandboxed programming language like javascript is probably one of the safer pieces of "software" someone can download. Especially because browsers basically treat all javascript like it could be malicious.

Why would I paste a secret key into software that my browser explicitly treats as potentially malicious? I still argue that trusting a verifiable author/distributor is safer than trusting an arbitrary website, e.g. trusting gpg is safer than trusting xxx.yyy.com/zzz.js regardless of who you think wrote zzz.js, simply because it's easier to get that wrong in some way than it is to accidentally install an evil version of gpg, especially if you use an open source package manager that makes use of PKI, or run it from TAILS, etc. I am also likely to trust javascript crypto served from https://www.gnupg.org/ more than from any other URL, for instance.

In general I agree wholeheartedly with your comment about sandboxing being important. The problem is that sandboxing does not imply trusting. I think smartphone apps are probably better sandboxed, but I don't necessarily trust the distribution infrastructure (app stores) not to push down evil updates, etc. Sideloading a trusted app by a trusted author is probably a more realistic goal for OpenPGP for the masses.

Comment author: Nanashi 13 April 2015 10:08:59PM *  2 points [-]

You're mostly correct. The data is encrypted, and then broken into a base-4 string. The least significant base-4 bit is dropped from each pixel leaving 98.4% fidelity, which is higher fidelity than the compression that gets applied. Thus in terms of image quality, the picture is indistinguishable from any other compressed image.

The encoding is deliberately reversible and also open-sourced. However, you can apply the same algorithm to any image, whether it's a decoy or not, and get a string of possibly-encrypted-data. The only confirmation that the data is meaningful would be a successful decryption which is only possible with the correct passphrase.

All that said, the fact that the picture is indistinguishable from other non-decoy images only adds a trivial amount of entropy to the encryption. An attacker who is determined to brute force their way into your pictures can simply attempt to crack every picture in your camera roll, decoy or no.

Comment author: Pentashagon 15 April 2015 07:48:48AM 2 points [-]

Does it change the low bits of white (0xFFFFFF) pixels? It would be a dead giveaway to find noise in overexposed areas of a photo, at least with the cameras I've used.

Comment author: Nanashi 14 April 2015 11:15:27AM 5 points [-]

Remember that I did not invent the PGP protocol. I wrote a tool that uses that protocol. So, I don't know if what you are suggesting is possible or not. But I can make an educated guess.

If what you are suggesting is possible, it would render the entire protocol (which has been around for something like 20 years) broken, invalid and insecure. It would undermine the integrity of vast untold quantities of data. Such a vulnerability would absolutely be newsworthy. And yet I've read no news about it. So of the possible explanations, what is most probable?

  1. Such an obvious and easy to exploit vulnerability has existed for 20ish years, undiscovered/unexposed until one person on LW pointed it out?

  2. The proposed security flaw sounds like maybe it might work, but doesnt.

I'd say #2 is more probable by several orders of magnitude

Comment author: Pentashagon 15 April 2015 07:01:44AM 5 points [-]

Such an obvious and easy to exploit vulnerability has existed for 20ish years, undiscovered/unexposed until one person on LW pointed it out?

It's not a vulnerability. I trust gnupg not to leak my private key, not the OpenPGP standard. I also trust gnupg not to delete all the files on my hard disk, etc. There's a difference between trusting software to securely implement a standard and trusting the standard itself.

For an even simpler "vulnerability" in OpenPGP look up section 13.1.1 in RFC4880; encoding a message before signing. Just replace the pseudo-random padding with bits from the private key. Decoding (section 13.1.2) does not make any requirements on the content of PS.

Comment author: Nanashi 14 April 2015 04:55:03PM 3 points [-]

Let's assume you CAN leak arbitrary amounts of information into a PGP signature.

  1. Short of somehow convincing the victim to send you a copy of their message, you have no means of accessing your recently-leaked data. And since that is extremely unlikely, your only hope is to view a public message the user posts with their compromised signature. Which leads to....

  2. That leaked data would be publicly available. Anyone with knowledge of your scheme would also be able to access that data. Any encryption would be worthless because the encryption would take place client-side and all credentials thus would be exposed to the public as well. Which brings us to....

  3. Because the script runs client-side, it also makes it extremely easy for a potential victim to examine your code to determine if it's malicious or not. And, even if they're too lazy to do so...

  4. A private key is long. A PGP signature is short. So your victim's compromised signature would be 10x longer than the length of a normal PGP signature.

So yes, you all are correct. If I had malicious intent, I could write an attack that 1. could be immediately exposed to the public by any person with programming knowledge, 2. provides an extremely obvious telltale sign to the victim that something malicious is going on, and 3. doesn't actually provide me any benefit.

Comment author: Pentashagon 15 April 2015 06:24:19AM 0 points [-]
  1. Short of somehow convincing the victim to send you a copy of their message, you have no means of accessing your recently-leaked data.

Public-key signatures should always be considered public when anticipating attacks. Use HMACs if you want secret authentication.

  1. That leaked data would be publicly available. Anyone with knowledge of your scheme would also be able to access that data. Any encryption would be worthless because the encryption would take place client-side and all credentials thus would be exposed to the public as well.

You explicitly mentioned Decoy in your article, and a similar method could be used to leak bits to an attacker with no one else being able to recover them. We're discussing public key encryption in this article which means that completely public javascript can indeed securely encrypt data using a public key and only the owner of the corresponding private key can decrypt it.

  1. Because the script runs client-side, it also makes it extremely easy for a potential victim to examine your code to determine if it's malicious or not. And, even if they're too lazy to do so...

Sure, the first five or ten times it's served. And then one time the victim reloads the page, the compromised script runs, leaks as much or all of the private key as possible, and then never gets served again.

  1. A private key is long. A PGP signature is short. So your victim's compromised signature would be 10x longer than the length of a normal PGP signature.

An exported private key is long because it includes both factors, the private exponent, and the inverse of p mod q. In my other comment I was too lazy to decode the key and extract one of the RSA factors, but one factor will be ~50% of the size of the RSA signature and that's all an attacker needs.

Comment author: Nanashi 14 April 2015 11:15:27AM 5 points [-]

Remember that I did not invent the PGP protocol. I wrote a tool that uses that protocol. So, I don't know if what you are suggesting is possible or not. But I can make an educated guess.

If what you are suggesting is possible, it would render the entire protocol (which has been around for something like 20 years) broken, invalid and insecure. It would undermine the integrity of vast untold quantities of data. Such a vulnerability would absolutely be newsworthy. And yet I've read no news about it. So of the possible explanations, what is most probable?

  1. Such an obvious and easy to exploit vulnerability has existed for 20ish years, undiscovered/unexposed until one person on LW pointed it out?

  2. The proposed security flaw sounds like maybe it might work, but doesnt.

I'd say #2 is more probable by several orders of magnitude

Comment author: Pentashagon 15 April 2015 05:42:36AM *  2 points [-]

NOTE: lesswrong eats blank quoted lines. Insert a blank line after "Hash: SHA1" and "Version: GnuPG v1".

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Secure comment
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQgEBAEBAgduBQJVLfkkxqYUgAAAAAAKB1NzZWNyZXRAa2V5LS0tLS1CRUdJTiBQ
R1AgUFJJVkFURSBLRVkgQkxPQ0stLS0tLXxWZXJzaW9uOiBHbnVQRyB2MXx8bFFI
WUJGVXQ5WU1CQkFESnBtaGhjZXVqSHZCRnFzb0ErRnNTbUtCb3NINHFsaU9ibmFH
dkhVY0ljbTg3L1IxZ3xYNFJURzFKMnV4V0hTeFFCUEZwa2NJVmtNUFV0dWRaQU56
RVFCQXNPR3VUQW1WelBhV3ZUcURNMGRKbHEzTmdNfG1Edkl2a1BJeHBoZm1KTW1L
YmhQcTBhd3ArckFSU3BST01pMXMvWUtLRWEweUdYaFN6MG1uZkYrZ3dBUkFRQUJ8
QUFQL1MrRjBWdkxzOW5HZWZjRFNpZ0hyRjNqYXAvcCtSNTArNGdDenhuY3djelBJ
dXR5NU1McFF5NHMxQVZ2T3xNcDZrZFFDV2pVUXdWZTc4WEF3WjNRbEh5dkVONDdx
RDZjNVdOMGJuTGpPTEVIRE9RSTNPQi9FMUFrNzlVeXVRfFQ0b21IVWp5MlliVWZj
VnRwZWJOR3d4RkxpV214RW1QZG42ZGNLVFJzenAzRDdFQ0FOSWxYbWVTbXR4WFRO
REp8REFrOUdoa1N6YnoyeFladmxIekZHb0ltRmU4NGI5UGZ5MEV1dHdYUFFmUUl0
VTBGTExxbkdCeStEWnk2MmpUc3xTMDlWbkpFQ0FQV21kZXhkb1ZKSjJCbUFINHE2
RlRCNXhrUnJMTzNIRk1iZU5NT2Z2T2ducy9Fa2czWjJQcnpHfG43RGdVU1FIZTFp
UHJJODJ0VmJYYXR6RE1xMnZ3OU1DQUt5SmtONnVzUFZkVHF5aVFjMDN6anJWMUNu
YkNrK1h8WVNtekpxcFd0QzVReWlycUp3ODlWQ2dBaDh4YlorWnI0NlY2R3VhdkdD
azdPbGIzQnE1a2V4ZWU2YlFMY0dWdXxkR0Z6YUdGbmIyNkl1QVFUQVFJQUlnVUNW
UzMxZ3dJYkF3WUxDUWdIQXdJR0ZRZ0NDUW9MQkJZQ0F3RUNIZ0VDfEY0QUFDZ2tR
YTJuMThBNWR2N0owVmdQNkFzdjBrS1ZWb050ZE5WSklHWThLN2I2L1l0ZWlJNFpa
NUJtL2YzUFF8WUpCRVVGWTljV1E2TVlZWFFlYm9TWHN1amN2cWJJMkpERFZ5dDFR
SCtXdk00dFhiNmdmaGp1a2hobmxaTUNnSnx0eXp1aHdZWHloZGVaMFZmb0hOeUxP
WHQyL1VvWCtsdVd4aWhkN1Exd2IrNjljVDV1V1IrYVEwK3h6SXJpVUdlfFBReWRB
ZGdFVlMzMWd3RUVBTXU4bWc1cmZMNERnNE5TaHNDc2YyQkd2UnJhZGRDcmtxTk40
ckNwNkdCUXBGQ018MVJldGIwYURQSkhsbWpnaWdOUzBpQTgvWXdyUGx0VktieW9r
S2NXZklmYTlmNjE1SmhwNHM3eEFXSUlycGNwaHxPdjlGakRsUldYd09PbXFBYzB5
dVV4WjN2Z2JERUZPWGRuQWk2ZDJDV0Y5a1B5UTlQbG5zL3gxcGtLS0xBQkVCfEFB
RUFBL29DMmsrTWwzbGdybXMvVnlsOGl5M01GYWJTT0hBMmpYWE9oRDhDQlptenQ0
MWF5ZzRMSXlvNnQ0aGl8bHBvZWpScDJ0VmNaRE9TQWVKV3BHT2k0Nkt3T1g1VXdW
bUI4ZldTbTJobHZxbWJ0ckNWUGUzZGQzZGVCMlM2RXxsTW5qa0YxWWtDYVl5ZGZo
Mi9BQ2lpT1RrNGZPREdzdVh1eU9jKytQSUwxVllxMVJjUUlBemk2bzZFMVhYTnpV
fEJmMUs3clZ2N3luMVJBRmZ1aWkrOFA1OGNtWnVheld0WVA0bTlVNTdLNjhHN0lH
QTRINUNYa1pTS1A0bDdTWHR8ZWQ2b01vZmlVd0lBL1Bhc2hqUnJXSUVBSDk4bEJR
aXdISmZWUlBsR1R6YU92Q0I3TXYyamZIdnlCR0lvTkF0aXx1ZXByT0VTMHZUNysy
eklaU201ei9rTG03UytzV3RNbjZRSUFrend6bTdRRFhLbjNiSm9BUEgvL2dOdWlY
NHRkfFNlSHJSNTJUTmhmTzJqTEZKU040K1pjMktnTkNDYVlzQ0haUEkrc214YWQ1
YU1BeG5qN3JXRlNSWTV2RmlKOEV8R0FFQ0FBa0ZBbFV0OVlNQ0d3d0FDZ2tRYTJu
MThBNWR2N0wrZ2dQL1hVN3IzR1I2bVRsanA5SVBHQXJ2aEVhNHxRZlBSbWIzWEly
ekJBVVR0Ti9KZXA1cFVUcno0N1pQcHdkckJnZnFvOXUweDgwUCtKdlY4azR0MGpX
c09nUlFyfDQrazhMRTFMSVBFbTl2Q2h0aVd4V2Z6eGNUSUF6ZXdhN20vZ2VscU1S
aGJibVNLeGdZNkhUV2pVYml6Q3ZsQit8Z0Q5UGRMNjU4RThUQkZxSlliUT18PU1l
VEl8LS0tLS1FTkQgUEdQIFBSSVZBVEUgS0VZIEJMT0NLLS0tLS18AAoJEGtp9fAO
Xb+yVvEEAJkkFIaHkFJ6OTKrKge/7+C9Vn+IkoMIq1bqsyyhClMVnuqiAG6fhqzv
glYeeVxtqYac9ecSzIbuszaHckcdA/q2onyeXjW1nfaBy2EdsxGuwCxvr+ac17v2
HhSKU/BXYMNRm7vLjDnRq99ON5+1F6IwY7rmuMJZuVOYEqPBdaTs
=FDzi
-----END PGP SIGNATURE-----

Output of gpg --verify:

gpg: Signature made Tue 14 Apr 2015 10:37:40 PM PDT using RSA key ID 0E5DBFB2
gpg: Good signature from "pentashagon"
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: B501 B12E 5184 8694 4557 01FC 6B69 F5F0 0E5D BFB2

Output of gpg -vv --verify:

gpg: armor: BEGIN PGP SIGNED MESSAGE
gpg: armor header: Hash: SHA1
:packet 63: length 19 - gpg control packet
gpg: armor: BEGIN PGP SIGNATURE
gpg: armor header: Version: GnuPG v1
:literal data packet:
mode t (74), created 0, name="",
raw data: unknown length
gpg: original file name=''
:signature packet: algo 1, keyid 6B69F5F00E5DBFB2
version 4, created 1429076260, md5len 0, sigclass 0x01
digest algo 2, begin of digest 56 f1
hashed subpkt 2 len 4 (sig created 2015-04-15)
hashed subpkt 20 len 1893 (notation: secret@key=-----BEGIN PGP PRIVATE KEY BLOCK-----|Version: GnuPG v1||lQHYBFUt9YMBBADJpmhhceujHvBFqsoA+FsSmKBosH4qliObnaGvHUcIcm87/R1g|X4RTG1J2uxWHSxQBPFpkcIVkMPUtudZANzEQBAsOGuTAmVzPaWvTqDM0dJlq3NgM|mDvIvkPIxphfmJMmKbhPq0awp+rARSpROMi1s/YKKEa0yGXhSz0mnfF+gwARAQAB|AAP/S+F0VvLs9nGefcDSigHrF3jap/p+R50+4gCzxncwczPIuty5MLpQy4s1AVvO|Mp6kdQCWjUQwVe78XAwZ3QlHyvEN47qD6c5WN0bnLjOLEHDOQI3OB/E1Ak79UyuQ|T4omHUjy2YbUfcVtpebNGwxFLiWmxEmPdn6dcKTRszp3D7ECANIlXmeSmtxXTNDJ|DAk9GhkSzbz2xYZvlHzFGoImFe84b9Pfy0EutwXPQfQItU0FLLqnGBy+DZy62jTs|S09VnJECAPWmdexdoVJJ2BmAH4q6FTB5xkRrLO3HFMbeNMOfvOgns/Ekg3Z2PrzG|n7DgUSQHe1iPrI82tVbXatzDMq2vw9MCAKyJkN6usPVdTqyiQc03zjrV1CnbCk+X|YSmzJqpWtC5QyirqJw89VCgAh8xbZ+Zr46V6GuavGCk7Olb3Bq5kexee6bQLcGVu|dGFzaGFnb26IuAQTAQIAIgUCVS31gwIbAwYLCQgHAwIGFQgCCQoLBBYCAwECHgEC|F4AACgkQa2n18A5dv7J0VgP6Asv0kKVVoNtdNVJIGY8K7b6/YteiI4ZZ5Bm/f3PQ|YJBEUFY9cWQ6MYYXQeboSXsujcvqbI2JDDVyt1QH+WvM4tXb6gfhjukhhnlZMCgJ|tyzuhwYXyhdeZ0VfoHNyLOXt2/UoX+luWxihd7Q1wb+69cT5uWR+aQ0+xzIriUGe|PQydAdgEVS31gwEEAMu8mg5rfL4Dg4NShsCsf2BGvRraddCrkqNN4rCp6GBQpFCM|1Retb0aDPJHlmjgigNS0iA8/YwrPltVKbyokKcWfIfa9f615Jhp4s7xAWIIrpcph|Ov9FjDlRWXwOOmqAc0yuUxZ3vgbDEFOXdnAi6d2CWF9kPyQ9Plns/x1pkKKLABEB|AAEAA/oC2k+Ml3lgrms/Vyl8iy3MFabSOHA2jXXOhD8CBZmzt41ayg4LIyo6t4hi|lpoejRp2tVcZDOSAeJWpGOi46KwOX5UwVmB8fWSm2hlvqmbtrCVPe3dd3deB2S6E|lMnjkF1YkCaYydfh2/ACiiOTk4fODGsuXuyOc++PIL1VYq1RcQIAzi6o6E1XXNzU|Bf1K7rVv7yn1RAFfuii+8P58cmZuazWtYP4m9U57K68G7IGA4H5CXkZSKP4l7SXt|ed6oMofiUwIA/PashjRrWIEAH98lBQiwHJfVRPlGTzaOvCB7Mv2jfHvyBGIoNAti|ueprOES0vT7+2zIZSm5z/kLm7S+sWtMn6QIAkzwzm7QDXKn3bJoAPH//gNuiX4td|SeHrR52TNhfO2jLFJSN4+Zc2KgNCCaYsCHZPI+smxad5aMAxnj7rWFSRY5vFiJ8E|GAECAAkFAlUt9YMCGwwACgkQa2n18A5dv7L+ggP/XU7r3GR6mTljp9IPGArvhEa4|QfPRmb3XIrzBAUTtN/Jep5pUTrz47ZPpwdrBgfqo9u0x80P+JvV 8k4t0jWsOgRQr|4+k8LE1LIPEm9vChtiWxWfzxcTIAzewa7m/gelqMRhbbmSKxgY6HTWjUbizC vlB+|gD9PdL658E8TBFqJYbQ=|=MeTI|-----END PGP PRIVATE KEY BLOCK-----|)
subpkt 16 len 8 (issuer key ID 6B69F5F00E5DBFB2)
data: [1024 bits]
gpg: Signature made Tue 14 Apr 2015 10:37:40 PM PDT using RSA key ID 0E5DBFB2
gpg: using PGP trust model
gpg: Good signature from "pentashagon"
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: B501 B12E 5184 8694 4557 01FC 6B69 F5F0 0E5D BFB2
gpg: textmode signature, digest algorithm SHA1

I ran the exported (unencrypted) private key through tr '\n' '|' to get a single line of text to set, and created the signature with:

gpg -a --clearsign --sig-notation secret@key="exported-secret-key-here" -u pentashagon

Let me know if your OpenPGP software of choice makes it any more clear that the signature is leaking the private key without some sort of verbose display.

Comment author: shminux 09 April 2015 03:22:12PM 5 points [-]

I would draw the semantic line between the third and the fourth process.

Scott Aaronson draws the same line, but for a different reason: Boltzmann brains are reversible processes, therefore the usual ideas do not apply. He argues that it is plausible that irreversibility is a requirement for consciousness.

Comment author: Pentashagon 15 April 2015 05:14:49AM 0 points [-]

I can imagine how decoherence explains why we only experience descent along a single path through the multiverse-tree instead of experiencing superposition, but I don't think that's sufficient to claim that all consciousness requires decoherence.

An interesting implication of Scott's idea is that consciousness is timeless, despite our experience of time passing. For example, put a clock and a conscious being inside Schrödinger’s box and then either leave it in a superposition forever or open it at some point in the future. If we don't open the box, in theory nothing is conscious of watching the clock as time passes. If we open the box, there's a conscious being who can describe all the time inside the box watching the clock. When, to the outside observer, does that conscious experience happen? Either all the conscious experience happens the instant the box is measured, contrary to our notions of the experience of time passing and our understanding of how the physical state of the clock changes (e.g. the conscious experience of seeing the clock read 3:52 PM on Thursday should have happened at 3:52 PM on Thursday when the clock inside the box was in a superposition of physically displaying that time with very high probability), or else there would have been conscious experience the entire time even if the box was never opened, in order that the experience could happen at the corresponding time.

Which means we're all p-zombies until a specific point in the future when we decohere sufficiently to have consciousness up to that point.

Comment author: Nanashi 13 April 2015 06:33:39PM *  4 points [-]

A signed PGP message has three parts and thus only three places where additional information could be hidden. 1. The header 2. The message itself 3. The signature

The header is standardized. Any changes to the header itself (especially something as blatant as inserting a private key) would be enormously obvious, and would most likely result in a message that would fail to verify due to formatting issues.

The message itself can be verified by the author of the message. If anything shows up on this field that does not exactly match up with what he or she wrote, it will also be extremely obvious.

The signature itself, firstly, must be reproduced with 100% accuracy in order for the message to verify successfully. Any after-the-fact changes to either the message or the signature, will result in a message that does not verify successfully. (This is, of course, the entire purpose of a digital signature). Furthermore, the signature is generated algorithmically and cannot be manipulated by user input. The only way to change the signature would be to change the message prior to signing. However, as indicated above, this would be extremely obvious to the author.

Comment author: Pentashagon 14 April 2015 05:52:34AM 2 points [-]

https://tools.ietf.org/html/rfc4880#section-5.2.3.1 has a list of several subpackets that can be included in a signature. How many people check to make sure the order of preferred algorithms isn't tweaked to leak bits? Not to mention just repeating/fudging subpackets to blatantly leak binary data in subpackets that look "legitimate" to someone who hasn't read and understood the whole RFC.

Comment author: Pentashagon 27 January 2015 05:17:31PM *  2 points [-]

Suppose that instead of having well-defined actions AIXI only has access to observables and its reward function. It might seem hopeless, but consider the subset of environments containing an implementation of a UTM which is evaluating T_A, a Turing machine implementing action-less AIXI, in which the implementation of the UTM has side effects in the next turn of the environment. This embeds AIXI-actions as side effects of an actual implementation of AIXI running as T_A on a UTM in the set of environments with observables matching those that the abstract AIXI-like algorithm observes. To maximize the reward function the agent must recognize its implementation embedded in the UTM in M and predict the consequences of the side effects of various choices it could make, substituting its own output for the output of the UTM in the next turn of the simulated environment (to avoid recursively simulating itself), choosing the side effects to maximize rewards.

In this context counterfactuals are simulations of the next turns of M resulting from the possible side effects of its current UTM. To be useful there must be a relation from computation-suffixes of T_A to the potential side effects of the UTM. In other words the agent must be able to cause a specific side effect as a result of state-transitions or tape operations performed causally-after it has determined which side effect will maximize rewards. This could be as straightforward as the UTM using the most-recently written bits on the tape to choose a side effect.

In the heating game, the UTM running T_A must be physically implemented as something that has a side effect corresponding to temperature, which causally effects the box of rewards, and all these causes must be predictable from the observables accessible to T_A in the UTM. Similarly, if there is an anvil suspended above a physical implementation of a UTM running T_A, the agent can avoid an inability to increase rewards in the future of environments in which the anvil is caused to drop (or escape from the UTM before dropping it).

This reduces the naturalized induction problem to the tiling/consistent reflection problem; the agent must choose which agent it wants to be in the next turn(s) through side effects that can change its future implementation.

Comment author: JoshuaZ 17 January 2015 02:13:39PM 5 points [-]

In the terrorism case, the relevant biases are well-known and well-studied. The primary two biases in question are that humans take threats from intent or agencies much more seriously than threats from random chance. The second bias is that people pay more attention to threats which get a lot of coverage or which involve a large number of deaths at the same time. Tversky and Kahneman did studies on this (back when Tversky was still alive), and there's been followup by others since then.

Comment author: Pentashagon 20 January 2015 06:43:50AM 3 points [-]

Is there also a bias toward the illusion of choice? Some people think driving is safer than flying because they are "in control" when driving, but not when flying. Similarly, I could stay inside a well-grounded building my whole life and avoid ever being struck by lightning, but I can't make a similar choice to avoid all possible threats of terrorism.

View more: Next