LESSWRONG
LW

All of Crownless_Prince's Comments + Replies

AlphaGo versus Lee Sedol
Crownless_Prince-10

At which point the humans running this NN will notice that it likes to go around risk control measures and will... persuade it that it's a bad idea.

How? By instituting more complex control measures? Then you're back to the problem Kaj mentioned above.

Reply
0Lumifer
In the usual way. Contemporary trading systems are not black boxes full of elven magic. They are models, that is, a bunch of code and some data. If the model doesn't do what you want it to do, you stick your hands in there and twiddle the doohickeys until it stops outputting twaddle. Besides, in most trading systems the sophisticated part ("AI") is an oracle. Typically it outputs predictions (e.g. of prices of financial assets) and its utility function is some loss function on the difference between the prediction and the actual. It has no concept of trades, or dollars, or position limits. Translating these predictions into trades is usually quite straightforward.
Genetic "Nature" is cultural too
Crownless_Prince10

Yes, but this means that a lot of very rich people are very incorrect as to what is important for their wealth.

They know about the factors they can control. After all, those are the ones they actually focus on.

Reply
AlphaGo versus Lee Sedol
Crownless_Prince-10

So? An opaque neural net would quickly learn how to get around trade size restrictions if given the proper motivations.

Reply
-2Lumifer
At which point the humans running this NN will notice that it likes to go around risk control measures and will... persuade it that it's a bad idea. It's not like no one is looking at the trades it's doing.
Genetic "Nature" is cultural too
Crownless_Prince-30

Consider that if it had been the opposite - IQ was more a personal benefit than a country benefit - we'd be explaining it as "obviously smart people benefit themselves at the expense of others".

Yes, it's called basing your beliefs on the evidence.

Reply
5Stuart_Armstrong
No, it's choosing the first plausible rationalisation for the data. People have been suggesting lots of plausible explanations for the effect, and they all sound plausible (and may all the true, to some extent), but we really don't know until we test.