I wrote something on Facebook recently that may interest people, so I'll cross-post it here.

Cem Sertoglu of Earlybird Venture Capital asked me: "will traders be able to look at their algorithms, and adjust them to prevent what happened yesterday from recurring?

My reply was:

I wouldn't be surprised if traders will be able to update their algorithms so that this particular problem doesn't re-occur, but traders have very little incentive to write their algorithms such that those algorithms would be significantly more robust in general. The approaches they use now are intrinsically not as transparent as (e.g.) logic-based approaches to software agent design, but they are more immediately profitable than logic-based approaches.

Wall Street has tried before to update its systems to be more robust, but their "band-aids" approach won't be sufficient. For example: in response to the flash crash of 2010, regulators installed a kind of "circuit breaker" that halts trading when there are extreme changes in a stock's price. Unfortunately, this did not prevent high-frequency trading programs from disrupting markets again on August 1st, 2012, in part because the circuit breaker wasn't also programmed to halt trading if there were extreme changes in the number of shares being traded (see: http://is.gd/vBqf53).

We can design multi-agent ecosystems using only logic-based agents that are (in some cases) subject to "formal verification" (mathematical proof of correct operation). See, for example, http://is.gd/XgRJYn. But these approaches haven't seen nearly as much development as the approaches currently in use on Wall Street, because they are not as immediately profitable.

Only regulators could have sufficient incentive to implement a more trustworthy ecosystem of high-frequency trading programs, but they succumbed to regulatory capture long ago, and therefore won't do anything so drastic.

I'm not too worried about the next 5 years, though. Mostly it will just be momentary scares, like the flash crash and the recent fake tweet disruption. I'm more worried about the far more powerful autonomous programs of the future, and those programs are the focus of our research at MIRI.