Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Comment author: lukeprog 15 April 2014 10:38:27PM 0 points [-]

Those who know me in person will know I regularly point out at least one of my own hypocrisies: specifically, that of eating meat. My hope is that it makes clear that I don't endorse my own behavior on the matter, and that I'm generally "on Team Vegetarian" even though I eat meat (largely for flesh-is-weak reasons). I'll even refer to meat as "animal suffering" in regular conversation, as in "I usually get the 'Tarzan' at Sandwich Spot, which is like the 'Erica Cato' but with animal suffering added" or "Spoonrocket's veggie option today look edible, so I'll pass on the one with animal suffering in it."

I do hope the all-vegan supermarket Veganz opens in Berkeley soon; it seems like a natural first stop for such a thing in the USA. And if they do, I hope they're available via Instacart. Then I would really have no excuse.

Comment author: lukeprog 13 April 2014 02:15:59AM 1 point [-]

From Lewis' Flash Boys:

Like every other trader on the Chicago exchanges, [Spivey] saw how much money could be made trading futures contracts in Chicago against the present prices of the individual stocks trading in New York and New Jersey. Every day there were thousands of moments when the prices were out of whack — when, for instance you could sell the futures contract for more than the price of the stocks that comprised it. To capture the profits, you had to be fast to both markets at once... The exchanges, by 2007, were simply stacks of computers in data centers. The speed with which trades occurred on them was no longer constrained by people. The only constraint was how fast an electronic signal could travel between Chicago and New York...

What Spivey had realized, by 2008, was that there was a big difference between the trading speed that was available between these exchanges and the trading speed that was theoretically possible... Incredibly to Spivey, the telecom carriers were not set up to understand the new demand for speed. Not only did Verizon fail to see that it could sell its special route to traders for a fortune; Verizon didn’t even seem aware it owned anything of special value. “You would have to order up several lines and hope that you got it,” says Spivey. “They didn’t know what they had.” As late as 2008, major telecom carriers were unaware that the financial markets had changed, radically, the value of a millisecond.

...The construction guy [driving the route] with him clearly suspected he might be out of his mind. Yet when Spivey pressed him, even he couldn’t come up with a reason why the plan wasn’t at least theoretically possible. That’s what Spivey had been after: a reason not to do it. “I was just trying to find the reason no [telecom] carrier had done it,” he says. “I was thinking: Surely I’ll see some roadblock.” Aside from the construction engineer’s opinion that no one in his right mind wanted to cut through the hard Allegheny rock, he couldn’t find one.

So Spivey began digging the line, keeping it secret for 2 years. He didn't start trying to sell the line to banks and traders until a couple months before the line was complete. And then:

The biggest question about the line — Why? — remained imperfectly explored. All its creators knew was that the Wall Street people who wanted it wanted it very badly — and also wanted to find ways for others not to have it. In one of his first meetings with a big Wall Street firm, Spivey had told the firm’s boss the price of his line: $10.6 million plus costs if he paid up front, $20 million or so if he paid in installments. The boss said he’d like to go away and think about it. He returned with a single question: “Can you double the price?”

Comment author: lukeprog 13 April 2014 02:46:38AM 0 points [-]

More (#1) from Flash Boys:

...why did the market in any given stock dry up only when he was trying to trade in it? To make his point, he asked the developers to stand behind him and watch while he traded. “I’d say, ‘Watch closely. I am about to buy one hundred thousand shares of AMD. I am willing to pay forty-eight dollars a share. There are currently one hundred thousand shares of AMD being offered at forty-eight dollars a share—ten thousand on BATS, thirty-five thousand on the New York Stock Exchange, thirty thousand on Nasdaq, and twenty-five thousand on Direct Edge.’ You could see it all on the screens. We’d all sit there and stare at the screen and I’d have my finger over the Enter button. I’d count out loud to five... Then I’d hit the Enter button and — boom! — all hell would break loose. The offerings would all disappear, and the stock would pop higher.” At which point he turned to the guys standing behind him and said, “You see, I’m the event. I am the news.”

And:

The deep problem with the system [high-frequency trading] was a kind of moral inertia. So long as it served the narrow self-interests of everyone inside it, no one on the inside would ever seek to change it, no matter how corrupt or sinister it became — though even to use words like “corrupt” and “sinister” made serious people uncomfortable, and so Brad avoided them. Maybe his biggest concern, when he spoke to investors, was that he’d be seen as just another nut with a conspiracy theory. One of the compliments that made him happiest was when a big investor said, “Thank God, finally there’s someone who knows something about high-frequency trading who isn’t an Area 51 guy.” Because he wasn’t a radical, it took him a while to figure out that fate and circumstance had created for him a dramatic role, which he was obliged to play. One night he actually turned to Ashley, now his wife, and said, “It feels like I’m an expert in something that badly needs to be changed. I think there’s only a few people in the world who can do anything about this. If I don’t do something right now — me, Brad Katsuyama — there’s no one to call.”

And:

Like a lot of regulations, Reg NMS was well-meaning and sensible. If everyone on Wall Street abided by the rule’s spirit, the rule would have established a new fairness in the U.S. stock market. The rule, however, contained a loophole: It failed to specify the speed of the SIP. To gather and organize the stock prices from all the exchanges took milliseconds. It took milliseconds more to disseminate those calculations. The technology used to perform these calculations was old and slow, and the exchanges apparently had little interest in improving it. There was no rule against high-frequency traders setting up computers inside the exchanges and building their own, much faster, better cared for version of the SIP. That’s exactly what they’d done, so well that there were times when the gap between the high-frequency traders’ view of the market and that of ordinary investors could be twenty-five milliseconds, or twice the time it now took to travel from New York to Chicago and back again.

Reg NMS was intended to create equality of opportunity in the U.S. stock market. Instead it institutionalized a more pernicious inequality. A small class of insiders with the resources to create speed were now allowed to preview the market and trade on what they had seen.

...By complying with Reg NMS, [Schwall] now understood, the smart order routers simply marched investors into various traps laid for them by high-frequency traders. “At that point I just got very, very pissed off,” he said. “That they are ripping off the retirement savings of the entire country through systematic fraud and people don’t even realize it. That just drives me up the fucking wall.”

His anger expressed itself in a search for greater detail. When he saw that Reg NMS had been created to correct for the market manipulations of the old NYSE specialists, he wanted to know: How had that corruption come about? He began another search. He discovered that the New York Stock Exchange specialists had been exploiting a loophole in some earlier regulation—which of course just led Schwall to ask: What event had led the SEC to create that regulation? Many hours later he’d clawed his way back to the 1987 stock market crash, which, as it turned out, gave rise to the first, albeit crude, form of high-frequency trading. During the 1987 crash, Wall Street brokers, to avoid having to buy stock, had stopped answering their phones, and small investors were unable to enter their orders into the market. In response, the government regulators had mandated the creation of an electronic Small Order Execution System so that the little guy’s order could be sent into the market with the press of a key on a computer keyboard, without a stockbroker first taking it from him on the phone. Because a computer was able to transmit trades must faster than humans, the system was soon gamed by smart traders, for purposes having nothing to do with the little guy. At which point Schwall naturally asked: From whence came the regulation that had made brokers feel comfortable not answering their phones in the midst of the 1987 stock market crash?

...Several days later he’d worked his way back to the late 1800s. The entire history of Wall Street was the story of scandals, it now seemed to him, linked together tail to trunk like circus elephants. Every systemic market injustice arose from some loophole in a regulation created to correct some prior injustice. “No matter what the regulators did, some other intermediary found a way to react, so there would be another form of front-running,” he said. When he was done in the Staten Island library he returned to work, as if there was nothing unusual at all about the product manager having turned himself into a private eye. He’d learned several important things, he told his colleagues. First, there was nothing new about the behavior they were at war with: The U.S. financial markets had always been either corrupt or about to be corrupted. Second, there was zero chance that the problem would be solved by financial regulators; or, rather, the regulators might solve the narrow problem of front-running in the stock market by high-frequency traders, but whatever they did to solve the problem would create yet another opportunity for financial intermediaries to make money at the expense of investors.

Schwall’s final point was more aspiration than insight. For the first time in Wall Street history, the technology existed that eliminated entirely the need for financial intermediaries. Buyers and sellers in the U.S. stock market were now able to connect with each other without any need of a third party. “The way that the technology had evolved gave me the conviction that we had a unique opportunity to solve the problem,” he said. “There was no longer any need for any human intervention.” If they were going to somehow eliminate the Wall Street middlemen who had flourished for centuries, they needed to enlarge the frame of the picture they were creating. “I was so concerned that we were talking about what we were doing as a solution to high-frequency trading,” he said. “It was bigger than that. The goal had to be to eliminate any unnecessary intermediation.”

Comment author: lukeprog 31 October 2013 10:27:12PM *  4 points [-]

Lately I've been listening to audiobooks (at 2x speed) in my down time, especially ones that seem likely to have passages relevant to the question of how well policy-makers will deal with AGI, basically continuing this project but only doing the "collection" stage, not the "analysis" stage.

I'll post quotes from the audiobooks I listen to as replies to this comment.

Comment author: lukeprog 13 April 2014 02:15:59AM 1 point [-]

From Lewis' Flash Boys:

Like every other trader on the Chicago exchanges, [Spivey] saw how much money could be made trading futures contracts in Chicago against the present prices of the individual stocks trading in New York and New Jersey. Every day there were thousands of moments when the prices were out of whack — when, for instance you could sell the futures contract for more than the price of the stocks that comprised it. To capture the profits, you had to be fast to both markets at once... The exchanges, by 2007, were simply stacks of computers in data centers. The speed with which trades occurred on them was no longer constrained by people. The only constraint was how fast an electronic signal could travel between Chicago and New York...

What Spivey had realized, by 2008, was that there was a big difference between the trading speed that was available between these exchanges and the trading speed that was theoretically possible... Incredibly to Spivey, the telecom carriers were not set up to understand the new demand for speed. Not only did Verizon fail to see that it could sell its special route to traders for a fortune; Verizon didn’t even seem aware it owned anything of special value. “You would have to order up several lines and hope that you got it,” says Spivey. “They didn’t know what they had.” As late as 2008, major telecom carriers were unaware that the financial markets had changed, radically, the value of a millisecond.

...The construction guy [driving the route] with him clearly suspected he might be out of his mind. Yet when Spivey pressed him, even he couldn’t come up with a reason why the plan wasn’t at least theoretically possible. That’s what Spivey had been after: a reason not to do it. “I was just trying to find the reason no [telecom] carrier had done it,” he says. “I was thinking: Surely I’ll see some roadblock.” Aside from the construction engineer’s opinion that no one in his right mind wanted to cut through the hard Allegheny rock, he couldn’t find one.

So Spivey began digging the line, keeping it secret for 2 years. He didn't start trying to sell the line to banks and traders until a couple months before the line was complete. And then:

The biggest question about the line — Why? — remained imperfectly explored. All its creators knew was that the Wall Street people who wanted it wanted it very badly — and also wanted to find ways for others not to have it. In one of his first meetings with a big Wall Street firm, Spivey had told the firm’s boss the price of his line: $10.6 million plus costs if he paid up front, $20 million or so if he paid in installments. The boss said he’d like to go away and think about it. He returned with a single question: “Can you double the price?”

Comment author: lukeprog 13 April 2014 12:46:49AM 0 points [-]

From Ayres' Super Crunchers, speaking of Epagogix, which uses neural nets to predict a movie's box office performance from its screenplay:

Some studios are utterly closed-minded to the idea that statistics could help them decide whether to greenlight a project. Copaken tells the extraordinary story of bringing two hedge fund managers to meet with a studio head. "These hedge fund guys had raised billions of dollars," Copaken explained, "and they were prepared to start with $500 million to fund films that would pass muster by our [neural net] test and be optimized for box office... [But] there was a lot of resistance to this new way of thinking... and finally one of these hedge fund guys sort of jumped into the discussion and said, 'Well, let me ask you a question. If Dick's system here gets it right fifty times out of fifty times, are you telling me that you wouldn't take that into account to change the way you decide which movies to make or how to make them?' And the guys said, 'No, that's absolutely right. We would not even if he were right fifty times out of fifty times... So what if we are leaving a billion dollars of the shareholder's money on the table; that is shareholders' money... Whereas if we change the way we do this, we might antagonize various people. We might not be invited. Our wives wouldn't be invited to the parties. People would get pissed at us. So why mess with a good thing?'"

Copaken was completely depressed when he walked out of the meeting, but when he looked over he noticed that the hedge fund guys were grinning from ear to ear. He asked them why they were so happy. They told him, "You don't understand, Dick. We make our fortunes by identifying small imperfections in the marketplace. They are usually tiny and they are usually fleeting and they are immediately filled by the efficiency of the marketplace. But if we can discover these things... we end up making lots of money before the efficiency of the marketplace closes out that opportunity. What you just showed us here in Hollywood is a ten-lane paved highway of opportunity. It's like they are committed to doing things the wrong way..."

Comment author: lukeprog 13 April 2014 01:01:49AM 0 points [-]

More (#1) from Super Crunchers:

...the Office of Education and the Office of Economic Opportunity sought to determine what types of education models could best break this cycle of failure. The result was Project Follow Through, an ambitious effort that studied 79,000 children in 180 low-income communities for twenty years at a price tag of more than $600 million... At the time it was the largest education study ever done. Project Follow Through looked at the impact of seventeen different teaching methods, ranging from models like DI [direct instruction], where lesson plans are carefully scripted, to unstructured models where students themselves direct their learning by selecting what and how they will study... Project Follow Through's designers wanted to know which model performed the best, not only in developing skills in its area of emphasis, but also across the board.

Direct Instruction won hands down. Education writer Richard Nadler summed it up this way: "When the testing was over, students in DI classrooms had placed first in reading, first in math, first in spelling, and first in language. No other model came close." And DI's dominance wasn't just in basic skill acquisition. DI students could also more easily answer questions that required higher-order thinking... DI even did better in promoting students' self-esteem than several child-centered approaches...

More recent studies by both the American Federation of Teachers and the American Institutes for Research reviewed data on two dozen "whole school" reforms and found once again that the Direct Instruction model had the strongest empirical support.

And:

The news media almost completely ignored the point that Summers was just talking about a difference in variability. It's not nearly as sexy as reporting "Harvard President Says Women Are Innately Deficient in Mathematics." (They might as easily have reported that Summers was claiming that women are innately superior in mathematics, since they are less likely to be really bad in math.) Many reporters simply didn't understand the point or couldn't figure out a way to communicate it to a general audience... At least in small part, Summers may have lost his job because people don't understand standard deviations.

And:

I remember when my partner, Jennifer, and I were expecting for the first time — back in 1994. Back then, women were told the probability of Down syndrome based on their age. After sixteen weeks, the mother could have a blood test for measuring her alphafetoprotein (AFP) level, and then they'd give you another probability. I remember asking the doctor if they had a way of combining the different probabilities. He told me flat out, "That's impossible. You just can't combine probabilities like that."

I bit my tongue, but I knew he was dead wrong. It is possible to combine different pieces of evidence, and has been since 1763 when a short essay by the Reverand Thomas Bayes was posthumously published...

Comment author: lukeprog 31 October 2013 10:27:12PM *  4 points [-]

Lately I've been listening to audiobooks (at 2x speed) in my down time, especially ones that seem likely to have passages relevant to the question of how well policy-makers will deal with AGI, basically continuing this project but only doing the "collection" stage, not the "analysis" stage.

I'll post quotes from the audiobooks I listen to as replies to this comment.

Comment author: lukeprog 13 April 2014 12:46:49AM 0 points [-]

From Ayres' Super Crunchers, speaking of Epagogix, which uses neural nets to predict a movie's box office performance from its screenplay:

Some studios are utterly closed-minded to the idea that statistics could help them decide whether to greenlight a project. Copaken tells the extraordinary story of bringing two hedge fund managers to meet with a studio head. "These hedge fund guys had raised billions of dollars," Copaken explained, "and they were prepared to start with $500 million to fund films that would pass muster by our [neural net] test and be optimized for box office... [But] there was a lot of resistance to this new way of thinking... and finally one of these hedge fund guys sort of jumped into the discussion and said, 'Well, let me ask you a question. If Dick's system here gets it right fifty times out of fifty times, are you telling me that you wouldn't take that into account to change the way you decide which movies to make or how to make them?' And the guys said, 'No, that's absolutely right. We would not even if he were right fifty times out of fifty times... So what if we are leaving a billion dollars of the shareholder's money on the table; that is shareholders' money... Whereas if we change the way we do this, we might antagonize various people. We might not be invited. Our wives wouldn't be invited to the parties. People would get pissed at us. So why mess with a good thing?'"

Copaken was completely depressed when he walked out of the meeting, but when he looked over he noticed that the hedge fund guys were grinning from ear to ear. He asked them why they were so happy. They told him, "You don't understand, Dick. We make our fortunes by identifying small imperfections in the marketplace. They are usually tiny and they are usually fleeting and they are immediately filled by the efficiency of the marketplace. But if we can discover these things... we end up making lots of money before the efficiency of the marketplace closes out that opportunity. What you just showed us here in Hollywood is a ten-lane paved highway of opportunity. It's like they are committed to doing things the wrong way..."

Comment author: ArisKatsaris 01 April 2014 07:15:04AM 1 point [-]

Short Online Texts Thread

Comment author: lukeprog 10 April 2014 06:07:05PM 0 points [-]
In response to AGI Quotes
Comment author: lukeprog 28 December 2011 02:39:42AM 0 points [-]

Technological progress is like an axe in the hands of a pathological criminal.

Albert Einstein

In response to comment by lukeprog on AGI Quotes
Comment author: lukeprog 09 April 2014 02:22:25AM 0 points [-]

Looking more closely, this much-duplicated "quote" seems to be a paraphrase of something he wrote in a letter to Heinrich Zaggler in the context of the first world war: "Our entire much-praised technological progress, and civilization generally, could be compared to an axe in the hand of a pathological criminal."

I do think about the AGI problem in much this way, though. E.g. in Just Babies, Paul Bloom wrote:

Families survive the Terrible Twos because toddlers aren’t strong enough to kill with their hands and aren’t capable of using lethal weapons. A two-year-old with the physical capacities of an adult would be terrifying.

I think our current civilization i like a two-year old. The reason we haven't destroyed ourselves yet, but rather just bit some fingers and ruined some carpets, is because we didn't have any civilization-lethal weapons. We've had nuclear weapons for a few decades now and not blown ourselves up yet but there were some close calls. In the latter half of the 21st century we'll acquire some additional means of destroying our civilization. Will we have grown up by then? I doubt it. Civilizational maturity progresses more slowly than technological power.

Comment author: lukeprog 09 April 2014 02:02:46AM 9 points [-]

There is nothing that can be said by mathematical symbols and relations which cannot also be said by words. The converse, however, is false. Much that can be and is said by words cannot be put into equations — because it is nonsense.

Clifford Truesdell

Comment author: CronoDAS 15 March 2014 09:36:44AM *  5 points [-]

Let's get a little more specific here.

Can anyone here name one currently living individual that MIRI would like to hire away from their current position to work on Friendly AI research, if money were no object? Terence Tao, perhaps? Do you think he would leave his current position as a university professor if you could offer him, say, a ten million dollar annual salary?

Comment author: lukeprog 07 April 2014 03:50:20AM *  3 points [-]

To do research, someone's got to have some actual interest in the problem space, or they'll end up fiddling around and doing stuff that's good for their interests or their long-term career but not necessarily for what their employer wants. So I don't know who has the capacity to acquire that interest. Tao would be good if he acquired an interest in the subject but I don't know if he could. Gowers at least commented on Baez's summary of the earlier Christiano result, but a short G+ comment isn't that much evidence. I don't currently know of any math superstars who want to work on FAI theory but only for a high salary — if I did, and I thought it would be a good hire, I'd reach out to MIRI's donors and try to solicit targeted donations for the hire.

Comment author: Eliezer_Yudkowsky 17 March 2014 08:17:03PM 11 points [-]

4, 5, and 2 in that order. You might think you could bypass 2 by advertising a high enough salary, but keep in mind that just advertising a high salary being available gives you problems 4 and 5 immediately, and if you don't advertise a superstar salary and don't have a reputation for paying it, then you may not be approached by any talent who's both money-desiring enough, and strong enough as a talent, to force you to confront the question of whether you need to actually take on disadvantages 4 and 5 for that particular person.

This reply is based on experience.

Comment author: lukeprog 06 April 2014 07:26:44PM *  2 points [-]

Based just on my experience at MIRI, I'll add another vote to "4, 5, and 2 in that order," especially if #5 includes funders and if #2 includes gwern's "shortage of reliably diagnosable talented people."

Item #4 is a pretty big deal in practice. 'Nuff said.

I've exhibited #5 throughout my tenure as CEO at MIRI, and perhaps still do. I've been repeatedly resistant to higher salaries and in retrospect I think the Board was right in two cases to be less timid than I was. Now the big worry is funders: the EA movement, in particular, may prefer martyr-ish salaries, though on that point I'm relieved to see that GiveWell's founders still make substantially more than I do.

On #2, consider MIRI's hiring of myself and Nate Soares. Neither of us are "superstars" — at least not yet; we'll try! — but we are clearly good for MIRI at the present stage, and yet I came in with no executive experience and no relevant technical background, and Nate came in with no research publications, having learned logic and model theory a few months before his hiring. There are probably other good hires out there available to MIRI but I just don't know what they look like. And of course in general, the world is not training FAI talent the way it trains, say, programming talent or finance talent. So in MIRI's case there is a pretty unusual "genuine absence of talented people."

View more: Next