Source article: http://news.berkeley.edu/2016/08/29/center-for-human-compatible-artificial-intelligence/

UC Berkeley artificial intelligence (AI) expert Stuart Russell will lead a new Center for Human-Compatible Artificial Intelligence, launched this week.

Russell, a UC Berkeley professor of electrical engineering and computer sciences and the Smith-Zadeh Professor in Engineering, is co-author of Artificial Intelligence: A Modern Approach, which is considered the standard text in the field of artificial intelligence, and has been an advocate for incorporating human values into the design of AI.

The primary focus of the new center is to ensure that AI systems are beneficial to humans, he said.

The co-principal investigators for the new center include computer scientists Pieter Abbeel and Anca Dragan and cognitive scientist Tom Griffiths, all from UC Berkeley; computer scientists Bart Selman and Joseph Halpern, from Cornell University; and AI experts Michael Wellman and Satinder Singh Baveja, from the University of Michigan. Russell said the center expects to add collaborators with related expertise in economics, philosophy and other social sciences.

The center is being launched with a grant of $5.5 million from the Open Philanthropy Project, with additional grants for the center’s research from the Leverhulme Trust and the Future of Life Institute.

Russell is quick to dismiss the imaginary threat from the sentient, evil robots of science fiction. The issue, he said, is that machines as we currently design them in fields like AI, robotics, control theory and operations research take the objectives that we humans give them very literally. Told to clean the bath, a domestic robot might, like the Cat in the Hat, use mother’s white dress, not understanding that the value of a clean dress is greater than the value of a clean bath.

The center will work on ways to guarantee that the most sophisticated AI systems of the future, which may be entrusted with control of critical infrastructure and may provide essential services to billions of people, will act in a manner that is aligned with human values.

“AI systems must remain under human control, with suitable constraints on behavior, despite capabilities that may eventually exceed our own,” Russell said. “This means we need cast-iron formal proofs, not just good intentions.”

One approach Russell and others are exploring is called inverse reinforcement learning, through which a robot can learn about human values by observing human behavior. By watching people dragging themselves out of bed in the morning and going through the grinding, hissing and steaming motions of making a caffè latte, for example, the robot learns something about the value of coffee to humans at that time of day.

“Rather than have robot designers specify the values, which would probably be a disaster,” said Russell, “instead the robots will observe and learn from people. Not just by watching, but also by reading. Almost everything ever written down is about people doing things, and other people having opinions about it. All of that is useful evidence.”

Russell and his colleagues don’t expect this to be an easy task.

“People are highly varied in their values and far from perfect in putting them into practice,” he acknowledged. “These aspects cause problems for a robot trying to learn what it is that we want and to navigate the often conflicting desires of different individuals.”

Russell, who recently wrote an optimistic article titled “Will They Make Us Better People?,” summed it up this way: “In the process of figuring out what values robots should optimize, we are making explicit the idealization of ourselves as humans. As we envision AI aligned with human values, that process might cause us to think more about how we ourselves really should behave, and we might learn that we have more in common with people of other cultures than we think.”

A few weeks ago, ShannonFriedman, an IFS counselor living in Berkeley, posted Female compatriots stay for a week in Berkeley an offer for rationalist women who are agenty to stay in her spare room for a few days. I replied, as I have a goal of getting to know the people associated with Less Wrong, the Singularity Institute and the Center for Applied Rationality (CFAR). 

She and I had a get to know you skype video interview, to talk about the trip. Shannon is a master connector, who genuinely enjoys introducing people who share common interests and complementary skills. As I had a goal of meeting interesting people, the planning was around helping me accomplish that.

I traveled with a friend, copt, who I met through the Less Wrong group I organize in Fort Collins Colorado. Shannon, fortunately, had two spare rooms for us. Her house, which she shares with several other people is in a residential neighborhood not too far from the secondary shopping district on San Pedro. It is on the main floor of a house, with a big back yard.

Berkeley is beautiful. The yards are full of flowers. There was an excellent coffee shop, Caffee Trieste (the sad coffee), within walking distance of the house, and a good breakfast restaurant also within walking distance. Downtown Berkeley is only a mile away, with good public transit connections to San Francisco, The mornings were foggy and cool (50F), the afternoons sunny and warm (75F).  

Over the 4 days of our visit, we met:

Julia Galef, mathy and elegant, formerly of New York, who is in Berkeley working with CFAR.  She's known for her organizing work within the skeptical community.  I had enjoyed watching her presentation to Skepticon this year, on The Straw Vulcan. She and Shannon are working on improving marketing for CFAR.

Luke Muehlhauser (lukeprog), tall, handsome, with dark hair, the Executive Director of the Singularity Institute, met briefly with us to discuss copt's work in finance.

Nisan, who organizes the Berkeley Less Wrong meetup, is a PhD student mathematician. He generously shared his Paleo groceries with us, and helped us navigate the BART on our trip into the city for a housewarming party. He explained operant conditioning as a technique for personal transformation while on the bus to Oakland. He and I are now writing companions and friends.

Peter de Blanc, a mathematician, programmer and musician, generously vacated his studio for me to stay in.

Aubrey de Grey, the founder of the SENS Foundation for rejuvenation technology,  is committed to the search for effective treatments for the disease of aging. We met to discuss fundraising and marketing. His focus is inspiring.

On Saturday night, we went to a housewarming party for Divia and Will Eden.  Their new house, in San Francisco, has a big back yard, and huge kitchen, is nicknamed Asgard.

At the housewarming party, everyone I spoke with had a clarity of intelligence and purpose which was invigorating to experience. There were approximately equal numbers of women and men at the party.  I spoke with a woman, who's name I don't remember, who is on her second successful startup as a developer, the first, a javascript based mockup tool, she accomplished while learning to program.  Charles, a physicist with SpaceX, red haired, and intense, who copt nick-named The Anti-Thor, because he works on preventing lightning strikes from damaging launch vehicles.  Valentine Michael Smith, intense, athletic, with a direct gaze, is working on combining martial arts and rationality to teach both of them faster and more effectively. He demonstrated the difference between compulsion and direction through two soft pushes on my sternum. I hope his project develops quickly, I'd learn new skills studying with him.

Eliezer Yudkowsky, is a smart, funny, big bear of a guy, who both copt and I liked immediately. I hope that we will have the chance to get to know each other better.

Outside of the Less Wrong connections, I met several other people, including a young musician with a great Magic deck, and at the coffee shop, I struck up a conversation with a couple, she is a photographer, and he is a musician, and she is 30 years older than him. Berkeley has high openness.

Also I was able to connect with a friend who I know through open source, who took me for lunch at the Thai Buddhist Temple, which has a pay as you like buffet lunch.

It was a full, but relaxed, 4 day trip. Shannon has a transformative talent of seeing others as powerful agents directing their lives, which encourages one to see themselves as a powerful interesting being. I'm so glad I took the chance to accept her offer. I'm glad to have her as a friend.