For this Tuesday's meetup I'm going to do a Q&A session about the AGI and alignment labs in the SF Bay area, and what the whole AI scene is like.
It's been wild since I moved from NYC, you meet people from all the major companies, learn what their internal politics are like, who the competent teams are and what they think about the dangers of what they're building. There's parties with celebrities, venture capitalists, CEOs and journalists, now convinced that AI powerful enough to dramatically change civilization is a dangerous prospect possibly coming in the next few decades.
I've now been working for a year on interpretability research trying to understand the internals... (read more)
This Tuesday January 17th I'll be running a meetup on the idea of how predictable and tradeable future payments after an outcome can create a specialized financial ecosystem to convert the uncertain future payment into a payment now for the expected value. That general description encompassess useful things we have today like insurance, crop futures, stocks and venture capital. But there's lots of other areas of life society could in theory apply the same idea to, which could be great if implemented.
We'll start by talking about why this kind of system is good:
On Tuesday December 20th I'll be running a meetup on the weirdest and perhaps coolest theory of how the universe and consciousness works that you've probably never heard of: UDASSA.
UDASSA is a simple-ish theory of what the universe is, which centers around the probability of a conscious observer finding themself experiencing a certain thing, which kinda elegantly combines the questions of "why this universe" and "why do we find ourselves experiencing it here and now". I don't want to over-hype it insofar as like all other theories of consciousness it has holes in it where you just hand-wave over confusion, but I like it because I think it does less of that... (read more)
On Tuesday November 15th I'll be running a meetup on the ways in which evolution has instilled instincts in us which outperform at a societal level the results from traditional game theory, and how fancier decision theories like Functional Decision Theory can help us understand those instincts and perhaps even outperform them.
This will be a combination presentation discussion where I'll introduce various ideas and we'll talk about each:
On Tuesday October 18th I'll be running a meetup on more deluxe takes of utilitarianism! The often-taught version of utilitarianism which says that you should act towards the greatest good for the greatest number, has a number of sticking points that often turns people away from it. I contend that there's a more deluxe moral-non-realist version of utilitarianism that gets at the benefits while elegantly handling many of the downsides!
This will be a combination presentation discussion where I'll introduce various ideas and we'll talk about each:
On Tuesday August 23rd I'll be running a meetup on the question of how to value the lives of potential future people. Basically inspired by this Cold Takes blog post: https://www.cold-takes.com/debating-myself-on-whether-extra-lives-lived-are-as-good-as-deaths-prevented/
You don't need to read the blog post first to come, although you can, I'll go over the gist of the arguments and then we can discuss the different arguments for and against whether making potential future people more likely to exist and live good lives is a compelling cause for altruism.
Join the Google Group for info about the location, it's at a group apartment in FiDi.
On Tuesday July 19th I'll be running a meetup on valuing life options! The trading world has lots of ideas about how to correctly value options that also apply to optionality in everyday life. I'll introduce various ideas about how to think about options and then we'll discuss various places in life where they might apply.
Here's some of the ideas I plan to cover:
On Tuesday May 24th I'll be running a meetup on pitches for fiction you think the rest of the group might be interested in! I plan to pitch a bunch of my favorite stories from the genre of fiction with unusually thoughtful characters and antagonists as well as coherent and novel worldbuilding. You bring your own ideas of stories to pitch as well as second other people's recommendations. Pitches are expected to be improvised and talk about what made you like a story, what's special about it, and what kind of person you think might particularly like or dislike it.
Come and hear my answer to what the most embarrassing fiction I wholeheartedly endorse is, which story is the best therapy-fic, or what you might want to read next if you liked Harry Potter and The Methods of Rationality or Worm.
=== WHEN+WHERE ===
7:00pm Tuesday, May 24th
New location in NoHo (Join the mailing list at https://groups.google.com/g/overcomingbiasnyc and say you came from LessWrong to access the post with the address)
This Tuesday I'll be leading a discussion on unusual ways to spend money to improve your overall life satisfaction. Spending money to make yourself happier is a classically hard problem, especially at higher incomes, because hedonic adaptation means you often adapt to new patterns and return to near your baseline happiness. But some things like social life and relationships are better at producing lasting happiness, and various unusual ways of spending money can be better at targeting those things.
Try to bring some ideas for unusual ways to spend money that you like, and we'll have people briefly pitch their ideas and then discuss them as a group. Here's some I'll pitch as... (read more)
This Tuesday marks the anniversary of the fall of the Berlin wall, possibly the most unexpected political event in modern history. We will celebrate this by learning to forecast the unexpected events of the future. The time has for a crash course in forecasting.
The plan: We'll have a list of questions assembled from various sources. (It can also be instructive to make predictions about your personal life - I can't come up with good questions about your personal life, but bringing a few of your own might be interesting). We'll go through them twice - the first time, by making off-the-cuff predictions without much thought.
The second time is more careful. There's a lot... (read more)
Anecdote from the NYC rationalist (OBNYC) group: Something I think we'd want that other groups might want too is an easy way for people organizing meetups to post to multiple channels like a website, mailing list, LessWrong and meetup.com.
Another issue we have that others may have too is that we tend to host meetups at people's apartments, and the people hosting don't necessarily want their addresses to be posted publicly. We currently handle this by only posting the address on a Google Group which is configured so you have to "apply" with a text box, and then we basically accept every application that seems like a reasonable human or mentions how they... (read more)
You mention having a second office in "the city proper": Would that be referring to Bellingham and Peekskill or Seattle and NYC? Alternatively would working from home some days of the week be viable for many employees?
I ask this because to me these would make the difference for the viability of living mainly in Seattle/NYC and spending 3 days a week at the campus, as opposed to the reverse case of living mainly near the campus and going into the city on weekends.
This isn't a huge difference from the perspective of doing things on weekends, but it makes a difference for having a significant other who lives in the city and going... (read more)
I went and checked and as far as I can tell they used the same 1024 batch size for the 12 and 6 hour time. The changes I noticed were better normalization, label smoothing, a somewhat tweaked input pipeline (not sure if optimization or refactoring) and updating Tensorflow a few versions (plausibly includes a bunch of hardware optimizations like you're talking about).
The things they took from fast.ai for the 2x speedup were training on progressively larger image sizes, and the better triangular learning rate schedule. Separately for their later submissions, which don't include a single-GPU figure, fast.ai came up with better methods of cropping and augmentation that improve accuracy. I don't necessarily
A relevant paper came out 3 days ago talking about how AlphaGo used Bayesian hyperparameter optimization and how that improved performance: https://arxiv.org/pdf/1812.06855v1.pdf
It's interesting to set the OpenAI compute article's graph to linear scale so you can see that the compute that went into AlphaGo utterly dwarfs everything else. It seems like DeepMind is definitely ahead of nearly everyone else on the engineering effort and money they've put into scaling.
I just checked and seems it was fp32. I agree this makes it less impressive, I forgot to check that originally. I still think this somewhat counts as a software win, because working fp16 training required a bunch of programmer effort to take advantage of the hardware, just like optimization to make better use of cache would.
However, there's also a different set of same-machine datapoints available in the benchmark, where training time on a single Cloud TPU v2 went down from 12 hours 30 minutes to 2 hours 44 minutes, which is a 4.5x speedup similar to the 5x achieved on the V100. The Cloud TPU was special-purpose hardware being trained with bfloat16 from the start, so that's a similar magnitude improvement more clearly due to software. The history shows incremental progress down to 6 hours and then a 2x speedup once the fast.ai team published and the Google Brain team incorporated their techniques.