Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Reaching young math/compsci talent

6 Post author: lukeprog 02 June 2012 09:07PM

Series: How to Purchase AI Risk Reduction

Here is yet another way to purchase AI risk reduction...

Much of the work needed for Friendly AI and improved algorithmic decision theories requires researchers to invent new math. That's why the Singularity Institute's recruiting efforts have been aimed a talent in math and computer science. Specifically, we're looking for young talent in math and compsci, because young talent is (1) more open to considering radical ideas like AI risk, (2) not yet entrenched in careers and status games, and (3) better at inventing new math (due to cognitive decline with age).

So how can the Singularity Institute reach out to young math/compsci talent? Perhaps surprisingly, Harry Potter and the Methods of Rationality is one of the best tools we have for this. It is read by a surprisingly large proportion of people in math and CS departments. Here are some other projects we have in the works:

  • Run SPARC, a summer program on rationality for high school students with exceptional math ability. Cost: roughly $30,000. (There won't be classes on x-risk at SPARC, but it will attract young talent toward efficient altruism in general.)
  • Print copies of the first few chapters of HPMoR cheaply in Taiwan, ship them here, distribute them to leading math and compsci departments. Cost estimate in progress.
  • Send copies of Global Catastrophic Risks to lists of bright young students. Cost estimate in progress.

Here are some things we could be doing if we had sufficient funding:

  • Sponsor and be present at events where young math/compsci talent gathers, e.g. TopCoder High School and the International Math Olympiad. Cost estimate in progress.
  • Cultivate a network of x-risk reducers with high mathematical ability, build a database of conversations for them to have with strategically important young math/compsci talent, schedule those conversations and develop a pipeline so that interested prospects have a "next person" to talk to. Cost estimate in progress.
  • Write Open Problems in Friendly AI, send it to interested parties so that even those who don't think AI risk is important will at least see "Ooh, look at these sexy, interesting problems I could work on!"

 

 

Comments (75)

Comment author: komponisto 03 June 2012 09:08:16AM 10 points [-]

This is basically my approach of choice, and I am very happy to see SI pursuing it. That said, I would like to make a couple of comments:

Specifically, we're looking for young talent in math and compsci, because young talent is...(3) better at inventing new math (due to cognitive decline with age).

So, if Edward Witten (age 60)* called you up tomorrow and said he was interested in working on Friendly AI, you would tell him to get lost? I think not. At least, I hope not.

I'm not saying you should target older people in your recruitment activities. (As if that were even possible.) But I am strongly advising against getting into any kind of mindset where you would end up closing the door on any mathematically accomplished people who happen to see the light on this matter.

AGI really might be decades or more away. The people who are "young" now won't be that way forever. You may want their help in the future. In particular, you may want the help of a future John Baez, who after a satisfying run in more mainstream topics, decides at age 40 to turn their attention to "helping humanity" -- only in the form of FAI research rather than environmentalism.

(Also, if you believe in the youth-worship-mythos, Yudkowsky is really getting up there, at age 32. When does he get kicked off the team?)

Write Open Problems in Friendly AI, send it to interested parties so that even those who don't think AI risk is important will at least see "Ooh, look at these sexy, interesting problems I could work on!"

You may be underestimating the degree to which perceived "sexiness" is correlated to perceived "importance". Nevertheless, this is still a good idea.


* Witten on the age question (7:20):

Q: Why do physicists have their best ideas in their 20s?

A: Well, I would like to say that it's not entirely true -- that...I don't know if I'll manage to have my best ideas in my 50s, but I definitely did better in my 30s and 40s than in my 20s!

Comment author: lukeprog 07 June 2012 12:03:07AM 4 points [-]

So, if Edward Witten (age 60)* called you up tomorrow and said he was interested in working on Friendly AI, you would tell him to get lost? I think not. At least, I hope not.

No, obviously not. We're targeting young people, but that doesn't mean we're closed to older people.

Comment author: Fhyve 17 June 2012 04:44:06AM *  5 points [-]

I am a young person (20) who is good at math and hasn't been entrenched in the system yet. I am also already on board with AI risk reduction. I would really like to work as a researcher.

However, I don't have much to show for myself, and I don't think I can substantiate my claims right now. I do not know enough about research to know if I am going to be good at it. At the moment, I have a pretty good topical view of math, but not a very good technical view - I am only into second year university math. Pure math and theoretical comp sci especially appeal to me.

How do I find out if I can be a researcher? How do I show you that I can be a good researcher if I find that I can in fact become a good researcher? What sort of math should I be studying - any textbooks to recommend?

Comment author: incariol 17 June 2012 09:52:11AM 2 points [-]

You can find a few suggestions here, for starters.

Comment author: DaFranker 20 July 2012 07:47:07PM 0 points [-]

I was reading this and preparing to post a questions-comment just like his, so thanks!

Comment author: lukeprog 17 June 2012 06:46:13AM 2 points [-]

Thanks for your interest! Please contact louie.helm [at] singinst.org.

Comment author: Risto_Saarelma 03 June 2012 09:39:22AM 2 points [-]

I'd like to see more counterarguments to the thing about mathematicians being much less useful for ground-breaking work after their 20s that don't rely on extreme outliers like Witten, Andrew Wiles or Paul Erdös.

Comment author: komponisto 03 June 2012 09:48:00AM 6 points [-]

That would be difficult, since "groundbreaking work" automatically implies "extreme outlier".

In fact, I would expect that typical mathematicians are much more useful above 30 than below -- to a greater extent than is the case for the extreme outliers.

Comment author: Kaj_Sotala 03 June 2012 12:55:11PM 16 points [-]

Simonton (1988) Age and Outstanding Achievement: What Do We Know After a Century of Research? Psychological Bulletin, Vol. 104, No. 2, 251-267.

Short version: the productivity for mathematicians seems to peak around late 20s or early 30s, with the productivity after the peak falling to less than one-quarter the maximum. However, the average quality of a contribution does not seem to vary with age, and exceptional researchers (in any field) tend to remain unusually profilic, as compared to an average researcher of the same age, even after passing their peaks.

Long version:

In the first place, the location of the peak, as well as the magnitude of the postpeak decline, tends to vary depending on the domain of creative achievement. At one extreme, some fields are characterized by relatively early peaks, usually around the early 30s or even late 20s in chronological units, with somewhat steep descents thereafter, so that the output rate becomes less than one-quarter the maximum. This agewise pattern apparently holds for such endeavors as lyric poetry, pure mathematics, and theoretical physics, for example (Adams, 1946; Dennis, 1966; Lehman, 1953a; Moulin, 1955; Roe, 1972b; Simonton, 1975a; Van Heeringen & Dijkwel, 1987). At the contrary extreme, the typical trends in other endeavors may display a leisurely rise to a comparatively late peak, in the late 40s or even 50s chronologically, with a minimal if not largely absent drop-off afterward. This more elongated curve holds for such domains as novel writing, history, philosophy, medicine, and general scholarship, for instance (Adams, 1946; Richard A. Davis, 1987; Dennis, 1966; Lehman, 1953a; Simonton, 1975a). Of course, many disciplines exhibit age curves somewhat between these two outer limits, with a maximum output rate around chronological age 40 and a notable yet moderate decline thereafter (see, e.g., Fulton & Trow, 1974; Hermann, 1988; Mc- Dowell, 1982; Zhao & Jiang, 1986). Output in the last years appears at about half the rate observed in the peak years. Productive contributions in psychology, as an example, tend to adopt this temporal pattern (Homer et al., 1986; Lehman, 1953b; Over, 1982a, 1982b; Zusne, 1976).

It must be stressed that these interdisciplinary contrasts do not appear to be arbitrary but instead have been shown to be invariant across different cultures and distinct historical periods (Lehman, 1962). As a case in point, the gap between the expected peaks for poets and prose authors has been found in every major literary tradition throughout the world and for both living and dead languages (Simonton, 1975a). Indeed, because an earlier productive optimum means that a writer can die younger without loss to his or her ultimate reputation, poets exhibit a life expectancy, across the globe and through history, about a half dozen years less than prose writers do (Simonton, 1975a). This cross-cultural and transhistorical invariance strongly suggests that the age curves reflect underlying psychological universals rather than arbitrary sociocultural determinants. In other words, the age functions for productivity may result from intrinsic information-processing requirements rather than extrinsic pressures due to age stereotypes about older contributors, a point that we shall return to in the theoretical section (see also Bayer & Dutton, 1977).

[...]

Generally, the top 10% of the most prolific elite can be credited with around 50% of all contributions, whereas the bottom 50% of the least productive workers can claim only 15% of the total work, and the most productive contributor is usually about 100 times more prolific than the least (Dennis, 1954b, 1955; also see Lotka, 1926; Price, 1963, chap. 2). Now from a purely logical perspective, there are three distinct ways of achieving an impressive lifetime output that enables a creator to dominate an artistic or scientific enterprise. First, the individual may exhibit exceptional precocity, beginning contributions at an uncommonly early age. Second, the individual may attain a notable lifetime total by producing until quite late in life, and thereby display productive longevity. Third, the individual may boast phenomenal output rates throughout a career, without regard to the career's onset and termination. These three components are mathematically distinct and so may have almost any arbitrary correlation whatsoever with each other, whether positive, negative, or zero, without altering their respective contributions to total productivity. In precise terms, it is clear that O = R(L - P), where O is lifetime output, R is the mean rate of output throughout the career, L is the age at which the career ended (longevity), and P is the age at which the career began (precocity). The correlations among these three variables may adopt a wide range of arbitrary values without violating this identity. For example, the difference L - P, which defines the length of a career, may be more or less constant, mandating that lifetime output results largely from the average output rate R, given that those who begin earlier, end earlier, and those who begin later, end later. Or output rates may be more or less constant, forcing the final score to be a function solely of precocity and longevity, either singly or in conjunction. In short, R, L, and P, or output rate, longevity, and precocity, comprise largely orthogonal components of O, the gauge of total contributions.

When we turn to actual empirical data, we can observe two points. First, as might be expected, precocity, longevity, and output rate are each strongly associated with final lifetime output, that is, those who generate the most contributions at the end of a career also tend to have begun their careers at earlier ages, ended their careers at later ages, and produced at extraordinary rates throughout their careers (e.g., Albert, 1975; Blackburn et al., 1978; Bloom, 1963; Clemente, 1973; S. Cole, 1979; Richard A. Davis, 1987; Dennis, 1954a, 1954b; Helson & Crutchfield, 1970; Lehman, 1953a; Over, 1982a, 1982b; Raskin, 1936; Roe, 1965, 1972a, 1972b; Segal, Busse, & Mansfield, 1980; R. J. Simon, 1974; Simonton, 1977c; Zhao & Jiang, 1986). Second, these three components are conspicuously linked with each other: Those who are precocious also tend to display longevity, and both precocity and longevity are positively associated with high output rates per age unit (Blackburn et al., 1978; Dennis, 1954a, 1954b, 1956b; Horner et al., 1986; Lehman, 1953a, 1958; Lyons, 1968; Roe, 1952; Simonton, 1977c; Zuckerman, 1977). [...]

While specifying the associations among the three components of lifetime output, we have seemingly neglected the expected peak productive age. Those creators who make the most contributions tend to start early, end late, and produce at above average rates, but are the anticipated career peaks unchanged, earlier, or later in comparison to what is seen for their less prolific colleagues? [...]

Comment author: Kaj_Sotala 03 June 2012 01:06:30PM 12 points [-]

...and after posting that comment, I remembered that I had made an earlier post citing studies that said that it's the middle-aged and not young scientists who are the most productive, which is in conflict with the results I just quoted. I feel silly now. I guess I should re-read the studies that I referenced three years ago to figure out what version is correct.

Comment author: gwern 03 June 2012 06:34:29PM 6 points [-]

I guess I should re-read the studies that I referenced three years ago to figure out what version is correct.

Just to make the obvious point, your earlier post seems to draw on citations using mostly post-60s and later data, while that 1988 paper uses many citations from the 60s or earlier.

Comment author: John_Maxwell_IV 03 June 2012 06:07:33PM 6 points [-]

If old and young mathematicians have different strengths and weaknesses maybe it's best to have a few of both.

Comment author: Kaj_Sotala 03 June 2012 12:55:27PM 4 points [-]

(part 2)

if one calculates the age curves separately for major and minor works within careers, the resulting functions are basically identical. Both follow the same second-order polynomial (as seen in Equation 1), with roughly equal parameters. Second, if the overall age trend is removed from the within-career tabulations of both quantity and quality, minor and major contributions still fluctuate together. Those periods in a creator's life that see the most masterpieces also witness the greatest number of easily forgotten productions, on the average. Another way of saying the same thing is to note that the "quality ratio," or the proportion of major products to total output per age unit, tends to fluctuate randomly over the course of any career. The quality ratio neither increases nor decreases with age nor does it assume some curvilinear form. These outcomes are valid for both artistic (e.g., Simonton, 1977a) and scientific (e.g., Simonton, 1985b) modes of creative contribution (see also Alpaugh, Renner,& Birren, 1976, p. 28). What these two results signify is that if we select the contribution rather than the age period as the unit of analysis, then age becomes irrelevant to determining the success of a particular contribution. For instance, the number of citations received by a single scientific article is not contingent upon the age of the researcher (Oromaner, 1977).

The longitudinal linkage between quantity and quality can be subsumed under the more general "constant-probability-ofsuccess model" of creative output (Simonton, 1977a, 1984b, 1985b, 1988b, chap. 4). According to this hypothesis, creativity is a probabilistic consequence of productivity, a relationship that holds both within and across careers. Within single careers, the count of major works per age period will be a positive function of total works generated each period, yielding a quality ratio that exhibits no systematic developmental trends. And across careers, those individual creators who are the most productive will also tend, on the average, to be the most creative: Individual variation in quantity is positively associated with variation in quality. There is abundant evidence for the application of the constant-probability-of-success model to cross-sectional contrasts in quantity and quality of output (Richard A. Davis, 1987; Simonton, 1984b, chap. 6; 1985b, 1988b, chap. 4). In the sciences, for example, the reputation of a nineteenthcentury scientist in the twentieth century, as judged by entries in standard reference works, is positively correlated with the total number of publications that can be claimed (Dennis, 1954a; Simonton, 1981 a; see also Dennis, 1954c). Similarly, the number of citations a scientist receives, which is a key indicator of achievement, is a positive function of total publications (Crandall, 1978; Richard A. Davis, 1987; Myers, 1970; Rushton, 1984), and total productivity even correlates positively with the citations earned by a scientist's three best publications (J. R. Cole & S. Cole, 1973, chap. 4). [...]

The constant-probability-of-success model has an important implication for helping us understand the relation between total lifetime output and the location of the peak age for creative achievement within a single career (Simonton, 1987a, 1988b, chap. 4). Because total lifetime output is positively related to total creative contributions and hence to ultimate eminence, and given that a creator's most distinguished work will appear in those career periods when productivity is highest, the peak age for creative impact should not vary as a function of either the success of the particular contribution or the final fame of the creator. Considerable empirical evidence indeed demonstrates the stability of the career peak (Simonton, 1987a). In the sciences, for instance, the correlation between the eminence of psychologists and the age at which they contribute their most influential work is almost exactly zero (Zusne, 1976; see also Lehman, 1966b; of. Homer et al., 1986). And in the arts, such as literary and musical creativity, the age at which a masterpiece is generated is largely independent of the magnitude of the achievement (Simonton, 1975a, 1977a, 1977c). Thus, even though an impressive lifetime output of works, and subsequent distinction, is tied to precocity, longevity, and production rate, the expected age optimum for quantity and quality of contribution is dependent solely on the particular form of creative expression (also see Raskin, 1936).

Comment author: Vaniver 03 June 2012 05:17:05PM 1 point [-]

Galenson's book on artists fascinated me: he identified two clusters, experimental artists who liked to sketch and rework things and whose quality increased with age, and conceptual artists, who liked doing preparatory work and outsourcing the actual production, who made massive contributions when young but whose productivity rapidly tapered off.

With art, there's room for both types, but I imagine that math and related fields are heavily biased towards the conceptual style, especially the theoretical components of those fields.

Comment author: jsteinhardt 03 June 2012 07:30:51PM 4 points [-]

Actually, one of the first things that new researchers have to learn is that just thinking about a problem and coming up with ideas will get you nowhere -- you have to actually get your hands dirty and try things out to make progress.

Comment author: Vaniver 03 June 2012 08:49:18PM 0 points [-]

Oh, definitely. I don't mean to imply that, say, Warhol never got his hands dirty- but that Rembrandt's skill was in the realm of dirty hands and that Warhol's skill was in the realm of insight.

(I know in my research the act of sitting down and writing out an idea or sitting down and coding an algorithm or sitting down and going through the math has been indispensable, and strongly recommend it to anyone else.)

Comment author: Risto_Saarelma 03 June 2012 12:21:11PM 1 point [-]

Depends on the surface area of unbroken ground. I understand there are quite a few marginal areas in mathematics where you can come up with novel approaches that will be quite impressive to the other five people working on that specific sub-sub-sub-area, but not necessarily that much to mathematics at large. Also, contemporary mathematicians whose names are actually recognizable by popular science literate non-mathematicians are a very small group even compared to the sort of top researchers who are working with the sort stuff the apocryphal wisdom about needing to be in your 20s seems to apply to.

Though I'd also like to see more arguments about how above 30 mathematicians can do all sorts of useful stuff when you don't get fixated on paradigm-upending world-class results, and what sort of stuff this is.

Comment author: Zetetic 02 June 2012 11:49:23PM 9 points [-]

Anything for undergrads? It might be feasible to do a camp at the undergraduate level. Long term, doing an REU style program might be worth considering. NSF grants are available to non-profits and it may be worth at least looking into how SIAI might get a program funded. This would likely require some research, someone who is knowledgeable about grant writing and possibly some academic contacts. Other than that I'm not sure.

In addition, it might be beneficial to identify skill sets that are likely to be useful for SI research for the benefit of those who might be interested. What skills/specialized knowledge could SI use more of?

Comment author: lukeprog 07 June 2012 12:04:17AM 3 points [-]

SPARC for undergrads is in planning, if we can raise the funding.

What skills/specialized knowledge could SI use more of?

See here.

Comment author: Zetetic 07 June 2012 02:34:48AM 1 point [-]

SPARC for undergrads is in planning, if we can raise the funding.

Awesome, glad to hear it!

See here.

Alright, I think I'll sign up for that.

Comment author: JoshuaZ 02 June 2012 09:58:59PM 21 points [-]

Send copies of Global Catastrophic Risks to lists of bright young students

This may come across as spamming and will likely send crank signals.

Comment author: Nisan 03 June 2012 07:50:22PM *  10 points [-]

I dunno. It's a book. If anyone sends me a book, I'll consider keeping it and likely look at the first couple of pages, even if it's Dianetics or The Book of Mormon. I don't regard books as the physical and memetic pollution that spam is.

If I got a vanity-press book like this, I'd regard it as cranky non-spam. But Global Catastrophic Risks is published by Oxford University Press, which matches the pattern "legitimate" rather than "crank".

If I got a religious text, I'd be unimpressed. But Global Catastrophic Risks differs from a religious text in that it's a collection of essays by different authors who no doubt disagree about many things, rather than a canonized text that's regarded as perfect. And the hidden agenda of someone who gives me Global Catastrophic Risks is to get me thinking about global catastrophic risks — which is pretty reasonable, although not universally compelling. It would be much less creepy than receiving a Bible.

In summary, while JoshuaZ might have been turned off by receiving a book when he was a mathematically talented youth, I wouldn't have. So, that's two data points.

tl;dr: Please send me a copy of the Book of Mormon.

Comment author: Alicorn 03 June 2012 09:17:42PM 1 point [-]

Do you actually want a copy of the Book of Mormon? It's online, and I bet you could get a free one by filling out a form on an LDS website.

Comment author: Nisan 03 June 2012 11:36:27PM 1 point [-]

You're right, I can request a free Book of Mormon delivered to my doorstep by missionaries. I notice some mediocre reasons not to do this, as well as some strong unreasonable aversions to doing this; which means maybe I should do it...

Comment author: Alicorn 04 June 2012 12:20:24AM *  4 points [-]

The (plenty of) Mormons I have met are all really nice, friendly people. Missionaries on duty might be different (more goal-oriented, probably) but I bet you they're still nice. Unless you're just concerned about wasting their time, what are your strong unreasonable aversions?

Comment author: Nisan 04 June 2012 03:55:06AM 1 point [-]

It feels dishonest because if their goal is to convert me, calling on me is a waste of time. I know I'm not going to convert. I can satisfy this aversion by being honest up-front about not being likely to convert. I bet if I tell them I nevertheless want to keep the book and look at it a little, they'll still come over and give it to me. That way I'm being honest; and from a more consequentialist point of view I'm not wasting anyone's time but my own because converting people to Mormonism isn't a great goal anyways.

Another aversion is telling me that I'm going to disagree strongly with what they have to say, and that suppressing disagreement will be awkward and that expressing disagreement will be rude. I can respond to this aversion by deciding beforehand how to approach the conversation: I could either have the goal of learning what Mormonism means to them, or I could practice expressing disagreement.

An actual reason for not casting Summon Mormon Missionaries is that the LDS Church will have my contact info forever, and will pester me in the future. If I can remove that cost, I'll do it.

Comment author: MileyCyrus 04 June 2012 04:38:28AM 5 points [-]

An actual reason for not casting Summon Mormon Missionaries is that the LDS Church will have my contact info forever, and will pester me in the future.

Speaking from experience, they won't. I called them once for a free Book of Mormon. They came over and I said thanks for the book and but I don't want to convert. They made a follow-up call, but I haven't heard from them since.

Comment author: Alicorn 04 June 2012 04:28:11AM 1 point [-]

casting Summon Mormon Missionaries

Hee hee hee.

The contact info thing probably is an actual problem and a legit reason to hold back.

Comment author: arundelo 04 June 2012 04:28:09AM 1 point [-]

If you really want a paper Book of Mormon but want to avoid interacting with people who will try to convert you I recommend checking a store that sells stuff that was donated to it (e.g., Goodwill or Salvation Army in the US). These places often sell books for a dollar or less.

(Because Mormons like to give copies of the Book of Mormon away, there are a lot of them that get taken to thrift shops when non-Mormons get rid of old stuff; because of supply and demand, I think you're less likely to find a copy in a regular used bookstore.)

(Insults to people's holy books rot13ed out of a possibly excessive sense of politeness: Or jnearq; abg sbe abguvat qbrf vg vapyhqr n Obbx bs *Rgure*, nf sebz jung yvggyr V'ir ernq bs vg, vg vf *rira zber obevat* guna gur Wrjvfu naq Puevfgvna fpevcgherf.)

Comment author: wedrifid 04 June 2012 04:33:34AM 0 points [-]

It feels dishonest because if their goal is to convert me, calling on me is a waste of time.

Their goal is to earn their heavenly reward (or possibly 'new earth' reward, not sure on the details). The heavenly reward scheme is not based on commission but on the work that they do so you are not doing them a disservice.

Comment author: Nisan 04 June 2012 05:20:16AM *  0 points [-]

The test for dishonesty I'd use here is: Would a missionary (or their superiors in the Church) be dismayed if they learned that a potential new latter-day saint had been leading them on? I suppose the answer is yes, no matter the theology.

Comment author: ciphergoth 04 June 2012 06:43:10AM 1 point [-]

I predict that if, when you ask for the book, you say "there is zero chance of me converting, I just want it for reference", they will send it to you, and follow up, anyway.

Comment author: Jayson_Virissimo 05 June 2012 03:58:56AM 0 points [-]

I tried getting a free copy of the Koran from here, but it never arrived. IDK why.

Comment author: JoshuaZ 02 June 2012 09:30:45PM 2 points [-]

Run SPARC, a summer program on rationality for high school students with exceptional math ability. Cost: roughly $30,000.

Do we have any reason to believe that such a program will be more effective than existing summer programs like Ross and PROMYS?

Comment author: lukeprog 02 June 2012 09:45:36PM 3 points [-]

Compare their syllabi. Ross and PROMYS don't teach what SPARC is teaching, and they don't put these young students into contact with us.

As for effectiveness... what measure of effectiveness did you have in mind?

Comment author: VincentYu 02 June 2012 11:11:40PM *  17 points [-]

SPARC and a number of mostly homogeneous math camps are all looking for pre-college students with strong mathematical ability. Since SPARC's syllabus is notably different from that of math camps, it seems like a bad idea to compete with these camps for the top students. But competition is inevitable if SPARC runs at the same time as these camps; below I have found and listed the 2012 start and end dates for the most prominent math camps:

  • Ross: June 18 – July 27
  • PROMYS: July 1 – August 11
  • HSMC: June 17 – July 28
  • Mathcamp: July 1 – August 5
  • HCSSiM: July 1 – August 11
  • SUMaC: July 15 – August 11
  • RSI*: June 24 – August 4
  • (SPARC: August 6 – August 13)

SPARC's starting date this year conflicts with the end dates of three of these seven camps. Perhaps there are other scheduling constraints, but if not, wouldn't it be a good idea to run SPARC a week later to avoid conflicts? (It is too late to change this year, of course.)

*I know RSI is not a math camp in the spirit of the others, but it's well-known and attracts some students away from math camps.

ETA: And since SPARC is free and relevant to math students, if it can guarantee that it will not conflict with the other program dates, I think many math programs will be happy to link to or otherwise mention SPARC to current and past students – this should help spread the word to more potential students.

Comment author: JoshuaZ 02 June 2012 09:56:14PM *  5 points [-]

I'm not sure what would be a good metric. But it isn't obvious to me that having a separate program like this is at all likely to be better than having kids go through other programs that teach a lot of math systematically and then snagging some of the kids up when they get a little older. This is especially the case because the existing programs have very good teaching, lot of long-term institutional knowledge, and much better funding. To really effectively run a summer program that attracts top talent you'll need a lot more money.

Incidentally, the website could use some work. Obvious things that kids and parents are thinking about when they look at a summer program, costs, dorming, how to apply, dates of the program, should be direct rather than vaguely answered on an FAQ.

Comment author: lukeprog 02 June 2012 10:06:33PM 3 points [-]

We are reworking the website, we just needed to get something up quickly. Also, we already maxed out our capacity for top young talent by mailing written invitations directly to a bunch of the people we wanted to apply.

Comment author: John_Maxwell_IV 02 June 2012 11:46:53PM 7 points [-]

You guys should have a simple mailing list to sign up for to get reminded about future camps, and maybe even to broadcast camp related materials (e.g. "here are video lectures from the camp you missed").

Comment author: lukeprog 03 June 2012 01:23:06AM 1 point [-]

Yes that will be part of the new CFAR website we're working on.

Comment author: paulfchristiano 03 June 2012 07:21:28AM 2 points [-]

Cost, logistics, and how to apply were all discussed on the front page, until the application process closed and they were replaced with "the application process is closed."

Comment author: JoshuaZ 03 June 2012 03:04:21PM 4 points [-]

Yeah, those would be good things to keep up in general. They signal careful planning and good design. And it helps for families who are planning out their summers for the next year or something similar. We don't lose anything by having that data.

Comment author: beoShaffer 04 June 2012 02:56:25AM 2 points [-]

Just keep in mind that having application information available can imply that applications are still open. So make it clear that the info is just for reference.

Comment author: Incorrect 02 June 2012 09:21:31PM 1 point [-]

Do you have any direct advice to young programmers?

Comment author: lukeprog 02 June 2012 09:46:11PM 2 points [-]

Advice toward what goal(s)? Reducing AI risk?

Comment author: Incorrect 02 June 2012 09:53:52PM 3 points [-]

Becoming involved with the SI and knowing if they are qualified to be involved with the SI and if not, becoming qualified to be involved with the SI.

Comment author: lukeprog 02 June 2012 10:04:05PM 3 points [-]

Well, we need lots of help besides elite young math/compsci talent. You could contact louie.helm [at] singinst.org and explain your experience and qualifications. Thanks for your interest!

Comment author: John_Maxwell_IV 03 June 2012 03:54:18AM 0 points [-]

Is it really optimal to dismiss Incorrect as not being elite math/computer science talent so quickly?

Also, are you familiar with growth versus static models of intelligence? This looks to me like you are promoting a static model, which amounts to destroying a public good in my view.

University professors don't tell students they are too stupid to contribute to the problems they are trying to solve. I don't see why SI should either.

Comment author: PECOS-9 03 June 2012 06:13:18AM 5 points [-]

I didn't interpret lukeprog's comment as dismissing Incorrect as not being elite talent. I thought he was just noting that, whether he is "elite" or not, he can contact Louie to find out how he can help.

Comment author: lukeprog 03 June 2012 10:36:06AM 6 points [-]

Correct.

Comment author: JoshuaZ 03 June 2012 04:07:51AM 2 points [-]

While I agree with most of this (and have upvoted) two points stand out:

Also, are you familiar with growth versus static models of intelligence

I don't think bringing this up helps your point very much. While there are individuals whose apparent extreme talent blooms fairly late (e.g. Steven Chu who didn't really start being that impressive until he was in college), the lack of change of IQ scores over time on average is very robust, dating back to Spearman's original research about a hundred years ago. This is also true for other metrics of intelligence. By and large, intelligence is pretty static.

University professors don't tell students they are too stupid to contribute to the problems they are trying to solve

This is true, but professors do sometimes tell students when a problem may just be out of their league. To use an extreme example, consider a grad student who walks into his adviser's office and says he wants to prove the Riemann Hypothesis. That said, your essential point is valid, because even in that case, a professor could still direct them to some easier related problem or helpful question related to some aspect of it. So your basic point is valid.

Comment author: Kaj_Sotala 03 June 2012 08:23:15AM *  9 points [-]

Intelligence seems relatively static, but AFAIK once you've reached a certain minimum threshold in intelligence, conscientiousness becomes a more important factor for actual accomplishment. (Anecdotally and intuitively, conscientiousness seems more amenable to change, but I don't know if the psychological evidence supports that.)

Comment author: Barry_Cotter 03 June 2012 02:07:34PM 4 points [-]

Wait, there's real evidence of durable changes in conscientiousness? Point me its way. The psychology literature does not appear (after a brief search) to support the idea of lasting change. I would be happy to be wrong.

Comment author: gwern 03 June 2012 05:13:41PM 5 points [-]
Comment author: Kaj_Sotala 03 June 2012 02:23:47PM 2 points [-]

Sorry, I should have been more clear: I only have anecdotal evidence, and a rather small sample at that. I'll edit my comment.

Comment author: John_Maxwell_IV 03 June 2012 04:28:07AM *  1 point [-]

Mind sharing your source for relatively static IQ? I feel like I've read otherwise, especially for children.

Comment author: [deleted] 03 June 2012 07:29:04AM *  3 points [-]

Childhood IQs don't correlate that tightly with adult IQs. But once people are in their late teens change already becomes very unlikely.

Comment author: JoshuaZ 03 June 2012 03:05:26PM 1 point [-]

Yes, in the lower end there's some flexbility, especially in the mid teens but after that change is relatively static.

Comment author: faul_sname 03 June 2012 06:36:35AM 0 points [-]

I'm not sure how strongly IQ correlates with real-world abilities (well, actually, I am sure: 0.2-0.6 depending on the task 1). You don't need exceptional IQ to do new math (see Richard Feynman) but you do need an interest in math and quite a bit of exposure. Synesthesia can also be helpful.

I'm not finding a non-paywalled version right now, and unfortunately am not at my university at the moment to access it.

Comment author: John_Maxwell_IV 03 June 2012 06:55:11AM 4 points [-]

How many mathematicians consciously try to extract heuristics from their problem-solving process and keep them in a database, or track how environmental factors like diet and activities affect their productivity?

Has there ever been a team of mathematicians teamed with the team of mathematician optimizers who observed the mathematicians like lab animals? :D

Comment author: gwern 03 June 2012 08:23:26PM 4 points [-]

Has there ever been a team of mathematicians teamed with the team of mathematician optimizers who observed the mathematicians like lab animals?

Soviet Russia produced a remarkable amount of math, and ideologically was well-suited to such testing or design; they ultimately created whole academic cities for science and math, optimized (or at least, not pessimized like the rest of Soviet Russia) for research.

In fact, what I know of the Russian math academic system strikes me as reminiscent of the impression I have of the very successful athletic systems in both Russia and America: take young kids showing promise with relatives in related areas, push them hard with experienced tutors themselves skilled in the area, provide the resources they might need, various incentives for them and the relatives, and don't let off the slack until they begin to flag in their late 20s/early 30s at which point they take their tutors' places.

Comment author: faul_sname 03 June 2012 07:11:06AM 2 points [-]

Did you just go meta on the process of going less meta?

Comment author: John_Maxwell_IV 02 June 2012 11:55:29PM *  2 points [-]

A syllabus of recommended reading for folks who think they might want to work on FAI could potentially have a really high benefit to cost ratio. Could potentially have just as high a net benefit for reaching young talent as SPARC. Wouldn't necessarily take too much effort either, maybe just EY spending an hour brainstorming books an ideal collaborator would have read, and setting up a google group for people working through the syllabus.

I guess this could potentially increase UFAI risk a little, but I still judge it to be positive expectation. (SPARC could potentially increase UFAI risk too.)

Comment author: lukeprog 03 June 2012 12:42:22AM 4 points [-]
Comment author: John_Maxwell_IV 03 June 2012 04:06:55AM *  11 points [-]

Another point: I seem to recall a joke among mathematicians that if only it was announced that some famous problem was solved, without there actually being a solution, someone would try to find the solution for themselves and succeed in finding a valid solution.

In other words, how problems are framed may be important, and framing a problem as potentially impossible may make it difficult for folks to solve it.

Additionally, I see little evidence that the problems required for FAI are actually hard problems. This isn't to say that it's not a major research endeavor, which it may or may not be. All I'm saying is I don't see top academics having hammered at problems involved in building a FAI the same way they've hammered at, say, proving the Riemann hypothesis.

EY thinking they are super hard doesn't seem like much evidence to me; he's primarily known as a figure in the transhumanist movement and for popular writings on rationality, not for solving research problems. It's not even clear how much time he's spent thinking about the problems in between all of the other stuff he does.

FAI might just require lots of legwork on problems that are relatively straightforward to solve, really.

Comment author: John_Maxwell_IV 03 June 2012 12:54:06AM 5 points [-]

Nice!

That list doesn't actually seem very intimidating; for some reason I expected more highly technical AI papers and books. Why do you guys feel you need elite math talent as opposed to typical math grad student level talent? Which problems, if any, related to FAI seem unusually difficult compared to typical math research problems?

Comment author: lukeprog 03 June 2012 12:59:18AM 4 points [-]

Now we've come to the point where I'd like to be able to hand you Open Problems in Friendly AI, but I can't.

Comment author: Vladimir_Nesov 03 June 2012 10:09:19AM *  4 points [-]

IMO the extent to which some/most of these books/documents are only tentative suggestions with unclear relevance to the problem should be emphasized, for example they shouldn't be referred to with "After learning these basics", as if the list is definitive and works as some sort of prerequisite.

Also, using the words "deep understanding of mathematics, logic, and computation" to refer to the section with Sipser's introductory text is not really appropriate.

Comment author: John_Maxwell_IV 03 June 2012 01:08:42AM *  4 points [-]

In the Singularity Institute open problems document, you write:

Many of the problems related to navigating the Singularity have not yet been stated with mathematical precision, and the need for a precise statement of the problem is part of the problem.

Are you sure raw math talent is the best predictor of a person's ability to do this? I tend to associate this skill with programming especially, and maybe solving math word problems.

Comment author: lukeprog 03 June 2012 01:18:07AM 1 point [-]

Are you sure raw math talent is the best predictor of a person's ability to do this?

No, I'm not sure. The raw math talent thing is aimed more at the "Eliezer-led basement FAI team" stage.

Comment author: John_Maxwell_IV 03 June 2012 03:41:00AM 9 points [-]

Does Eliezer have experience with managing research teams?

Comment author: lukeprog 03 June 2012 10:35:34AM 5 points [-]

No. I should have said "Eliezer-guided," or something. Eliezer doesn't think it's a good idea for him to manage the team. We need our "Oppenheimer" for that.

Comment author: Will_Newsome 03 June 2012 01:54:45AM 5 points [-]

That's cool and a good intro, but you could also have a list of weaker suggestions over ten times that size to show people what sorts of advanced maths &c. might or might not end up being relevant. E.g., a summary paper from the literature on abstract machines, or even extremely young, developing subfields such as quantum algorithmic information theory that teach relevant cognitive-mathematical skills even if they're not quite fundamental to decision theory. This is also a sly way to interest people from diverse advanced disciplines. Is opportunity cost the reason such a list isn't around? My apologies if this question is missing the point of the discussion, and I'm sorry it's only somewhat related to the post, which is an important topic itself.