Vaniver comments on Rationality Quotes September 2011 - Less Wrong

7 Post author: dvasya 02 September 2011 07:38AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (482)
Tags: quotes

Comments (482)

You are viewing a single comment's thread. Show more comments above.

Comment author: gwern 11 February 2013 02:35:06AM 2 points [-]

Most importantly, we appear to have figured out the answer to my original question: no, it is not easy. :P

And inadvertently provided an object lesson for anyone watching about the value of researchers providing code...

The value of the smart fraction cutoff appears to have a huge impact on the napping from smart fraction to gdp, but doesn't appear to have a significant effect on the goodness of fit, which troubles me somewhat. I'm also surprised that deleting the outliers seems to have improved the performance of the exponential fit more than the smart fraction fit, which is not what I would have expected from the graphs.

My intuition so far is that La Griffe found a convoluted way of regressing on a sigmoid, and the gain is coming from the part which looks like an exponential. I'm a little troubled that his stuff is so hard to reproduce sanely and that he doesn't compare against the exponential fit: the exponent is obvious, has a reasonable empirical justification. Granting that Dickerson published in 2006 and he wrote the smart fraction essay in 2002 he could at least have updated.

[edit] Sorry, it looks like the formatting for my code is totally ugly.

You need to delete any trailing whitespace in your indented R terminal output. (Little known feature of LW/Reddit Markdown code blocks: one or more trailing spaces causes the newline to be ignored and the next line glommed on. I filed an R bug to fix some cases of it but I guess it doesn't cover nls or you don't have an updated version.)

I don't understand your definition

egdp <- function(iq,iq0,m,b) (m*sf(iq,iq0)+b)

sf(iq,iq0) makes sense, of course, and m presumably is the multiplicative scale constant LG found to be 69k, but what is this b here and why is it being added? I don't see how this tunes how big a smart fraction is necessary since shouldn't it then be on the inside of sf somehow?

But using that formula and running your code (using the full dataset I posted originally, with outliers):

R> erf <- function(x) 2 * pnorm(x * sqrt(2)) - 1
R> sf <- function(iq,iq0) ((1+erf((iq-iq0)/(15*sqrt(2))))/2)
R> egdp <- function(iq,iq0,m,b) (m*sf(iq,iq0)+b)
R> summary(nls(rGDPpc ~ egdp(IQ,iq0,m,b), lynn, start=list(iq0=110,m=40000,b=0)))
Formula: rGDPpc ~ egdp(IQ, iq0, m, b)
Parameters:
Estimate Std. Error t value Pr(>|t|)
iq0 102.08 4.89 20.88 < 2e-16
m 37108.87 9107.73 4.07 0.00011
b 1140.94 1445.76 0.79 0.43241
Residual standard error: 5320 on 78 degrees of freedom
Number of iterations to convergence: 7
Achieved convergence tolerance: 5.09e-06
Comment author: Vaniver 11 February 2013 03:25:34AM 0 points [-]

he doesn't compare against the exponential fit: the exponent is obvious, has a reasonable empirical justification.

The theoretical justifications are entirely different, though. It seems reasonable to me to suppose there's some minimal intelligence to be wealth-producing in an industrial society, and the smart fraction estimates that well and it predicts gdp well. But, it also seems reasonable to treat log(gdp) as a more meaningful object than gdp.

It's also bothersome that the primary empirical prediction of the smart fraction model (that there is some stable gdp level that you hit when everyone is higher than the smart fraction) is entirely from the extrapolated part of the dataset, and this doesn't seem noticeably better than the exponential model, whose extrapolations are radically different.

Granting that Dickerson published in 2006 and he wrote the smart fraction essay in 2002 he could at least have updated.

Yeah; I'm curious what they'd have to say about the relative merits of the two models. I'll see if I can get this question to them.

You need to delete any trailing whitespace in your indented R terminal output.

Fixed, thanks!

but what is this b here and why is it being added?

It's an offset, so that it's an affine fit rather than a linear fit: the gdp level for a population with no people above 108 IQ doesn't have to be 0. Turns out, it's not significantly different from zero, but I'd rather discover that than enforce it (and enforcing it can degrade the value for m).

Comment author: gwern 11 February 2013 03:46:31AM 0 points [-]

But, it also seems reasonable to treat log(gdp) as a more meaningful object than gdp.

I'm not entirely sure... For individuals, log-transforms make sense on their own merits as giving a better estimate of the utility of that money, but does that logic really apply to a whole country? More money means more can be spent on charity, shooting down asteroids, etc.

It's also bothersome that the primary empirical prediction of the smart fraction model (that there is some stable gdp level that you hit when everyone is higher than the smart fraction) is entirely from the extrapolated part of the dataset, and this doesn't seem noticeably better than the exponential model, whose extrapolations are radically different.

The next logical step would be to bring in the second 2006 edition of the Lynn dataset, which increased the set from 81 to 113, and use the latest available per-capita GDP (probably 2011). If the exponential fit gets better compared to the smart-fraction sigmoid, then that's definitely evidence towards the conclusion that the smart-fraction is just a bad fit.

Yeah; I'm curious what they'd have to say about the relative merits of the two models. I'll see if I can get this question to them.

I'd guess that he'd consider SF a fairly arbitrary model and not be surprised if an exponential fits better.

It's an offset, so that it's an affine fit rather than a linear fit: the gdp level for a population with no people above 108 IQ doesn't have to be 0. Turns out, it's not significantly different from zero, but I'd rather discover that than enforce it (and enforcing it can degrade the value for m).

Why can't the GDP be 0 or negative? Afghanistan and North Korea are right now exhibiting what such a country looks like: they can barely feed themselves and export so much violence or fundamentalism or other dysfunctionality that rich nations are sinking substantial sums of money into supporting them and fixing problems.

Comment author: Vaniver 11 February 2013 04:28:59PM *  0 points [-]

For individuals, log-transforms make sense on their own merits as giving a better estimate of the utility of that money, but does that logic really apply to a whole country?

The argument would be that additional intelligence multiplies the per-capita wealth-producing apparatus that exists, rather than adding to it (or, in the smart fraction model, not doing anything once you clear a threshold).

Why can't the GDP be 0 or negative?

There's no restriction that b be positive, and so those are both options. I wouldn't expect it to be negative because pre-industrial societies managed to survive, but that presumes that aid spending by the developed world is not subtracted from the GDP measurement of those countries. Once you take aid into account, then it does seem reasonable that places could become money pits.

Comment author: gwern 11 February 2013 06:13:57PM 0 points [-]

The argument would be that additional intelligence multiplies the per-capita wealth-producing apparatus that exists, rather than adding to it (or, in the smart fraction model, not doing anything once you clear a threshold).

That's the intuitive justification for an exponential model (each additional increment of intelligence adds a percentage of the previous GDP), but I don't see how this justifies looking at log transforms.

There's no restriction that b be positive, and so those are both options. I wouldn't expect it to be negative because pre-industrial societies managed to survive

The difference would be a combination of negative externalities and changing Malthusian equilibriums: it has never been easier for an impoverished country like North Korea or Afghanistan to export violence and cause massive costs they don't bear (9/11 directly cost the US something like a decade of Afghanistan GDP once you remove all the aid given to Afghanistan), and public health programs like vaccinations enable much larger populations than 'should' be there.

Comment author: Vaniver 11 February 2013 06:31:44PM *  0 points [-]

That's the intuitive justification for an exponential model (each additional increment of intelligence adds a percentage of the previous GDP), but I don't see how this justifies looking at log transforms.

GDP ~ exp(IQ) is isomorphic to ln(GDP) ~ IQ, and I think log(dollars per year) is an easier unit to think about than something to the power of IQ.

[edit] The graph might look different, though. It might be instructive to compare the two, but I think the relationships should be mostly the same.

Comment author: Kindly 11 February 2013 10:16:36PM 4 points [-]

It's worth pointing out that IQ numbers are inherently non-parametric: we simply have a ranking of performance on IQ tests, which are then scaled to fit a normal distribution.

If GDP ~ exp(IQ), that means that the correlation is better if we scale the rankings to fit a log-normal distribution instead (this is not entirely true because exp(mean(IQ)) is not the same as mean(exp(IQ)), but the geometric mean and arithmetic mean should be highly correlated with each other as well). I suspect that this simply means that GDP approximately follows a log-normal distribution.

Comment author: Vaniver 11 February 2013 11:32:38PM 1 point [-]

I suspect that this simply means that GDP approximately follows a log-normal distribution.

This doesn't quite follow, since both per capita GDP and mean national IQ aren't drawn from the same sort of distribution as individual production and individual IQ are, but I agree with the broader comment that it is natural to think of the economic component of intelligence measured in dollars per year as lognormally distributed.

Powered by Reddit Powered by Reddit