Comment author: halcyon 24 June 2016 09:42:52PM 2 points [-]

I don't want to live forever myself, but I want people who want to live forever to live forever. Does that make me a transhumanist?

Comment author: Lumifer 23 June 2016 05:53:25PM *  0 points [-]

Not being stupid is an admirable goal, but it's not well-defined.

It's not a goal. It is a criterion you should apply to the steps which you intend to take. I admit to it not being well-defined :-)

Is there a standard term for the error you are referring to?

In statistics that used to be called "data mining" and was a bad thing. Data science repurposed the term and it's now a good thing :-/ Andrew Gelman calls a similar phenomenon "garden of the forking paths" (see e.g. here).

Basically the problem is paying attention to noise.

Can't I have my common sense, but make all possible comparisons anyway

You can. It's just that you shouldn't attach undue importance to which comparison came the first and which the second. You're generating estimates and at the very minimum you should also be generating what you think are the errors of your estimates -- these should be helpful in establishing how meaningful your ranking of all the pairs is.

And you still need to define a goal. For example, a goal of explanation/understanding is different from the goal of forecasting.

I'm not telling you to ignore the data. I'm telling you to be sceptical of what the data is telling you.

Comment author: halcyon 23 June 2016 08:42:21PM *  0 points [-]

Thank you! Those data mining algorithms are exactly what I was looking for.

(Personally, I would describe the situation you are warning me against as reducing it "more than is possible" rather than "as much as possible". I am definitely in favor of using common sense.)

Comment author: Lumifer 23 June 2016 04:27:27PM 1 point [-]

the whole idea was to minimize that as much as possible

I believe this idea to be misguided. The point of the process is to understand. You can't understand without "interpretation" -- looking for just the biggest numbers inevitably leads you astray.

The issue isn't what you can rationalize -- "don't be stupid" is still the baseline, level zero criterion.

What conditions must my goal satisfy in order to qualify as a "well-defined goal"?

A specification of what kind of answers will be acceptable and what kind will not.

Have I made any actual (meaning technical) mistakes so far?

Are you asking whether your spaghetti factory mixes flour and water in the right ratio?

Comment author: halcyon 23 June 2016 04:48:49PM -1 points [-]

Not being stupid is an admirable goal, but it's not well-defined. I tried Googling "spaghetti factory analysis" and "spaghetti factory analysis statistics" for more information, but it's not turning up anything. Is there a standard term for the error you are referring to?

Can't I have my common sense, but make all possible comparisons anyway just to inform my common sense as to the general directions in which the winds of evidence are blowing?

I don't see how informing myself of correlations harms my common sense in any way, and the only alternative I can think of is to stick to my prejudices, but whenever some doubt arises as to which of my prejudices has a stronger claim, I should thoroughly investigate real world data to settle the dispute between the two. As soon as that process is over, I should stop immediately because nothing else matters.

Is that the course of action you recommend?

Comment author: Lumifer 23 June 2016 03:19:04PM 1 point [-]

I'm trying to get at least a vague handle on what I can legitimately infer

That's not a very well-defined goal. You are engaging in what's known as a spaghetti factory analysis: make a lot of spaghetti, throw it on the wall, pick the most interesting shapes. This doesn't tell you anything about the world.

Sure, you can start with correlations. But that's only a start. Let's say you've got a high correlation between A and B. The next questions should be: Does it make sense? Is there a plausible mechanism underlying this correlation? Is it stable in time? Is it meaningful? And that's before diving into causality which correlations won't help you much with.

You still need a better goal of the analysis.

Should I try Bayesian causal inference anyway, just to see what I get? Support vector machines? Markov random fields?

Nooooo! You don't understand basic stats, trying to (mis)use complicated tools will just let you confuse yourself more thoroughly.

Comment author: halcyon 23 June 2016 03:31:33PM *  0 points [-]

Sure, I can always offer my own interpretations, but the whole idea was to minimize that as much as possible. I can rationalize anything. Watch: Milk consumption is negatively correlated with income inequality. Drinking less milk leads to stunted intelligence, resulting in a rise in income inequality. Or income inequality leads to a drop in milk consumption among poor families. Or the alien warlord Thon-Gul hates milk and equal incomes.

What conditions must my goal satisfy in order to qualify as a "well-defined goal"? Have I made any actual (meaning technical) mistakes so far? (Anyway, thanks for reminding me to check for temporal stability. I should write a script to scrape the data off pdfs. (Never mind, I found a library.))

Comment author: Lumifer 23 June 2016 02:34:53PM 0 points [-]

What is it that you want to do?

Just looking at correlations and nothing else can lead to funny results.

Comment author: halcyon 23 June 2016 02:43:15PM *  -1 points [-]

I'm trying to get at least a vague handle on what I can legitimately infer from what using data that might, and probably does, contain circular causation. I'm looking for statistical tools that might help me do that. Should I try Bayesian causal inference anyway, just to see what I get? Support vector machines? Markov random fields? Does the Spurious Correlations book have ideas on that? (No, it just seems to be an awesome set of correlations. Thanks, BTW.)

(Also notice that these are not just any correlations. These are the strongest correlations that pertain among a large number of variables relative to each other. I mean, I computed all possible correlations among every combination of 2 variables in hopes that the strongest I find for each variable might show something interesting.)

Comment author: halcyon 23 June 2016 01:58:56PM *  -1 points [-]

I collected some social statistics from the internet and computed their correlations: https://drive.google.com/open?id=0B9wG-PC9QbVERHdiTi1uTlFMMlU My sources were: http://pastebin.com/ERk1BaBu

But I'm not sure how to proceed from there: https://drive.google.com/open?id=0B9wG-PC9QbVEWlRZSG9KM0ZFeVk ?? Dotted lines represent positive correlations and arrowed lines negative correlations.

I obtained that confusing chart by following this questionable method: https://drive.google.com/open?id=0B9wG-PC9QbVEVHg1T1lQNE1ZTk0 First, drop some of the trivial correlations like the ones among the different measures of national wealth, and weaker correlations between +.5 and -.5. For each variable, select the correlation furthest from 0 and throw it into the chart. I also tried keeping only one measure of national wealth in the model in hopes of less confusion: https://drive.google.com/open?id=0B9wG-PC9QbVEZlExWmhoOWRjVk0

I'm looking for help in analyzing this data. Are there any methods you would recommend? Which variables should I drop for better results? I tried keeping only proportions at one point. (Bayesian causal inference assumes the nonexistence of circular causation AIUI, a condition I can't guarantee with this data, to say the least.)

(Fixed the links. Sorry about that.)

Comment author: ChristianKl 05 May 2016 06:27:46PM -1 points [-]

Hitler had a huge party of supporters behind him that he spend a decade gathering around him. Trump on the other hand is much more of an one-man show. One of the biggest role of the president is making personal choices and there simply no comparable pool of talent. Under a Trump administration someone like Chris Christie who's a long-term friend of the Trump family is likely going to get a post in his administration.

When it comes to totalitarism it's a mistake to assume that the past will repeat exactly the same way. It's hard to believe a US government would simply torture random people with Arabic names intentionally just because they have Arab names. It's more likely that privacy will get completely eroded. Today we have face recognition that's strong enough to hook up all camera's on streets to it and get general movement profiles. Forbidding encryption would also be on the table.

Comment author: halcyon 05 May 2016 09:02:35PM 0 points [-]

Thanks, I'm basically ignorant about contemporary American politics. (But I've read Tocqueville. This is probably not a desirable state of affairs.)

Comment author: Dagon 05 May 2016 04:47:51PM 3 points [-]

assume, as a rule of thumb, that politicians intend to fulfill all their promises.

This is a horrible rule of thumb. It's not anywhere close to true, and even if it were, their ability lags their intent by orders of magnitude. Instead, assume that politicians will very slightly alter existing trends in order to encourage their constituents.

I suspect you are at somewhat higher risk of being targetted by officials for your foreign-ness than you were last year. Trump being president will somewhat increase as well, but more because it'll be a sign that the general populace is more racist than we thought than because of any actual policy change.

I think it's really unlikely you'd be imprisoned or tortured, with or without Trump, unless there are stronger ties to enemy groups than just your nationality.

Comment author: halcyon 05 May 2016 08:46:41PM 0 points [-]

I assume that because I read on the SEP that strategic voting skews results in democracies. The rule of thumb is more like a Schelling point than a lower order rational principle. I said that's what I usually do because I'm aware it's not very applicable in this context since I'm not voting in these elections, but it's a habit I've indulged in for years, unfortunately.

Comment author: knb 05 May 2016 03:03:02PM 6 points [-]

Seems like you're just falling for partisan media histrionics and conflating a lot of different things out of context.

On Fox News, Trump said that regarding Muslims in the US, he would do "unthinkable" things, "and certain things will be done that we never thought would happen in this country".

In context, Trump is giving a tough-sounding but vague and non-committal response to questions about whether there should be a digital database of Muslims in the country. He later partially walked this back, saying it was a leading question from a reporter and he meant we should have terrorism watch lists. Which obviously already exist.

I've read Cory Doctorow's Little Brother, and this might be a generalization from fictional evidence, but I can't help asking: As a foreign student in the US, how likely is Trump to have me tortured for no reason?

I'd say it's about as likely as you giving yourself a heart attack reading political outrage porn.

Comment author: halcyon 05 May 2016 08:35:21PM *  1 point [-]

Thanks, I guess. I knew he was talking about a digital database, but I was wondering if it could have been a dogwhistle for something else. I don't have a favorable opinion of human decency in general.

FWIW, that wasn't a political comment. I hardly ever read or watch anything political. Some TV clips were shown to me by an acquaintance and I wanted an honest assessment of what he had told me it was about. I don't have any opinions on the subject myself.

Comment author: halcyon 05 May 2016 12:40:35PM *  -1 points [-]

On Fox News, Trump said that regarding Muslims in the US, he would do "unthinkable" things, "and certain things will be done that we never thought would happen in this country". He also said it's impossible to tell with absolute certainty whether a Syrian was Christian or Muslim, so he'd have to assume they're all Muslims. This suggests that telling US officials that I'm a LW transhumanist might not convince them that I have no connection with ISIS. I'm not from Syria, but I have an Arabic name and my family is Muslim.

I've read Cory Doctorow's Little Brother, and this might be a generalization from fictional evidence, but I can't help asking: As a foreign student in the US, how likely is Trump to have me tortured for no reason? Should I drop everything and make a break for it before it's too late? Initially, many Germans didn't take Hitler's extremist rhetoric seriously either, right? (If I get deported in a civilized manner, well, no harm done to me as far as I'm concerned.)

I normally assume, as a rule of thumb, that politicians intend to fulfill all their promises. If a politician says he wants to invade Mars, that could be pure rhetoric, but I'd typically assume that he might try it in the worst case scenario. I have observed it is often the case that when we think other people are joking, they are in fact exaggerating their true desires and presenting them in an ironic/humorous light.

View more: Next