Comment author: moridinamael 28 March 2016 01:49:30PM 4 points [-]

I'm a little unclear on how to proceed. I didn't establish a "save", so I can't really resume the survey. Does that mean I should start a new survey and pick up where I left off, or ... ?

Comment author: ingres 29 March 2016 05:34:13PM *  2 points [-]

If you'd be willing to go through the trouble of doing it, yes that's exactly what you should do. I didn't think of that, thanks.

Though from a data-consistency perspective people doing this would skew our response rate higher than it really is, I'd rather have the question data than an accurate response rate though so. shrug

On the session timeout front, we're trying something out to make the sessions longer, which should cut down on that particular problem significantly.

Comment author: moridinamael 27 March 2016 06:43:48PM 26 points [-]

Was taking it, and it crashed with a "This webpage is not available" error.

Comment author: ingres 27 March 2016 09:28:42PM 4 points [-]

We had some power outage related downtime for three hours or so, should be back up now.

Comment author: ingres 27 March 2016 01:32:13AM 17 points [-]

I'd like to make a miniature announcement so there isn't any confusion:

Most of the time when somebody writes in a suggestion for improving the questions I don't reply to it, I just silently upvote the post and write down the question in a list of things to do for the next survey. But I am reading them, and I plan to go through and read them again before I wrap up the final survey analysis.

Comment author: benwr 26 March 2016 08:33:27AM 7 points [-]

Great survey!

However, when you save your progress and are asked to save a password, there's no indication that it will be sent to you in an email or saved at all in recoverable form. I used my least-secure password generation algorithm anyway, but: Do you think you could add a note to the effect that users should not use passwords that they use elsewhere?

Comment author: ingres 26 March 2016 10:16:36PM *  5 points [-]

Looking into it now.

EDIT: Added this warning to the save form:

"We store the password and send it to you by email, so please do not use a 'trusted' password for this that you use for anything important." (Not our design decision by the way.)

Comment author: Viliam 26 March 2016 09:38:18PM *  5 points [-]

Error
We are sorry but your session has expired.
Either you have been inactive for too long, you have cookies disabled for your browser, or there were problems with your connection.
Please contact namespace ( root@localhost ) for further assistance.

If you have to leave the computer in the middle of the survey, the software will punish you by throwing away your already completed answers. Really sucks after having completed about 100 of them. :(

What the hell was the purpose of checking whether someone was "inactive for too long"? So what, they were inactive, now they are active again, what's the big deal? Sometimes real life intervenes.

(Problems with connections happen too; I have a crappy wi-fi connection that I often have to restart several times a day. But that wasn't the case now. Also, why can't the software deal with disabled cookies? Calling root@localhost and waiting for an explanation...)

EDIT: If you happen to find yourself in a similar situation, use the e-mail mentioned in the article. As long as you remember enough data to uniquely identify your half-written response, the situation can be fixed.

Comment author: ingres 26 March 2016 10:01:59PM *  5 points [-]

Hi.

What the hell was the purpose of checking whether someone was "inactive for too long"? So what, they were inactive, now they are active again, what's the big deal? Sometimes real life intervenes.

I have no idea why that happened and I'm really sorry. It's definitely not supposed to. root@localhost isn't a real email address it's just there to stymie system 'error' messages we were receiving that were bogus.

The real mailing address you want is jd@fortforecast.com. We'd love to talk to you.

Comment author: Morgrim 26 March 2016 05:11:49AM 5 points [-]

The questions on donating to charity only relate to donating money to charity. Some people who have sufficient free time but little disposable income donate time to charities instead. I have seen reports that donating time over money is more common amongst students and people of low income, who seem to be a smaller proportion of the LW diaspora, but it may be interesting to compare donated time vs money on future surveys.

In my experience donating one's time is also seen as being extra keen on that cause, presumably because it requires more effort, and there are certain causes that consider time more valuable than funds (eg local environmental causes, where hiring sufficient people to remove invasive weeds from a local swamp is more expensive than holding a big weeding exercise on a Saturday afternoon).

Comment author: ingres 26 March 2016 05:26:11AM *  3 points [-]

This is a really good point. It'd make an especially interesting question set because it would give us some idea of how seriously LWers take the comparative advantage idea when it comes to charity, as measured by their actions.

Comment author: Yvain 26 March 2016 02:49:37AM 1 point [-]

If you throw out the data, I request you keep the thrown-out data somewhere else so I can see how people responded to the issue.

Comment author: ingres 26 March 2016 02:55:51AM *  4 points [-]

I don't throw out data. Ever. I only control for it. (Well barring exceptional circumstances.)

Comment author: Yvain 26 March 2016 02:42:51AM *  6 points [-]

"In general I planned to handle the "within 10 cm" thing during analysis. Try to fermi estimate the value and give your closest answer, then the probability you got it right. We can look at how close your confidence was to a sane range of values for the answer."

But unless I'm misunderstanding you, the size of the unspoken "sane range" is the entire determinant of how you should calibrate yourself.

Suppose you ask me when Genghis Khan was born, and all I know is "sometime between 1100 and 1200, with certainty". Suppose I choose 1150. If you require the exact year, then I'm only right if it was exactly 1150, and since it could be any of 100 years my probability is 1%. If you require within five years, then I'm right if it was any time between 1145 and 1155, so my probability is 10%. If you require within fifty years, then my probability is effectively 100%. All of those are potential "sane ranges", but depending on which one you the correctly calibrated estimate could be anywhere from 1% to 100%.

Unless I am very confused, you might want to change the questions and hand-throw-out all the answers you received before now, since I don't think they're meaningful (except if interpreted as probability of being exactly right).

(Actually, it might be interesting to see how many people figure this out, in a train wreck sort of way.)

PS: I admit this is totally 100% my fault for not getting around to looking at it the five times you asked me to before this.

Comment author: ingres 26 March 2016 02:46:40AM *  3 points [-]

Yeah, you're right.

Currently trying to figure out how to do that in the least intrusive way.

EDIT: Good news it turns out that I can edit the calibration question 'answers' after all. The ones where a range would make sense have been edited to include one. Questions such as "which is heavier" have not been because the ignorance prior should be fairly obvious.

Fri Mar 25 19:50:41 PDT 2016 | Answers on or before this date where the ranges have been added will be controlled for at analysis time.

Comment author: Elo 26 March 2016 02:14:30AM 6 points [-]

Since you are such a huge part of the diaspora community I would be delighted if you could share the survey to both your readers and your friends.

We will get that suggestion sorted asap.

In response to comment by Elo on Lesswrong 2016 Survey
Comment author: ingres 26 March 2016 02:27:17AM *  1 point [-]

We will get that suggestion sorted asap.

I actually can't do that. The way our survey engine works changing the question answers mid-survey would require taking it down for maintenance and hand-joining the current respondents to the new respondents. In general I planned to handle the "within 10 cm" thing during analysis. Try to fermi estimate the value and give your closest answer, then the probability you got it right. We can look at how close your confidence was to a sane range of values for the answer.

I.E, if you got it within ten and said you had a ten percent chance of getting it right you're well calibrated.

Note: I am not entirely sure this is sane, and would like feedback on better ways to do it.

EDIT: I should probably be very precise here. I cannot change the question answers in the software, presumably because it would involve changing the underlying table schema for the database. I can change the question/ question descriptions so if there's a superior process for answering these I could describe it there.

Comment author: Huluk 26 March 2016 12:55:37AM *  26 points [-]

[Survey Taken Thread]

By ancient tradition, if you take the survey you may comment saying you have done so here, and people will upvote you and you will get karma.

Let's make these comments a reply to this post. That way we continue the tradition, but keep the discussion a bit cleaner.

Comment author: ingres 26 March 2016 01:51:02AM 37 points [-]

I have taken the survey.

View more: Prev | Next