[META] My Negative Results - Less Wrong Discussion

[META] My Negative Results

Sun, 29 Jan 2012 07:44:47 +1100

Submitted by RobertLumley • 37 votes • 23 comments

Yesterday, I made a post asking if anyone else had noticed LW being particularly slow. I offered to collect data on this, and was fairly sure (Probably about 80%) that it would show that LW loaded slower than other webpages. I took the post down after about 20 seconds (sorry if I confused you) since it was almost entirely insubstantial, and resolved to collect some actual data to report.

So I did that. Data was taken using this website. I was on my school's wireless network at the time, running Firefox 8.0. I didn't think to disable my addons before doing this, but I was running Adblock Plus, FastestFox, Greasemonkey, IE Tab 2, Movable Firefox Button, Omnibar, and Web of Trust. Data points were generated at the following websites. Each website was measured five times.
http://lesswrong.com/r/discussion/new
http://lesswrong.com
http://lesswrong.com/user/RobertLumley
http://lesswrong.com/promoted/
http://predictionbook.com
http://predictionbook.com/predictions
http://predictionbook.com/users/rlumley
http://www.nature.com/news/index.html
http://volokh.com/
http://online.wsj.com/public/page/news-opinion-commentary.html
http://www.webdiplomacy.net/index.php

After doing this, I downloaded an offline copy of each of these websites, and calculated load time per byte of website size. I plotted these results. To my surprise, LessWrong ended up being one of the fastest, although PredictionBook could use some work. I considered deleting the obvious outlier, but trends are clear even with it in there, and all things equal, I'd rather not delete data. Data points (in groups of 5) correspond to the webpages above, in order, ie. the first five points are from the discussion page of LW.

Surprisingly, LW is still one of the best even when only raw load time is compared. But the discussion page (remember, that's the first 5 points) takes somewhat longer than the other pages:

The excel spreadsheet used to generate this can be viewed here.

Since there is no real problem here, and it was all in my head, I considered not publishing this, but after all of the discussion we have about positive bias, I wasn't about to turn around and do the same thing. So here they are: my negative results.

23 comments

Douglas_Knight on [META] My Negative Results

2012-01-29T09:26:06.797975+11:00

You are pretty much wasting the x-axis, which is pretty much redundant with the colors. It would be better to do a scatter plot of time and size, rather than imposing the ratio on people like me who think it is a silly metric.

This may be a realistic way to measure time for new users, but that is irrelevant to your complaint, since you are not a new user. The large size of the site includes files that don't change much, like the javascript. These load quickly, at least per byte, contributing to a fast measure. Some users will be bandwidth-limited. But when you return to the site, these files are cached and the total download is much smaller. But you still have to wait for database queries to build the main content. This has only a very loose relation to number of bytes, even the number of bytes the pages, and using number of bytes as an excuse for speed seems silly to me. (But there is a lot of variation from time to time and if you had caught it at a bad time it would probably look bad by your metric, too.)

RobertLumley on [META] My Negative Results

2012-01-29T09:29:10.572886+11:00

Hm. I don't know much about the way websites are parsed, but I figured it wasn't a perfect metric. At any rate, that's why I did both raw time and website size.

Douglas_Knight on [META] My Negative Results

2012-01-29T15:21:28.084359+11:00

No, you did not do raw time. That's the whole point of my comment.

RobertLumley on [META] My Negative Results

2012-01-30T00:42:23.931990+11:00

Perhaps you can't see the graph for some reason. (But why you could see one and not the other is beyond me) There were other complaints as well, and I had a problem getting them to show up just for me. I tried uploading them as images, but I got a "Bad image" message. I'd suggest just opening up the spreadsheet I linked to, it's in there.

Douglas_Knight on [META] My Negative Results

2012-01-30T01:56:55.842943+11:00

Sorry. My fault.

Alex_Altair on [META] My Negative Results

2012-01-29T08:09:18.568843+11:00

Upvoted for actually doing science.

Grognor on [META] My Negative Results

2012-01-29T08:23:08.728288+11:00

I defy your data. LW has been acting consistently damn slow lately, and I've seen other people in the LW irc channel have been complaining about this.

Edit, 1 day later: on reflection, "consistently" was a terrible word choice. I stand by this statement with that word removed.

RobertLumley on [META] My Negative Results

2012-01-29T08:26:16.555900+11:00

Hm. I thought so too. Maybe I hit it at a good time?

daenerys on [META] My Negative Results

2012-01-29T09:28:55.924414+11:00

Also upvoted for Doing Science!

But I agree with your hypothesis that you hit it at a good time. Occasionally LW has been working great, but occasionally these past few days it won't load AT ALL.

RobertLumley on [META] My Negative Results

2012-01-29T09:30:33.352252+11:00

There have been several instances where it cut out completely for me. But my internet at home is very sketchy, so I wasn't sure which it was. Other sites would typically load, but they were usually pretty slow.

jsalvatier on [META] My Negative Results

2012-01-29T16:09:45.802924+11:00

I and others have experienced both slowness and totally cut out of the site.

daenerys on [META] My Negative Results

2012-01-29T09:32:26.959256+11:00

But my internet at home is very sketchy, so I wasn't sure which it was.

Ditto.

Graph over time?

RobertLumley on [META] My Negative Results

2012-01-29T09:34:21.231106+11:00

The spikes for me don't last more than two minutes (if that), and are pretty infrequent, as far as I've noticed. It might be helpful to create a thread to point out when we see them, but a reliable graph over time would require far more resources than I am willing to devote.

Luke_A_Somers on [META] My Negative Results

2012-01-30T02:02:07.434944+11:00

Inappropriate application of Denying your Data. This data is quite consistent with erratic performance.

nyan_sandwich on [META] My Negative Results

2012-01-30T13:57:52.363812+11:00

I can confirm that LW is really really slow (15 second load time) sometimes.

jimrandomh on [META] My Negative Results

2012-01-29T12:51:29.678604+11:00

These days, in practice, "slow" is usually a special case of "broken"; and the tricky thing about software brokenness is that there tend to be additional preconditions before it appears. I have occasionally seen Less Wrong being slow, but it doesn't appear to be slow consistently.

The best way to investigate page-load time issues is to open Chrome's developer tools panel (ctrl+shift+i or command+shift+i), go to the Network tab, and refresh. This will give you a list of all the individual parts of the page (including things loaded from third party CDNs, which are frequently sources of trouble), and a breakdown of when they started loading, how big they were, and how long they took. On the other hand, the problem might be on the server side. In that case, most of the time will go to the first entry in the list, the page's HTML; and further investigation will have to be done on the server, rather than on the browser.

STL on [META] My Negative Results

2012-01-29T21:54:35.124643+11:00

Load times should be measured in milliseconds, not seconds.

Related: Shit-ass Websites

RobertLumley on [META] My Negative Results

2012-01-30T00:43:44.868396+11:00

I measured to millisecond precision, I just didn't bother changing the graph.

Related: Hyphen

STL on [META] My Negative Results

2012-01-30T02:14:33.026778+11:00

Let me try again - if a page takes more than a second to load, it's too slow.

ahartell on [META] My Negative Results

2012-01-29T09:13:54.847414+11:00

Some of the images aren't loading for me, specifically, those immediately above and below the quoted text:

Surprisingly, LW is still one of the best even when only raw load time is compared. But the discussion page (remember, that's the first 5 points) takes somewhat longer than the other pages:

I'm using Chrome and Mac OSX.

RobertLumley on [META] My Negative Results

2012-01-29T09:19:57.664317+11:00

Hm. Works for me in Chrome. They're just the graphs of load time and load type per byte. I'd recommend using firefox (That may not work though, if chrome isn't) or just looking at the spreadsheet - they're in there.

Normal_Anomaly on [META] My Negative Results

2012-01-29T14:03:48.467465+11:00

Firefox shows them for me.

RobertLumley on [META] My Negative Results

2012-01-29T07:51:35.104252+11:00

Sorry for any confusion. I had a lot of problems with the graphs. I think it's fixed now though.