Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Persol comments on Who Wants To Start An Important Startup? - Less Wrong

41 Post author: ShannonFriedman 16 August 2012 08:02PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (407)

You are viewing a single comment's thread. Show more comments above.

Comment author: Micaiah_Chang 17 August 2012 12:56:05AM 6 points [-]

I imagine this would be very hard to monetize and get customers as-is. The below is merely a brief list of problems that I've thought about

The average user needs to be sold on the effectiveness of a product very fast, on the first usage (or perhaps even sooner!) in order for them to continue using. However, SRS software in general are almost by definition antithetical to that goal: Their benefits do not come until far into the future, worse still it's an undefined time in the future. Sure you can use arguments about the benefits of SRS and the psychology of memory and <insert gwern.net FAQ here>, but it would appear to be an uphill battle to make the benefits immediately relevant and immediately relevant to the people who wouldn't already be using Anki and other free equivalents.

In addition, before you can even start using the product as advertised, you have to learn how to make cards that are easy to memorize or download a deck which is already well made. The first is "Wait so you want me to learn all these tiny rules before I can even start learning? <browser back button>". The second presents a chicken-and-egg problem. How are you going to have high quality decks that teach things? By having users! How are you going to get users? By having high qual- oh, darn.

It would appear that your general idea is going in the right direction; to make the best SRS program as painless as possible and to extend it to be more powerful. Your emphasis though, would appear to be more oriented toward existing power users of SRS. So there's the matter of getting them to switch which... I have no idea how hard that would be. (Sample size of two; you'd obviously build something you'd want to use; I'd jump on board instantly if I could transfer my existing Anki decks).

One possible solution to the adoption is to piggyback it on an existing service; if users get to use it as an additional option on something they already use habitually then getting consistent usage wouldn't be as much of a problem. I believe Khan Academy has expressed interest in including SRS in there. Another is to try and "gamify" it (argh I hate that word) by either making the entire application a sort of game or incorporating cow clicker like features in there to get the user hooked (IT'S NOT EVIL IF THE ADDICTION IS GOOD).

The making your own decks feature can be mostly hidden from the normal user, with a gradual introduction to it as they use the product more (paid feature?). As for having high quality decks; you can try porting the entire Anki library of downloadable decks, filter them in some way and use that to bootstrap up to a much higher standard of quality.

Of course, any and all advice here means absolutely jack compared to the behavior of actual users; release a minimal version, see who bites and check to see what the users complain about before even thinking about what I said here. Making money is and should probably be a distant 4th or 5th consideration behind making a product that you would use and making it easily extensible.

Comment author: Persol 17 August 2012 01:29:20PM *  7 points [-]

I think Micaiah_Chang mostly nailed this. I actually wrote a site that did this a few months ago. I had about 4000 users who had actually gone through a complete session.

it would appear to be an uphill battle to make the benefits immediately relevant

As guessed, the problem is that I couldn't get people to start forming it as a habit. There is no immediate payback. Less than 20 people out of 4000 did more than one session.

you have to learn how to make cards that are easy to memorize or download a deck which is already well made

This one is easily solved. The Anki decks have a (weak) rating system, and allowing people to import anki decks was easy.

Additionally, there are at least 18 competitors. Here's the list I made at the time. Very few seem to be successful.

I shut the site down about a month ago. There are numerous free competitors which don't have any great annoyances. I wouldn't suggest starting another of these sites unless you figured out an effective way to "gamify" it.

Comment author: gwern 18 August 2012 01:33:21AM 3 points [-]

As guessed, the problem is that I couldn't get people to start forming it as a habit. There is no immediate payback. Less than 20 people out of 2000 did more than one session.

Wow. You did a spaced repetition site which had 4000 people try, 2000 finish a session, and <20 return for a second review session?

Comment author: Persol 18 August 2012 02:43:42AM *  3 points [-]

Sorry, typo. ~4000 people finished a session. Many more 'tried' than 4000... I just couldn't determine which users were bots that registered randomly vs users that didn't finish the first session.

  • Tried: lots (but unknown)
  • Finished 1 session: ~4000
  • Finished >1 session: ~20
Comment author: Shak 18 August 2012 05:35:48AM 1 point [-]

Persol, what traffic generating methods did you use to get those kind of figures?

Comment author: Persol 18 August 2012 12:13:26PM *  2 points [-]

About 75% of the hits came from Google adWords, which was on for about 8 months. Maybe about 10% from search results. I also had a few links from subject specific websites. Average CTR was about 0.25%. Best CTR were ads that mentioned 'flashcards' and 'online'. The best conversion rate (answered a study session question) was 17% with the ad below:

  • Remember Facts
  • Spend less time studying.
  • Remember more material.
  • www.superbrain.me

Graphic/animated ads were a waste of money, but at least I learned how to make animated GIFs.

Comment author: supermemofan 21 August 2012 01:10:47AM 6 points [-]

Supermemo has been working on this problem since 1982, and they struggle to make Supermemo popular because it is not "sticky" enough. Basically it involves hard work for future benefit. This is the mental equivalent of "earning an honest living," or "getting rich slowly." We do not live in a world that portrays honest, slow but meaningful progress. We live in a world that is obsessed with instant gratification, and SRS methods go contrary to the river-like "current" of this system. See: http://wiki.supermemo.org/index.php?title=Why_isn'tSuperMemomore_popular%3F Also, the only way I could see SRS hitting massive appeal is if it were designed from the ground up to be a game of some sort where doing flashcard repetitions resulted in progress. I have an idea of how it would work, but I doubt we will see anything like that for the time being.

Comment author: shminux 21 August 2012 07:08:57PM 3 points [-]

Also, the only way I could see SRS hitting massive appeal is if it were designed from the ground up to be a game of some sort

Judging by the user-hostile interface of the linked site, there is probably a much lower hanging fruit in the app's UI redesign.

Comment author: supermemofan 22 August 2012 01:59:17AM 0 points [-]

Supermemo was designed by a scientist, and a simple user interface is not as high on the priority list as optimized algorithms, incremental reading, etc. I highly doubt that outside of Supermemo a more effective flashcard software exists. Thus, to make flashcard-based learning hit critical mass, it must be packaged in a way that is extremely appealing and fun (Aside from the joy of learning) so that people can survive the two-week "hump" that most people stop using Supermemo. Other than a game or a more aesthetically pleasing UI, I think we're kind of stuck with our present situation, unless a higher authority (Education system, government) makes such learning techniques mandatory, which will not happen due to the inefficiency of modern institutions to fulfill their explicit purpose. At least they exist for the self-motivated to use, and it gives us a mental edge on those that do not use (Or care to use) SRS.

Comment author: Persol 21 August 2012 06:54:46PM *  2 points [-]

That link (fixed version ) is very accurate. I wish I'd considered the first few points BEFORE programming/advertising the site.