Me and feep have implemented a slightly-tweaked version of this using a DQN agent in Reinforce.js. (Tabular turns out to be a bit infeasible.)

• A live demo: http://www.gwern.net/docs/rl/armstrong-controlproblem/index.html
• The core JS: http://www.gwern.net/docs/rl/armstrong-controlproblem/controlproblem.js
• Browseable source: https://github.com/gwern/gwern.net/tree/master/docs/rl/armstrong-controlproblem

At the moment, if you want to modify settings like in Karpathy's demos, you'll have to do something like download it locally to edit, with a command like wget --mirror 'www.gwern.net/docs/rl/armstrong-controlproblem/index.html' && firefox ./www.gwern.net/docs/rl/armstrong-controlproblem/index.html