I chose exactly the wrong D&D.Sci to decide to not try building a model on, and instead try to solve just by eyeballing simple scatterplots.
Despite coming in "last place" I'm pretty happy with my results!
I think this was a perfectly reasonable setup. Even more so given that without any straightforward scenarios people won't think "what if it's just straightforward though".
I thought many times during eyeballing "look, probably no one else has tried to build a model using all combos of mins and maxes of stats as features, just do it" but I stuck to my guns. For, uh, reasons. Presumably.
My biggest mistake, as I see it, was failing to generalize from "I have several max-of-two-formulas and a Dragonslayer distribution that is obviously a mix of two which I can't seem to resolve into something nice, probably all of them are max-of-two-formulas, let's see if I can refactor things to look like that and get a better idea of what Dragonslayer's two might look like if the rest do factor well".
I liked this one a lot. I imagined that 'train a linear classifier' would be the next step, but didn't do it due to laziness: it looks like that would probably have worked.
I do feel like my approach should have worked worse than it did - I did most of my evaluation by ignoring the scores and looking only at your historical classifications, and the one place where I let scores guide me into overriding my initial model (moving Student P from Humblescrumble to Serpentyne) it turned out I was incorrect and the initial model would have scored better (oops).
Thank you for posting this. I didn't think this was too straightforward. Prior to reading the solution I actually thought it was one of the more difficult ones. Possibly because I focused on trying to copy the allocation helms early choices instead of on the ratings.
This is a followup to the D&D.Sci post I made ten days ago; if you haven’t already read it, you should do so now before spoiling yourself.
Here is the web interactive I built to let you evaluate your solution; below is an explanation of the rules used to generate the dataset (my full generation code is available here, in case you’re curious about details I omitted). You’ll probably want to test your answer before reading any further.
Ruleset
Stats
Each of a student’s five stats – Intellect, Integrity, Courage, Reflexes and Patience – is generated by rolling a d80 four times and picking the second-highest result. There is no correlation between stats, no censorship, and no upper or lower limit on what qualities a student can have.
Ratings, Potential, and Archetypes
The Ofstev Rating for a given student is given by rolling [Potential] four-sided dice, and counting the number of times you roll a four; this can be closely approximated as Poisson([Potential]/4).
Each House has two Archetypes associated with it, which determine how much Potential a student will have if they’re Allocated there. Students in a House always come to embody the Archetype which would grant them more Potential.
Dragonslayer Archetypes
Guardians require a high level of all five stats. Guardians have 5*(min([all stats])-1) Potential.
Warriors require Courage and Reflexes; more the former than the latter. A Warrior has 3*min(Courage-9, Reflexes+9) Potential.
Thought-Talon Archetypes
Innovators require a high level for all stats except Reflexes. Innovators have 5*(min([all stats except Reflexes])-3) Potential.
Scholars require Intellect and Patience; more the former than the latter. A Scholar has 3*min(Intellect-4, Patience+4) Potential.
Serpentyne Archetypes
Like Scholars, Schemers require Intellect and Patience; unlike Scholars, Patience is more often the limiting factor. A Schemer has 3*min(Patience-7, Intellect+7) Potential.
Duelists require Reflexes and Intellect; more the former than the latter. A Duelist has 3*min(Reflexes-8, Intellect+8) Potential.
Humblescrumble Archetypes
Organizers require Integrity and Intellect; more the former than the latter. An Organizer has 3*min(Integrity-6, Intellect+6) Potential.
Citizens don’t require anything much, but do benefit from Patience and/or Integrity. A Citizen has 35+max(Patience, Integrity) Potential
Allocations
When you were first wrought, you made perfect decisions 91% of the time, but glitched out and Allocated randomly for the other 9%. The frequency of glitches has steadily increased, at a rate which has itself steadily increased. Students (and their families) started picking up on this in 1980, and average class size has decreased linearly since then, though the average incoming student remains drawn from the same distribution.
Strategy
The Potential for the incoming class given specific Allocations looks like this (optimal choices highlighted):
Leaderboard & Commendations
(Let me know if I got your score wrong somehow and I'll edit this.)
Congratulations to new (?) player Thomas Sepulchre for taking the top spot. Congratulations also to gjm for reaching a good answer incredibly swiftly, to aphyer for qualitatively analyzing the nature of the Helm's mistakes, and to GuySrinivasan for his impressive if quixotic progress towards finding analytic solutions (I did not expect anyone to figure out that this scenario ran almost entirely on "min(x,y)", or to realize that Humblescrumble's treatment of Integrity and Patience was the exception to this rule).
Reflections
Unlike most of my D&D.Sci games, this scenario has no clever trickery, and can be solved with blunt application of ML: the main (intended) challenge for players was converting it into a machine learning problem, and the big twist was that there was no big twist. Pedagogically and philosophically, I wanted to acknowledge the occasions when selection biases and similar distortions don’t render automated solutions unworkable; personally and pragmatically, I felt the need to make at least one unusually usual entry before starting on the list of experimental, high-variance, possibly-a-waste-of-everyone’s-time-including-mine games I plan to run next year.
Despite my cornucopia of reasonable reasons, I may have taken such straightforwardness a shade too far. In the name of minimalism, I made Swineboils’ curriculum and amenities remain unchanged through the centuries, and let students be completely unaffected by their House-mates’ qualities and quantities. You may find it overly-convenient that an institution could be that static, or a population that standoffish: my sole excuse is that as a Harry Potter parody, this scenario is implicitly set in the UK. Feedback on this point, and on all other points, is greatly appreciated.