I think this is plausibly the best scenario either of us have made to date.
The basic game was very good, layering simple rules on top of each other to create a complex system which was challenging to detangle but easy to understand once you know the rules. I was particularly impressed by the fact that you managed the (imo, near-impossible) feat of making an enjoyable D&D.Sci where handling data starvation was a key part of the problem: most players (including me) seem to have had the (sensible) thought "okay, let's filter for only potions with Onyx and Bone", and success past that point was predicated on realizing there weren't quite enough rows to justify being that picky.
The twist struck me as fair, funny and fun. It provided an object lesson in noticing when things don't quite add up, and letting the qualitative shade the quantitative; it also expanded the scope of the genre in ways I realize I've been neglecting to.
All that said, I have some (minor, petty) criticisms . . . not of the game itself, but how it was presented. Namely:
.This entry was billed as "relatively simple", but I think it was about median difficulty by the standards of D&D.Sci; pretty sure it was harder than (for example) The Sorceror's Personal Shopper.
."STORY (skippable)" was kind of misleading this time: the flavortext had a lot of little hints that the Archmage wasn't on the level, so someone who didn't read it (or failed to read between the lines, like me) would be at a (small) disadvantage.
."Archmage Anachronos is trying to brew Barkskin Potion" was A) the GM saying something false directly to the players, and B) a missed opportunity: if you'd written something like "Your goal is to help Archmage Anachronos brew Barkskin Potion", that would have been a subtle confirmation that giving him exactly what he asked for would lead to the best outcome (vs more aggressive / galaxy-brained forms of sabotage, or refusing to cast judgement on his pursuit of immortality, or any other reaction).
I'm glad you liked it, thank you!
This entry was billed as "relatively simple", but I think it was about median difficulty by the standards of D&D.Sci; pretty sure it was harder than (for example) The Sorceror's Personal Shopper.
I guess that's fair. There's a complication here in that...uh...almost all of my scenarios have been above-median complexity and almost all of yours have been below-median. (I should probably write down my thoughts on this at some point). I agree that this one wasn't simpler than most of yours, but I think that it was still a much more approachable entry point than e.g. Duels & D.Sci, or League of Defenders.
(It's possible we should try to standardize a 1-10 complexity scale or some such so that we can stick a difficulty rating on the top of each scenario.)
"STORY (skippable)" was kind of misleading this time
Fair enough, I can tweak that for anyone who finds the scenario in future.
I intended that the story should not provide much help...the intent was not for players to notice that Anachronos was suspicious in-story, the intent was for them to notice from the data, and for the hints in the story to be just some quiet confirmation for a player who realized the twist from the data and then went back to reread the story.
On the other hand, I was expecting more players to get the twist, and thought that I'd only really catch players who ignored the ingredient names entirely and just fed the data into an ML algorithm, so I'm clearly not very well calibrated on this. I was really quite surprised by how many players analyzed the data well enough to say "Barkskin potion requires Crushed Onyx and Ground Bone, Necromantic Power Potion requires Beech Bark and Oaken Twigs" and then went on to say "this sounds reasonable, I have no further questions." (Maybe the onyx-necromancy connection is more D&D lore than most players knew? But I thought that the bone-necromancy and bark-barkskin connections would be obvious even without that).
"Archmage Anachronos is trying to brew Barkskin Potion" was A) the GM saying something false directly to the players
I...think I'm in general allowed to say false things directly to the players as a D&D GM? If the Big Bad is disguised as your innkeeper while the real innkeeper is tied up in the cellar, I think I can say 'The innkeeper tells you it'll be six silver for a room', I don't think I need to say 'The man who introduced himself to you as the innkeeper.'
(Also, you are a Data Scientist. Sense Motive is not a class skill for you. Clearly you failed a Sense Motive check and so believed him!)
...I'll think about whether I want to tweak that line for potential future players.
I thought the flavour text was just right - I got it from the data, not the flavour text, and saw the flavour text as confirmation, as you intended.
I was really quite surprised by how many players analyzed the data well enough to say "Barkskin potion requires Crushed Onyx and Ground Bone, Necromantic Power Potion requires Beech Bark and Oaken Twigs" and then went on to say "this sounds reasonable, I have no further questions." (Maybe the onyx-necromancy connection is more D&D lore than most players knew? But I thought that the bone-necromancy and bark-barkskin connections would be obvious even without that).
Illusion of transparency I think, hints are harder than anyone making them thinks.
When I looked at the ingredients for a "barkskin potion", as far as I knew at this point the ingredients were arbitrary, so in fact I don't recall finding it suspicious at all. Then later I remember looking at the ingredients for a "necromantic power potion" and thinking something like... "uh... maybe wood stuff is used for wands or something to do necromancy?". It was only when I explicitly made a list of the ingredients for each potion type, rather than looking at each potion individually, and could see that everything else make sense, that I realized the twist.
If the Big Bad is disguised as your innkeeper while the real innkeeper is tied up in the cellar, I think I can say 'The innkeeper tells you it'll be six silver for a room', I don't think I need to say 'The man who introduced himself to you as the innkeeper.'
Perhaps, but you could also simply say "Yeah, the guy at the counter tells you the room will be 6 silver."
...bearing this out, it looks like Lorxus managed to get a perfect score with relatively little actual Data Science just by thinking about what it might mean that including lots of ingredients led to Magical Explosions and including few ingredients led to Inert Glop.
Not quite true! That's where I started to break through, but after that I noticed the Mutagenic Ooze issue as well. It also took me a lot of very careful graceful use of pivot tables. Gods beyond, that table chugged. (And if I can pull the same Truth from the void with less powerful tools, should that not mark me as more powerous in the Art? :P)
I guess I'm not clear on what "actual Data Science" would involve, if not making hypotheses and then conducting observational-experiments? I figured out the MO mechanic specifically by looking at brews that coded for pairs of potions, for the major example. The only thing that would have changed if I'd known SQL would be speed, I suspect.
...and documented his thought process very well, thank you Lorxus!
Always a pleasure! I had a lot of fun with this one. I was a little annoyed by the undeclared bonus objective - I would have wanted any indication at all in the problem statement that anything was not as it appeared. I did notice the correspondence in (i.a.) the Farsight Potion but in the absence of any reason to suspect that the names were anything but fluff, I abstracted away anything past the ingredients being a set of names. Maybe be minimally more obvious? At any rate I'd be happy to be just as detailed in future, if that's something you want.
1I liked the bonus objective myself, but maybe I'm biased about that...
As a someone who is also not a "data scientist" (but just plays one on lesswrong), I also don't know what exactly actual "data science" is, but I guess it's likely intended to mean using more advanced techniques?
(And if I can pull the same Truth from the void with less powerful tools, should that not mark me as more powerous in the Art? :P)
Perhaps, but don't make a virtue of not using the more powerful tools, the objective is to find the truth, not to find it with handicaps...
Speaking of which one thing that could help making things easier is aggregating data, eliminating information you think is irrelevant. For example, in this case, I assumed early on (without actually checking) that timing would likely be irrelevant, so aggregated data for ingredient combinations. As in, each tried ingredient combination gets only one row, with the numbers of different outcomes listed. You can do this by assigning a unique identifier to each ingredient combination (in this case you can just concatenate over the ingredient list), then counting the results for the different unique identifiers. Countifs has poor performance for large data sets, but you can sort using the identifiers then make a column that adds up the number of rows (or, the number of rows with a particular outcome) since the last change in the identifier, and then filter the rows for the last row before the change in the identifier (be wary of off-by-one errors). Then copy the result (values only) to a new sheet.
This also reduces the number of rows, though not enormously in this case.
Of course, in this case, it turns out that timing was relevant, not for outcomes but only for the ingredient selection (so I would have had to reconsider this assumption to figure out the ingredient selection).
Perhaps, but don't make a virtue of not using the more powerful tools, the objective is to find the truth, not to find it with handicaps...
I'm obviously seeking out more powerful tools, too - I just haven't got them yet. I don't think it's intrinsically good to stick to less powerful tools, but I do think that it's intrinsically good to be able to fall back to those tools if you can still win.
And when I need to go out and find truth for real, I don't deny myself tools, and I rarely go it alone. But this is not that.
You don't need to justify - hail fellow D&Dsci player, I appreciate your competition and detailed writeup of your results, and I hope to see you in the next d&dsci!
1
1...Lorxus managed to get a perfect score with relatively little in the way of complicated methods/tools...
I have struckthrough part of the previous comment, given the edit. I need no longer stand by it as a complaint.
This is a follow-up to last week's D&D.Sci scenario: if you intend to play that, and haven't done so yet, you should do so now before spoiling yourself.
There is a web interactive here you can use to test your answer, and generation code available here if you're interested, or you can read on for the ruleset and scores.
RULESET
There are two steps to brewing a potion:
STEP 1: MAGICAL POTENCY
Any ingredient that doesn't exist in the mundane world is Magical, while any ingredient that exists in the mundane world is not:
The first step of potion-brewing is to dissolve the magical potency out of the Magical Ingredients to empower your potion. This requires the right amount of Magical Ingredients: too few, and nothing magical will happen and you will produce Inert Glop, while too many and there will be an uncontrolled Magical Explosion.
If you include:
If your potion got past this step OK, move on to:
STEP 2: DIRECTION
Some ingredients are used to direct the magical power into the desired resulting potion. Each potion has two required Key Ingredients, both of which must be included to make it:
*Well. Sort of. See the Bonus Objective section below.
Some ingredients (Angel Feather, Crushed Sapphire, Faerie Tears and Quicksilver) aren't Key Ingredients for any potion in the dataset. Angel Feather and Faerie Tears are nevertheless useful - as magical ingredients that don't risk creating any clashing potion, they're good ways to add magical potential to a recipe. Crushed Sapphire and Quicksilver have no effect, including them is entirely wasteful.
If you've gotten through Step 1, the outcome depends on how many potions you've included both the Key Ingredients of:
So, for example, if you brew a potion with:
STRATEGY
For best success, you needed to:
This was made trickier by the fact that you had only 4 Magical Ingredients available (Demon Claw, Giant's Toe, Troll Blood and Vampire Fang), and that Troll Blood and Vampire Fang were the Key Ingredients of a Regeneration Potion (and hence you could not include both of them).
Nevertheless, you could get a 100% success rate by:
BONUS OBJECTIVE
There was a secret bonus objective hidden in the dataset. It could not be found purely with math on the dataset, it required you to think about the ingredient names and the patterns you were seeing.
On examination of the Key Ingredients of each potion, most potions had Key Ingredients that seemed in some way vaguely metaphorically linked to the potion's effect:
There were two apparent exceptions to this, however.
This was not, in fact, true.
The Key Ingredients of a Barkskin Potion are in fact Beech Bark and Oaken Twigs, and the Key Ingredients of a Necromantic Power Potion are Crushed Onyx and Ground Bone.
Archmage Anachronos is not trying to brew a Barkskin Potion. He is trying to brew a Necromantic Power Potion in order to conduct a Dread Necromantic Ritual during the upcoming magical conjunction.
Not wanting to reveal this, he's swapped around 'Barkskin Potion' and 'Necromantic Power Potion' in his records of results, telling you that every recipe that actually produced a Barkskin Potion produced a Necromantic Power Potion and vice versa.
simon realized this, I don't think anyone else figured it out independently (as opposed to seeing his comment mentioning it).
DATASET GENERATION
Archmage Anachronos's algorithm for deciding which ingredients to use is as follows:
The main relevance of this algorithm to players was that some combinations of ingredients, especially ones that could produce potions and especially ones that could produce multiple different potions, had been brewed many more times than would happen under pure randomness.
LEADERBOARD
This scenario was intended to be relatively straightforward, and bearing this out we had several perfect scores:
Congratulations to all players! Particular congratulations to simon for figuring out the Bonus Objective! Maniacal cackling to all other players!
REFLECTION & FEEDBACK REQUEST
The goal I was shooting for with this scenario was to reward people for actually paying human attention to patterns, rather than just hoping for [YOUR FAVORITE ALGORITHM HERE] to solve all your problems for you.
The Bonus Objective was obviously written to require out-of-context thought. I also tried to set up the scenario to be amenable to thinking about what patterns might mean even with relatively little math: bearing this out, it looks like Lorxus managed to get a perfect score with relatively little in the way of complicated methods/tools just by e.g. thinking about what it might mean that including lots of ingredients led to Magical Explosions and including few ingredients led to Inert Glop (and documented his thought process very well, thank you Lorxus!).
How did this feel from the player end?
As usual, I'm also interested to hear more general feedback on what people thought of this scenario. If you played it, what did you like and what did you not like? If you might have played it but decided not to, what drove you away? What would you like to see more of/less of in future? Do you think the scenario was more complicated than you would have liked? Or too simple to have anything interesting/realistic to uncover? Or both at once? Do you have any other feedback?