Tldr: some not-particularly-ordered thoughts about how learning works in humans, via the example of singing.

A high-tech feedback loop

I recently discovered Singer’sStudio, an iPhone app for voice training that is approximately the Best Thing Ever (h/t Raemon). It’s a work of pedagogical art, and describing what it does right pulls together various nebulous thoughts I’ve had about learning and developing intuitions for a particular domain.

How the app works: it gives me various singing exercises, and then tells me in real-time, via a pretty graph, if I’m singing on pitch. That’s it.

Okay, it does have a few other features, like:

• A total score and breakdown by % of notes correct after each exercise.
• Letting me play myself back to see what being on vs slightly off pitch sounds like.
• Helpful text prompts such as “keep your tongue behind your teeth.”
• A built-in progression from easier to more complicated exercises.
• Exercises for a specific skill, like switching from head to chest voice.
• The ability to add challenge by turning down/off the piano playing my notes as I sing them.
• Not a built-in part of the app, but I can glance away from the screen and sing without the visual feedback, and then immediately check how I did.

But I’m pretty confident that the pitch contour is the most important part, and an app that only did that thing would be almost as good on the skill-building side.

There are obviously a lot of singing sub-skills other than hitting notes, like vocal timbre, head versus chest voice, breathing from the diaphragm, enunciating consonants clearly, and singing vowels that don’t sound weird. A voice teacher would be able to comment on these directly.

Still, in practice the app ends up causing me to improve on a bunch of these sub-skills as well, to the extent that getting them wrong results in singing off-pitch. The instantaneous feedback is doing a lot of work here – if the app only gave me a score at the end of each exercise, I predict I would end up quite stuck on what to change up in order to do better. However, I’ve ended up e.g. remembering to breathe from my diaphragm, because not doing that reliably resulted in my voice being wobbly, which was obvious on my pitch contour. (It also has instructions for breathing exercises as part of each warm-up, but I only actually did those exercises once or twice; they are boring compared to singing and getting pretty graphs.)

It’s amazing how fast my brain learned to turn a smooth, stair-like pitch contour into dopamine hits, and feel a flicker of pain every time I jumped above or below the note before finding it. I end up very motivated to play around with any variables that make the graph prettier.

Rewardingness

It occurs to me that another feature, which I think is less key than the instantaneous pitch contour but still pretty important, is how it slightly gamifies the entire thing, and thus makes it addictive. There are % scores at the end of the exercise! And points! It logs my all-time high score for each exercise so I can try to beat it! It also logs how many minutes a day I’ve practiced. All of this makes me more likely to use it, and actually putting in the time is a key part of training any skill.

It’s also cool that I can listen to my voice and, in addition to catching mistakes (ouch!), notice when hey, wow, I actually sounded good there. This lets me gradually figure out what correlates with liking how my voice sounds, and it also gives me a warm glow of satisfaction and helps me feel like a Real Singer.

(I’m not sure if the app is cheating by doing some kind of post-processing on these recordings to make them sound pretty. Normally I hate recordings of myself speaking, let alone singing. Still, I’ll take it.)

It also seems relevant that there’s zero embarrassment factor – this isn’t a human watching me and judging me for daring to sing when I kind of suck. I’m pretty shameless, as humans go, but I’ve been slightly nervous with every voice teacher I’ve had – they’re an expert! they’re probably really unimpressed! – and tension is not good for singing well. With this, I can sing my heart out in the privacy of my apartment, and even feel safe experimenting.

How does this generalize?

This is a post about singing, but it’s also a post about learning skills in general.

Learning to sing (or to play the piano, tie one’s shoes, draw, dance, swim; all the things commonly known as procedural memory) isn’t like memorizing a list of dates for a history test. There are some steps that can usefully happen in the explicit verbal loop, like “remember to breathe from the diaphragm”, but the end goal is that basically nothing is being held in working memory, and everything happens on the level of microsecond-to-second intuitions and muscle memory.

I think there are a really large number of skills, physical and mental, that are somewhere on a gradient between this and explicit memorization. Chess seems like a good example; it’s possible to play it from knowledge of the rules and explicit reasoning/strategizing, and there’s always some of this, but analyzing the entire game explicitly is intractable, and from what I’ve heard, experts heavily rely on intuition. ICU nursing and running events, both things I have more personal inside knowledge of, are maybe a bit closer to the intuitive or procedural end; the time pressure and urgency, and the sheer amount of stuff going on, makes explicitly reasoning through each decision less useful. Something like accounting is further towards the explicit end, but even that gets a lot more automatic with practice, and intuition is what makes something jump out as “wrong” to me.

Practicing a procedural skill without the benefit of a clever app involves two steps – trying it out (controlling my vocal chords to sing a set of notes), and being able to evaluate how good or bad a given attempt was (my “ear”). Becoming an expert involves improving both of these functions; at any given moment, in order for an instance of practicing to lead in the direction of progress, the evaluation process needs to be more accurate than the execution process.

I’ve heard anecdotes about the phenomenon of having discernment for “good X” that far exceeds one’s ability to perform X, and I’ve occasionally experienced this for other things, like writing. Therein lies frustration and embarrassment and self-judgement. In my experience, this often causes people to avoid practicing a skill, even though really it’s a good thing if your taste for X is refined enough to notice your own mistakes – that's how you know what to do differently the next time!

With singing, past a certain level I struggle to improve my ear enough to notice mistakes, which limits how much progress I can make just singing by myself at home. Voice teachers and other sources of feedback can help, but this is slower and thus less useful for shaping, in the same way that giving a dog a treat with a 5-min delay is going to be much less effective for training purposes.

This app creates a feedback loop on the sub-second level, using a different channel (my visual cortex) that doesn’t interfere with singing or listening, and which is a lot more accurate than my own internal sense for whether I’m on pitch. My hypothesis is that some lucky people do have this sort of high-quality discernment for pitch already developed in their brains, or develop it young, and they’re the ones who learn how to sing without much effort – and, because singing badly is tied up in embarrassment and shame, they’re probably the people who tend to become excellent singers at all. I expect most people with my level of innate talent, or lack thereof, just give up.

The app also lets me test and calibrate my sense of pitch more directly, so that maybe, someday, I’ll be able to tell in real-time, on my own, if I’m landing the notes in a song. (Currently I cheat and use VocalPitchMonitor for this, since SingerStudio isn't quite cool enough to let me upload random sheet music and use that to get feedback on real songs.) I play myself back and stare at the pitch contour, and try to hear the slight wrongness when the line indicates I’m just barely sharp or flat; it’s better than previous “ear training” I’ve done with a teacher or out of a book, and I’m hopeful.

Implications

This app is a really cool category of thing, that’s only possible at all due to fairly recent technological advances, and there are probably a ton more instances that I don’t know about.

I’m curious where else this has been explored. Singing may be an easy case, because measuring a single straightforward variable, pitch, gets you so far. I can imagine an app that trains, say, krav maga fighting techniques, via video analysis and/or accelerometer data, but I’m not sure that’s possible yet given current tech.

It has me thinking about other pedagogical techniques, though. Martial arts teachers will shout real-time feedback at you ("turn your hips more! get your knee higher!") I’ve taught swimming, and one issue is that waiting until a swimmer finishes a lap before giving any feedback introduces a huge delay, but grabbing onto them every time they do something slightly wrong is incredibly irritating and disruptive. Now I’m imagining giving them waterproof headphones and narrating the feedback in real time (“elbow higher please”, “roll your shoulder deeper into the water”, “keep your head back when you breathe”, etc etc.) This would be so cool.

My friend, when I brought this up, recommended I look up Tagteach. It seems to be largely based on clicker training, but lays out a bunch of thoughts I’ve had on pedagogy, like the importance of breaking a skill down into really small increments where success is easily measurable. Their site claims this protocol has been used in dance and sports coaching, but also business skills training and medical school; I’d be curious to know what this actually means in practice, but it does hint that it could be useful beyond purely physical skills (though presumably it requires success to be easily visible to the trainer; clicker-training accounting, or anything where a lot of the process is mental, seems hard!)

Of course, one of the awesome things about my app is that it removes the need for costly one-on-one time with an instructor. For swimming, could an accelerometer measure forward speed and narrate that? Speed relies on getting an absurdly large number of muscle movements just right, but the same thing is true of singing on pitch, and the single-input feedback seems to be enough to guide me towards progress. As a bonus, it could play generic reminders and prompts (I’m assuming the Singer’sStudio prompts aren’t responsive to specific mistakes I make, but they’re still useful.)

It also occurs to me that if getting direct and immediate feedback on your evaluation process is key to improving it, it could be useful to focus on that directly, separately from executing the skill itself – if you’re better at catching your own mistakes, later practice will be more valuable. I’m imagining a gymnast watching videos of Olympic gymnastics that have real-time commentary analyzing what the athletes are doing, noting successes and mistakes.

I would be curious to hear any examples others have of this.