Triggered by: Future Story Status
A helpful key to understanding the art and technique of character in storytelling, is to consider the folk-psychological notion from Internal Family Systems of people being composed of different 'parts' embodying different drives or goals. A shallow character is a character with only one 'part'.
A good rule of thumb is that to create a 3D character, that person must contain at least two different 2D characters who come into conflict. Contrary to the first thought that crosses your mind, three-dimensional good people are constructed by combining at least two different good people with two different ideals, not by combining a good person and a bad person. Deep sympathetic characters have two sympathetic parts in conflict, not a sympathetic part in conflict with an unsympathetic part. Deep smart characters are created by combining at least two different people who are geniuses.
E.g. HPMOR!Hermione contains both a sensible young girl who tries to keep herself and her friends out of trouble, and a starry-eyed heroine, neither of whom are stupid. (Actually, since HPMOR!Hermione is also the one character who I created as close to her canon self as I could manage - she didn't *need* upgrading - I should credit this one to J. K. Rowling.) (Admittedly, I didn't actually follow that rule deliberately to construct Methods, I figured it out afterward when everyone was praising the characterization and I was like, "Wait, people are calling me a character author now? What the hell did I just do right?")
If instead you try to construct a genius character by having an emotionally impoverished 'genius' part in conflict with a warm nongenius part... ugh. Cliche. Don't write the first thing that pops into your head from watching Star Trek. This is not how real geniuses work. HPMOR!Harry, the primary protagonist, contains so many different people he has to give them names, and none of them are stupid, nor does any one of them contain his emotions set aside in a neat jar; they contain different mixtures of emotions and ideals. Combining two cliche characters won't be enough to build a deep character. Combining two different realistic people in that character's situation works much better. Two is not a limit, it's a minimum, but everyone involved still has to be recognizably the same person when combined.
Closely related is Orson Scott Card's observation that a conflict between Good and Evil can be interesting, but it's often not half as interesting as a conflict between Good and Good. All standard rules about cliches still apply, and a conflict between good and good which you've previously read about and to which the reader can already guess your correct approved answer, cannot carry the story. A good rule of thumb is that if you have a conflict between good and good which you feel unsure about yourself, or which you can remember feeling unsure about, or you're not sure where exactly to draw the line, you can build a story around it. I consider the most successful moral conflict in HPMOR to be the argument between Harry and Dumbledore in Ch. 77 because it almost perfectly divided the readers on who was in the right *and* about whose side the author was taking. (*This* was done by deliberately following Orson Scott Card's rule, not by accident. Likewise _Three Worlds Collide_, though it was only afterward that I realized how much of the praise for that story, which I hadn't dreamed would be considered literarily meritful by serious SF writers, stemmed from the sheer rarity of stories built around genuinely open moral arguments. Orson Scott Card: "Propaganda only works when the reader feels like you've been absolutely fair to other side", and writing about a moral dilemma where *you're* still trying to figure out the answer is an excellent way to achieve this.)
Character shallowness can be a symptom of moral shallowness if it reflects a conflict between Good and Evil drawn along lines too clear to bring two good parts of a good character into conflict. This is why it would've been hard for Lord of the Rings to contain conflicted characters without becoming an entirely different story, though as Robin Hanson has just remarked, LotR is a Mileu story, not a Character story. Conflicts between evil and evil are even shallower than conflicts between good and evil, which is why what passes for 'maturity' in some literature is so uninteresting. There's nothing to choose there, no decision to await with bated breath, just an author showing off their disillusionment as a claim of sophistication.
They're hard to pin down, and different people I know have different explanations.
The one in my head is basically that they pay too much attention to theme and perspective; while in many cases litfic is directly about perspectives (themes), lots of people tend to be reductio ad absurdums of this, focusing on these things in rather simplistic ways that sometimes ignore how the world works or the basic potentially interesting things in the setting**. This is made worse because it's less obvious to the unpractised eye by the very nature of what's being tackled what the difference between Nabokov and McEwan is than is that between Arthur C Clarke and a generic bad SF writer; and by the fact that the average litfic writer has been through a professional course in writing and therefore sounds very polished.
Here's China Mieville's explanation, since you shouldn't be limited by my account found in the Guardian (it's not a coincidence that he chose Saturday too, it's partly that it's too good an example and partly that he put it in my head back when I read this piece):
"Literary fiction of that ilk – insular, socially and psychologically hermetic, neurotically backslapping and self-congratulatory about a certain milieu, disaggregated from any estrangement or rubbing of aesthetics against the grain – is in poor shape." Miéville identifies Ian McEwan's Saturday, set around the 2003 demonstration against the Iraq war, as a "paradigmatic moment in the social crisis of litfic".
"In the early 2000s there was this incredible efflorescence of anger and excitement . . . It seemed to me that Saturday quite bolshily said, 'OK, you accuse us of a neurotic obsession with insularity and a certain milieu. I'm going to take the most extraordinary political event that has happened in Britain for however many years and I am going to doggedly interiorise it and depoliticise it with a certain type of limpid prose . . . It was a combative novel that met that sense of there being a crisis and de-crisised it through its absolute fidelity to a set of generic tropes."
*Another particularly appropriate example: Cormac McCarthy's The Road is a post-apocalyptic novel involving, among other things, cannibals and an earth that can't grow food that, at page 300, suddenly reveals that it's about A Father's Concerns About Setting His Child Free and nothing else*. I'm sympathetic to the theme, but not when it funges on everything else potentially interesting about the story.
Edit: I consider China Mieville more able to answer this question properly than Eliezer because he has read a lot of litfic and incorporates techniques from that side into his writing.
Also, I just realised that this whole thread must have been a bit frustrating to you because of my laziness. Sorry about that.
I have heard that the decline in the compelling qualities of literary fiction is due to classes in writing taught by literature professors, who know how to identify things like themes but who have no idea how to write compelling writing. Does this seem like a plausible statement to you?