Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Compression progress, social niches, and culture

4 [deleted] 19 January 2012 04:11PM

I'm probably succumbing to "guy with a hammer" syndrome in a big way, but here goes...

From Jürgen Schmidhuber's "Compression Progress" talk:

[How] should an unsupervised intelligent agent... deal with data that is streaming in through the input centers in response to the actions that it's executing?


You have to find regularities in this history of inputs and actions that you store and... compress it.


So... what is the interestingness of some data X?  Well, it's not the number of bits that you need to encode the data.  It's the first derivative, the change of the number of bits as your subjective learning algorithm based on your subjective previous knowledge is improving the compressibility.

If I understand him correctly, Schmidhuber is saying that interestingness is your brain rewarding you for giving it new data that allows it to function more efficiently by encoding fewer bits.  This he calls compression progress.

Why should this process be confined to visual and auditory patterns?  Shouldn't it also apply to improvements in the compressibility of beneficial social data?  Furthermore, shouldn't your brain have to constantly negotiate between rewards for novel, compressible social patterns and rewards for novel visual and auditory patterns?  With social encoding occasionally -- or mostly -- winning?

Seen this way, culture looks like iterated niche-forming.  Imagine a hunter-gatherer band with synchronized aesthetic values.  Everybody follows the beat of the stone on the hollow log with nary an iconoclastic thought.  Then the tribe gets bigger.  Change in scale weakens synchronization.  There is suddenly more variation in ideas.  Subgroups form that may value intentional suppression of their former values, opening up social niches where the ability to notice small differences wins out over innate reactions.  Basically, if difference-noticing can create a status niche, it will.

In other words -- and I'm only half-joking -- compression progress explains hipsters.

Comments (15)

Comment author: brilee 20 January 2012 08:18:13AM 2 points [-]

I've thought along these lines for a while too, although I can't help but feel that this might be another "just-so" ev-psych story.

I don't see it so much as a compression scheme as a refined-model-of-the-world. I see it more as a lossy vs. lossless compression improvement. For example, as we learn more, we create more, not fewer categories. (For example, once you learn the resolution to the tree falls in a forest story, you now have refined sound into "sound-1" and "sound-2")

This neatly explain why we find irony so delicious - it presents to us directly a contradiction in our categorization scheme, and an opportunity for improving our model of the world.

Comment author: sark 15 April 2012 06:48:42PM 1 point [-]

Hi, thanks for linking to your post here. It seems relevant to what I tweeted. But please help me understand what you are saying here. I think I'm having trouble at "Subgroups form that may value intentional suppression of their former values". Why would they value suppression of former values?

I'm guessing you're trying to say that subgroups will find their aesthetic more interesting because they experience their aesthetic as providing greater improvement in compressibility given preexisting inculcation in that aesthetic?

Comment author: [deleted] 04 July 2012 08:06:05AM *  2 points [-]

Sorry, didn't see you commented until just now.

Why would they value suppression of former values?

Because a new niche -- even one based around subcultural ephemera -- means first dibs on resources for its members (especially the high status ones). This iteration increases as groups move away from subsistence level. If being contrary has no life-threatening consequences, what is to stop contrarians from acting like superstimuli pumps that spoof compression progress?

Comment author: sark 04 July 2012 12:58:06PM 1 point [-]

Ok got it, thanks.

Comment author: gwern 19 January 2012 05:09:52PM 1 point [-]

Perhaps it also explains language diversity?

Comment author: timtyler 19 January 2012 09:34:16PM 0 points [-]

That is surely more down to drift - and xenophobia.

Comment author: David_Gerard 19 January 2012 10:52:08PM *  2 points [-]

Dick Hebdige's book (a standard sociology text) Subculture: The Meaning Of Style puts it down to something like that: the essence of a subculture is an awareness of style and differences in style. This obviously expands to fine linguistic distinctions between dialects and so on. (e.g. the language spoken across much of the former Yugoslavia which now has four or five names.)

Comment author: lessdazed 23 January 2012 11:47:47AM 0 points [-]


What do you mean? When in history are you referring to?

Comment author: timtyler 23 January 2012 01:44:17PM *  0 points [-]

The period during which language evolution happened.

Xenophobia acted to help prevent cultures from combining, probably resulting in increased cultural diversity.

Comment author: Armok_GoB 04 February 2012 12:54:17PM 0 points [-]

Upvoted for being subtly self demonstrating and hilarious, because of each other.

Comment author: [deleted] 01 March 2012 07:31:55AM 0 points [-]


Comment author: Armok_GoB 01 March 2012 06:04:23PM 1 point [-]

Given that it is hilarious, and that hilarity is because of exactly the kind of compression progress discussed, it is self demonstrating.

I find this kind of metaness and self demonstration hilarious, because it in a more compressed manner helps compression progress, given this theory.

Comment author: [deleted] 01 March 2012 06:11:08PM 0 points [-]

I flat out don't understand. Can you try again, but more concretely? With example(s)?

Comment author: Armok_GoB 01 March 2012 09:15:24PM 0 points [-]

I don't think so. It's already about a specific example - your article - and entirely abstract and subtle.

Comment author: [deleted] 01 March 2012 09:21:20PM 0 points [-]

Can you at least tell me the line that you found funny? And maybe what about it is compressed well (or poorly)?