This post is intended to be a guide on the meaning of and distinction between several related concepts. What makes them related is that they're all pretty abstracted; they all feel sort of "up there". This also makes them unusually easy to confuse.

These ideas are fairly important if you're going to try to understand similarly "up there" domains like mathematics, computer science, or AI. But they're also just really fun! Another thing that unites them is that uses of them feel somehow tricky or clever. Sometimes it's for play and puzzles, and sometimes it's the crux of an astounding theorem.

I'm not a linguistic prescriptivist, so I'm not intending to declare that these words should mean exactly these things for the rest of time. But I am something of a conceptual prescriptivist, in the sense that I think reality has joints to be carved; I think all of the below words refer to concepts that are clearly distinct and useful, and deserve their own words.

I hope this guide is helpful in clarifying any latent confusions that people may have had!

Summary

Meta

The word "meta" has a somewhat strange and meandering history. It started out as a technical term, but has become somewhat common in everyday use.^[1] It originally was (and still is) a greek prefix meaning something like "after", and there happened to be a chapter of a book of Aristotle named "Metaphysics". This initially meant "the book after the book about physics", but people apparently mistakenly inferred the meaning of meta from the contents of the book instead. The fact that it now has other meanings in narrow contexts, paired with the fact that its most common meaning is somewhat esoteric, means that it's often used incorrectly.

In its modern usage, a meta-X is an X about X^[2]. A meta-book is a book about books. If you take the conversation meta, then you start talking about conversations.^[3] Metadata, like a timestamp or file size, is data about data. Metaprogramming is code that modifies other code (and is therefore programming about programming) and metamathematics proves theorems about whole axiomatic systems (and is thus math about math). Unknown unknowns are meta-epistemological.

It is the trendiness of this word that gave me the idea for this post; I have heard people use "meta" to mean every other word listed here! I personally care a lot about sorting concepts correctly, so years ago I installed a TAP where the trigger is "I hear or use the word meta" and the action is "ask myself if this is an X about X".

Self-reference

This one is really easy to remember because it's just two separate words whose meanings are clearer. X is self-referential if X refers to itself. (An obligatory example is that this sentence references itself, and so does this post.) However that simplicity belies its usefulness. Applying the concept has led to the discovery of many paradoxes which themselves have led to whole branches of mathematics.

Embedded agency is closely related to self-reference. AIXI has no concept of itself, and thus cannot entertain hypotheses or plans involving itself. Specifying intelligent systems which can conceptualize themselves is an active area of research.

Recursive

X is recursive if it somehow "uses" an instance of itself. The most frequent context in which I've heard this is recursive functions, which call [another instance of] themselves somewhere inside their function definition. For example, here's the classic recursive factorial function defined in python. Note the token factorial appearing  twice.

def factorial(n):
    if n <= 1:
    	return 1
    else:
        return n*factorial(n - 1)

Math also has recursive functions, where the value of a function for some inputs is defined in terms of the same function at other inputs. It is structures like these that give rise to proof by induction, which shares the mind-bendy nature of the other concepts in this post.

I think it would be fair to say that these functions are also self-referential, but they don't merely reference themselves, they in some sense actively use themselves, and they also rarely use the same instance of themselves; otherwise the recursion would not terminate, and there would be an infinite loop or cycle of some kind.

A recursively self-improving AI is an AI that uses itself to make another, improved instance of itself. The word "self" is there to distinguish it from AIs that recursively improve something else. If an AI uses a factory to manufacture better parts for some machines in the factory, then it is a recursively factory-improving AI.

Just about every modern machine learning system will use recursion heavily, such as Monte Carlo tree search and reinforcement learning.

Generalization

Concept X is a generalization of concept Y if X is just Y with some specific features removed. Often, as on wikipedia, definitions are given in so-called genus-species form. For example, "The horse is a domesticated, odd-toed, hoofed mammal". That is to say, a horse is a specific kind of mammal, namely a kind with hooves etc. If we remove those specifics, then we just end up with mammal. Thus, "mammal" is a generalization of "horse". You could also remove the "mammal" part and keep the "odd-toed" part; "odd-toed things" is another valid generalization of horses. The generalization "abstractions away" these specifics, and thus the word "abstraction" is often used synonamously.

In math, numbers represent a generalization of things with that quantity; the number 4 is a generalization over 4 apples, 4 letters, 4 seconds, etc. Algebra uses variables to generalize over specific numbers, and abstract algebra, well, abstracts over sets of stuff that obey certain axioms. Fields of mathematics are virtually defined by their place on a big hierarchy of generalizations, because if you have proved something using only a certain collection of properties (the specifics), then you have proved it about everything that has those properties.

In programming, the object-oriented concept of a class hierarchy most cleanly maps onto generalizations. For example, maybe one class Record defines an abstraction around database records in general, while a subclass User of it denotes a user record, which has the specific properties of an email and a password.

A lot of the field of AI has been about figuring out exactly how we can get machines to represent these hierarchies of concepts. Deep neural networks seem to be doing a pretty good job. That said, I don't know if ML researchers ever actually work with the definition I gave above, or if they just look at classification performance and think to themselves, "it sure does seem to have captured the general concept of a cat".

You could say that the difference between self-reference and meta is that meta references the genus of the self. A book about itself is a self-referencing book, but a book about books is a book about the genus of which itself is an instance.

Repetition

We're getting down to the crumbs now, but there are still simpler things that I've heard people use the word "meta" for. Repetition is basically what you're left with when a concept is used multiple times, but it's not any of the above concepts. "She sells sea shells by the sea shore" feels like it's doing something weird and clever, and that weird and clever thing is just repeating slightly shifted around phonemes.

Other-than-X

This is the most distant use of the word meta that I've noticed in the wild. People will sometimes be talking about a topic X, and then they'll start going on a tangent, and then they'll say something like, "okay this has gotten a little meta, let's get back to the main topic". Or maybe in a conversation they'll say "I have a meta question, where is the bathroom?" The question is not about questions, and it doesn't make the conversation be about conversations. It's just, a question that was not about what the conversation was about.

Blurry boundaries

The distinctions I've drawn between these concepts relies on english verbs like "uses", "references", and "is about"^[4]. Thus, there is a noticeable grey area.

Consider the look-and-say sequence. Start with 1. Then describe how many of each digit there are; one one. Then make that into digits; 11. Now there are two ones; 21. Now there is one two, one one; 1211. One one, one two, two ones; 111221. Et cetera.

Does each term in this sequence "refer" to the previous term, or does it "use" the previous term, or is it merely "repeating" the previous term with a modification? I wouldn't fret about it. If you're in a context where the distinction matters, then that context will likely provide a more precise definition to check against.

1. ^{^}

   In this it reminds me of "random", which colloquially often just means "arbitrary" or "unwanted" or "not appropriate to the context".

2. ^{^}

   The popularization of this meaning is credited to Douglas Hofstadter's Pulitzer-winning Gödel, Escher, Bach.

3. ^{^}

   Often people will "go meta" by talking about the specific conversation they're already having. In this case, they're also going self-referential, but fortunately it's also going meta, so it's not a terribly inaccurate usage.

4. ^{^}

   Is this a verb? I'm not really a grammar person. I don't think it's syntactically a verb, but it seems to have the same semantic role as the other verbs.