= 27d567bc2d48d9f40e9ccc20ea370b15)
Kawoomba comments on PSA: Eugine_Nier evading ban? - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (68)
Friend of mine did it via computational complexity: using gzip (as an approximation for KC) for attributing classical latin literature to their respective authors by checking which strings add the least additional complexity (due to shared writing styles, word choice, etc.) when compressed together and then clustering. Worked like a charm.
ETA: These were large bodies of text, however. Probably not gonna work for a bundle of comments, except for me, due to my overuse of "obviously", obviously.
I thought this was pretty impressive:
[...]
[...]
Difference was one of scale. Much easier when just taking three dozen? pieces of classical latin literature, some of which were different parts of the same opus magnum, then see them cluster to their respective authors and to the other parts of the same piece. More of a "put the pieces into the box" as opposed to a 100,000 pieces puzzle. In the latter case, you just know most of the puzzle pieces will either show the blue sky, or the blue sea, both a similar shade of blue.
Commenting to 'save' this comment. That's a really clever way to handle that.
KC = Kolmogorov Complexity?
Yeap, the paper linked in my other comment explains how it works.
gzip is a "crude upper bound" on KC, and afaik there is no known bound on the error.
EDIT: I'm pretty sure the following result isn't even true: If KC(x) < KC(y), then gzip(x) < gzip(y). Or even quantitatively weaker variants of the same.
There can't be, since KC isn't computable (could be mistaken on that in itself precluding a (edit:) lower error bound). It's still kinda nice and works on a similar principle (and also, somewhat, in practice). Let's plug the Hutter prize while we're at it, for the 5 people reading this. Also, just saw a cool paper which kinda describes the same principle, here.
KC may not be uncomputable in general, but I'm pretty sure that doesn't preclude all possible proofs or constructions*, and it seems odd to say that there is no upper bounds when we have what sure look like such things.
* I have just invented a Scheme-like programming language in which all tokens except numbers are 2 alphanumeric characters long; what is the Kolmogorov complexity of the bitstring '1'? 'Who knows, KC is uncomputable -' Bzzt! The answer is that the KC is exactly 1, since the shortest program which emits the bitstring '1' is a program consisting of the constant '1' which evaluates to '1', which as you can see, is indeed of length 1, and all other programs emitting the bitstring '1' are longer by definition.
Or if you don't like that example, consider taking your favorite language and enumerating all possible programs in length order; consider the outputs of the programs you've just enumerated when evaluated up to the highest available Busy Beaver in steps (to avoid nontermination issues), and the respective lengths of outputs - have you not found the KC for those outputs? If you run gzip over the outputs to get estimated KCs, are those not upper bounds on the actual KCs you proved?
Computing upper bounds on on Kolmogorov Complexity is not very difficult: gzip and all the other compression algorithms do it. The difficulty is computing non-trivial lower bounds:
For all programming languages (with a self-terminating encoding), there is a trivial lower bound that doesn't depend on the string. This bound is at least one token.
But there is also a language-dependent constant L_max which is the maximum KC complexity and lower bound on KC complexity that you can compute for any string: L_max is the length of the shortest program for which the halting property is uncomputable ( * ) (which makes L_max is uncomputable as well).
This implies that you can compute the KC complexity only for a finite number of strings.
( * And doesn't provably emit any token)
There is no such thing as "the shortest program for which the halting property is uncomputable". That property is computable trivially for every possible program. What's uncomputable is always the halting problem for an infinity of programs using one common algorithm.
It is also easy to make up artificial languages in which Kolmogorov complexity is computable for an infinite subset of all possible strings.
You were probably thinking of something else: that there exists a constant L, which depends on the language and a proof system T, such that it's not possible to prove in T that any string has Kolmogorov complexity larger than L. That is true. In particular, this means that there's a limit to lower bounds we can establish, although we don't know what that limit is.
The halting property is semi-decidable: if a program halts, then you can always trivially prove that it halts, you just need run it. If a program does not halt, then sometimes you can prove that it doesn't halt, and sometimes you can't prove anything.
For any programming language, there exist a length such that you can compute the halting property for all programs shorter than that. At length L_max, there will be program Bad_0, which:
You can never prove that any string S has Kolmogorov complexity K if K > L_max, as it would imply that you proved that Bad_0 doesn't halt, or at least doesn't emit any symbol which is not a prefix of S.
Since there are only finitely many strings with complexity up to L_max, we can only compute Kolmogorov complexity, for finitely many strings.
If the language is Turing-complete I don't think this is possible. If you think that an arbitrary string S has complexity K, how can you prove that there exist no program shorter than K that computes S?
[retracted]
Yea, only talking about the general case. Even the Halting Problem is usually computable in daily life, just by running the program (ETA: and seeing it terminate).
Watch the program run
See it die before your eyes
Halting Problem? Solved!
Exactly. So why bother saying gzip is an approximation of KC? (I assume: because KC is a well-known theoretical object with good properties, and one shouldn't let the fact that none of these properties carry over to gzip ruin one's chance of getting published cheaply.)
Because gzip is being used to measure complexity. That's literally the reason they used gzip and not, I don't know, rot13. It's an explanation of the causal role that gzip is playing in the whole process.
No.
"gzip is being used to measure complexity" is an explanation of the causal role that gzip is playing in this study.
"gzip is an approximation of KC" is either 1) flatly not true, see edit to grandparent or 2) not relevant to the study at all.
And while we're at it, why do we even care about Turing Machines, it's not like we could ever build one anyways. ;-)
goes back to tending his potato garden