It seems to me that the mind is just a generator of random ideas based on things experienced recently, where the ideas are checked in various layers for how much sense they make and passed to the conscious mind when they have already passed some filters. In essence, our thinking process is a combination of semi-random idea generation and tweaking, combined with validation and testing.
There seems to be no reason why the same could not be implemented in a machine. People who argue that machines cannot do random stuff have apparently never dealt with cryptography.
What you're referring to is called a genetic algorithm. It has been implemented in a machine, and it can be rather impressive, but it's nothing compared to what humans can do. Whether this is because humans do something else, they have a whole lot of heuristics to make it work better, or what is unknown.
I don't think that you'll be able to develop a wholly deterministic machine that will surpass a human mind though.It's not clear that human minds aren't wholly deterministic, themselves.
denis, are you positing that pseudo-randomness is not enough, and you need true randomness? Perhaps you're even trying to claim that P does not equal BPP?
This is unimportant, but in the original human experience of milk, somewhat-spoiled milk was not in fact bad to drink. Old milk being actually rotten came as a surprise to my family when we moved to North America from Eastern Europe.
I'd like to bring up a wonderful field to keep in mind when you try to bring randomness and random solution generation into creative processes: genetic algorithms.
Genetic algorithms show two interesting things, relative to randomness. First, genetic algorithms are scored based on how much faster they are than generating random solutions and checking them. Second, you don't actually need randomness to implement genetic algorithms!
All you need to pull off a genetic algorithm -- or any similar search algorithm -- is /a means of generating solutions similar to your current best solution(s)/. This process does not have to be random! If your algorithm enumerated all of the neighboring solutions to a given solution, and tried them in turn, it would work just as well (more or less) as a genetic algorithm which mutated its solutions randomly.
I agree. Creativity is not just being random. The old masters used measurement and perspective when painting their masterpieces, they didn't just sit there and hum and at the sky and wait for inspiration to strike them.
I think the idea that creativity is somehow mystical comes from a religious model of the human body. If you think your body has causal flesh and a supernatural/acausal soul, and that creativity comes from your soul (the part that is "you") then it follows that creativity comes from the acausal.
"So do we reason that the most unexpected events, convey the most information, and hence the most surprising acts are those that give us a pleasant shock of creativity - the feeling of suddenly absorbing new information?"
This is very cool.
This post reminded me of the discussion of creativity in Carse's Finite and Infinite Games. He wrote that finite games are games (in a loose sense) with definite rules, with beginnings and ends, for which one can speak of preference orderings and optimizations. Infinite games have no end but may include finite games, and can be played but not won; they are played for the sake of playing. It makes no sense to talk about optimizing on an infinite game.
Modern art's surface-level boundary breaking can certainly be thought of as a winnable competition, but I'd be much more hesitant to say the same of all of human life, the higher-level creative process that gave rise to modern art and the problem of transportation. I'm not convinced that optimization captures all the important properties of intelligence (or creativity). Perhaps someone with a better understand could elucidate this for me?
It makes no sense to talk about optimizing on an infinite game.
If I get the difference between finite and infinite games... then I'm afraid I disagree.
Take, for example, the difference between "baseball" and "playing house".
Baseball is clearly a finite game - it makes sense to talk about a "winner" of baseball. Contrariwise, it makes no sense to talk about a "winner" of playing house - so I'd conclude that the latter is an infinite game.
From my own experience of "playing house" as a girl, I'd say there are definite candidates for optimisation - especially when playing with others. The most common (in my experience) being to optimise the average happiness level of each of the players (by sharing, avoiding or resolving disagreements etc).
Even if nobody "wins" you can still "play better" against this optimisation target.
As explained it doesn't fit the definition given: playing house does not have "definite rules" and does not have a defined beginning/end.
okay, playing house isn't actually a coherent category. there are ways to play house that have carse-jargon-“definite rules” and have a carse-jargon-defined-beginning&end, and there are ways to play house that don't. most instances of playing house are of the former type, likely including your experiences.
carse uses words in weird ways.
Paul: It makes no sense to think of life as optimization of a utility function over three-dimensional slices that stays the same over all time, since there's no end state we want and no fixed state we want the universe to be locked into; but we can talk about stable preferences over broad ranges of 3D states (e.g. 'freedom good'), changing preferences over detailed 3D states (e.g. 'I want to win this finite game'), or (most primitively, the way I like to think of it) stable (rather, timeless) preferences over detailed 4D histories of the universe.
The field of writing still has a lot of unnecessary rules. I was so glad when "ain't" became acceptable. Now I create words and break rules at will. I do take the position that not all creativity is art.
Modern Art cannot delight the untrained senses of a mere novice.
Whatever gets sophisticated critics to praise your rule-breaking is good Modern Art, and whatever fails in this end is poor Modern Art.
Both these statements are complete bullshit, as any visit to a modern art* gallery will confirm.
I think what you're trying to mock is conceptual art, a small sub-field of modern art, but your straw man bears so little resemblance to anything that actually happens in the art world that it's impossible to be sure.
*Scare caps are as bad as scare quotes.
I'm getting Deja Vu again. Are you recycling bits of older posts or other things you've written?
It's important to state that we do not see so much as recognize. Before data from the eyes is processed it is synthesized with memories, and so as much as 80% of what we 'see' is simply an exercise in matching visual data with memory data. This goes some way to explaining why being a tourist can be mentally exhausting; work cannot be outsourced to memory, but rather the mind works double-time to both take in full-bandwidth visual data and to store new information. Quite apart from this, basic values -- normally guarded closely by confirmation bias -- are being challenged. Travel broadens the mind in the same way that flooding broadens a river.
This tendency, to recognize rapidly, works against creative thought in everyday life. Creativity is bedeviled by routine, even routines that once successfully generated fresh concepts, and eventually elements of chaos are required to shake off the recognize-and-dismiss pattern that comes to us so easily.
It makes no sense to think of life as optimization of a utility function over three-dimensional slices that stays the same over all time, since there's no end state we want and no fixed state we want the universe to be locked into [...]
If that was true, it would represent a refutation of Dewar's maximium entropy principle - as applied to biology. Unfortunately, the logic doesn't seem to make any sense. Water can still act to maximise a utility function by flowing downhill - although it has no thought of the distant ocean.
If you are constantly surprised by solutions that are high in you preference ordering but low in your search ordering, that is a problem with your search ordering. If your search ordering is correct, creativity is useless.
If your search ordering is your preference ordering, either you're looking at a very simple problem or you have godlike intellect. In the first case, creativity is unneeded. In the second, no solution will appear creative to you, but your solutions will tend to appear very creative to mere mortals.
Creativity is having a good search ordering.
Eliezer, For systematic approach to creativity you need to inform yourself about Genrich Altshuller's TRIZ - the easiest way to do that is to have a look at the TRIZ journal at www.aitriz.com (Altshuller's deepest book that has been translated from the Russian into English is 'Creativity as an Exact Science').
And, on reflection, since you want to build an autonomous optimiser in the form of self-improving AI - and now I understand your concern that it might become a paperclip maximiser since autonomy includes the ability to select ones own goals, and why would they be such as to permit us to pursue our human goals - you might want to look at another optimising system, Eli Goldratt's 'Theory of Constraints'
HTH
Here is the ultimate work of Modern Art, that truly defies all rules: It isn't mine, it isn't real, and no one knows it exists...
It's...it's beautiful.
Great post for the most part, though I do have to agree with Tim's straw man alert.
Something I learnt while studying postmodern fiction (yeah Eliezer, that's right): Art can be referential, or memetic, or both, or neither. Most is both, in that it (very roughly) is 'like' reality (i.e. it's memetic) and 'seeks to tell us something about' reality (i.e. it's referential). However, there's some really interesting stuff that is neither - defying ideas like logic, causation and induction (let alone plot, character etc) and blatantly having no regard for what Eliezer would call terminal values. (Except, in some cases, at a meta-level outside the text. But not in all cases.) Read up on Alain Robbe-Grillet's fiction and Sam Beckett's 'Trilogy' (and later poetry) for a start. Oh, and John Cage - yes, even 4'33.
Randomness, noise and so on can be astonishingly beautiful, in art or in nature, even to the novice. Or do you think that there are two parts of your brain, one which finds a painting beautiful, and one that finds the night sky beautiful? Yes, there's some high-minded bullshit out there, but as Tim says, please don't draw false boundaries simply to justify your profound bottom line.
And beware of putting a nuts-and-bolts heuristic in place of a sense of aesthetic beauty. You may then find yourself conflicted between finding something beautiful and being unable to understand why. And that truly would be a tragedy.
To all defending Modern Art: Please point to at least one item available online which exemplifies that which you think I'm ignoring or missing.
Eliezer: "To all defending Modern Art: Please point to at least one item available online which exemplifies that which you think I'm ignoring or missing."
This Al Held piece. Upon first glance, it's just a white canvas with a black triangle at the top and the bottom. This is not True Art, you say--but then you read the title, and it all makes sense! Clever! Shocking!
Art! (Hat tip Scott McCloud.)
Not meant as a critique of your piece of conceptual art, but usually all blank paintings have some kind of story behind them.
Robert Rauschenberg: Erased de Kooning Drawing (1953) http://www.tate.org.uk/tateetc/issue8/erasuregenteel.htm
You might also get a kick out of The Clockworks Project: http://www.museumofconceptualart.com/clockworks/clockworks.html
Please point to at least one item available online which exemplifies that which you think I'm ignoring or missing.
Here are the four exhibits I saw last time I went to SFMOMA:
http://www.sfmoma.org/exhibitions/232 http://www.sfmoma.org/exhibitions/266 http://www.sfmoma.org/exhibitions/264 http://www.sfmoma.org/exhibitions/292
All of them "delight the untrained senses of a mere novice" (not so much on a computer screen--these are large and/or detailed works). None of them was there because "sophisticated critics praised their rule-breaking."
Those exhibitions are gone, but if you go down to SFMOMA and see whatever's on now, I'm confident the same will be true.
And, sticking to conceptual art, I'll happily defend John Cage's 4'33": a few sentences on a piece of paper that read like a stunt, but when actually experienced gave me a new understanding of the process of listening. If that's not "a skillful archer send[ing] an arrow into an exceedingly narrow target," I don't know what is.
The same is true of LaMonte Young's X For Henry Flynt. But you have to hear it. Reading about it won't do much for you.
The ongoing popularity of Magritte's "Ceci n'est pas une pipe" and Escher's "Ascending and Descending" as logic/science illustrations would seem to indicate that rationalists are comfortable enough with conceptual art when it suits them.
"If you are constantly surprised by solutions that are high in you preference ordering but low in your search ordering, that is a problem with your search ordering. If your search ordering is correct, creativity is useless."
... Yeah, and optimization is trivial, just use the correct search ordering.
If your search ordering is correct, creativity is useless.
Perhaps, but then would having the correct search ordering be isomorphic to already knowing everything? No free lunch theorem mumble mumble.
To all defending Modern Art: Please point to at least one item available online which exemplifies that which you think I'm ignoring or missing.
Funny -- I didn't actually read the post as an attack on Modern Art. The point seemed to be that, appearances to the contrary, Modern Artists are in fact trying to hit a narrow target, albeit not the one you might at first think. It is presumably this attempted optimization that makes Modern Art (to the extent such a thing does in fact exist) a worthwhile or interesting activity to those who practice it.
Here is the ultimate work of Modern Art, that truly defies all rules: It isn't mine, it isn't real, and no one knows it exists
I read that in the voice of George Carlin delivering a punchline. Can't unhear!
This is reminding me of a book called Plain and Simple by a woman who spent some time as a guest in Amish Families. She found that she'd mistakenly believed that having lots of options was the right way to live, but the actual effect was that she wasn't making decisions. The revelation hit when she realized she actually wanted something in particular, and ferociously re-decorated her kitchen in as somewhat Amish style. "Ferociously" seems like weirdly strong language, but she seemed surprised that she could really want something and go for it.
It's a smallish thing, but I think it's pointing at a pervasive modern error.
Previously in Series: Recognizing Intelligence
Creativity, we've all been told, is about Jumping Out Of The System, as Hofstadter calls it (JOOTSing for short). Questioned assumptions, violated expectations.
Fire is dangerous: the rule of fire is to run away from it. What must have gone through the mind of the first hominid to domesticate fire? The rule of milk is that it spoils quickly and then you can't drink it - who first turned milk into cheese? The rule of computers is that they're made with vacuum tubes, fill a room and are so expensive that only corporations can own them. Wasn't the transistor a surprise...
Who, then, could put laws on creativity? Who could bound it, who could circumscribe it, even with a concept boundary that distinguishes "creativity" from "not creativity"? No matter what system you try to lay down, mightn't a more clever person JOOTS right out of it? If you say "This, this, and this is 'creative'" aren't you just making up the sort of rule that creative minds love to violate?
Why, look at all the rules that smart people have violated throughout history, to the enormous profit of humanity. Indeed, the most amazing acts of creativity are those that violate the rules that we would least expect to be violated.
Is there not even creativity on the level of how to think? Wasn't the invention of Science a creative act that violated old beliefs about rationality? Who, then, can lay down a law of creativity?
But there is one law of creativity which cannot be violated...
Ordinarily, if you took a horse-and-buggy, unstrapped the horses, put a large amount of highly combustible fluid on board, and then set fire to the fluid, you would expect the buggy to burn. You certainly wouldn't expect the buggy to move forward at a high rate of speed, for the convenience of its passengers.
How unexpected was the internal combustion engine! How surprising! What a creative act, to violate the rule that you shouldn't start a fire inside your vehicle!
But now suppose that I unstrapped the horses from a buggy, put gasoline in the buggy, and set fire to the gasoline, and it did just explode.
Then there would be no element of "creative surprise" about that. More experienced engineers would just shake their heads wisely and say, "That's why we use horses, kiddo."
Creativity is surprising - but not just any kind of surprise counts as a creative surprise. Suppose I set up an experiment involving a quantum event of very low amplitude, such that the macroscopic probability is a million to one. If the event is actually observed to occur, it is a happenstance of extremely low probability, and in that sense surprising. But it is not a creative surprise. Surprisingness is not a sufficient condition for creativity.
So what kind of surprise is it, that creates the unexpected "shock" of creativity?
In information theory, the more unexpected an event is, the longer the message it takes to send it - to conserve bandwidth, you use the shortest messages for the most common events.
So do we reason that the most unexpected events, convey the most information, and hence the most surprising acts are those that give us a pleasant shock of creativity - the feeling of suddenly absorbing new information?
This contains a grain of truth, I think, but not the whole truth: the million-to-one quantum event would also require a 20-bit message to send, but it wouldn't convey a pleasant shock of creativity, any more than a random sequence of 20 coinflips.
Rather, the creative surprise is the idea that ranks high in your preference ordering but low in your search ordering.
If, before anyone had thought of an internal combustion engine (which predates cars, of course) I had asked some surprisingly probability-literate engineer to write out a vocabulary for describing effective vehicles, it would contain short symbols for horses, long symbols for flammable fluid, and maybe some kind of extra generalization that says "With probability 99.99%, a vehicle should not be on fire" so that you need to use a special 14-bit prefix for vehicles that violate this generalization.
So when I now send this past engineer a description of an automobile, he experiences the shock of getting a large amount of useful information - a design that would have taken a long time for him to find, in the implicit search ordering he set up - a design that occupies a region of low density in his prior distribution for where good designs are to be found in the design space. And even the added "absurdity" shock of seeing a generalization violated - not a generalization about physical laws, but a generalization about which designs are effective or ineffective.
But the vehicle still goes somewhere - that part hasn't changed.
What if I try to explain about telepresence and virtual offices, so that you don't even need a car?
But you're still trying to talk to people, or get work done with people - you've just found a more effective means to that end, than travel.
A car is a more effective means of travel, a computer is a more effective means than travel. But there's still some end. There's some criterion that makes the solution a "solution". There's some criterion that makes the unusual reply, unusually good. Otherwise any part of the design space would be as good as any other.
An amazing creative solution has to obey at least one law, the criterion that makes it a "solution". It's the one box you can't step outside: No optimization without goals.
The pleasant shock of witnessing Art arises from the constraints of Art - from watching a skillful archer send an arrow into an exceedingly narrow target. Static on a television screen is not beautiful, it is noise.
In the strange domain known as Modern Art, people sometimes claim that their goal is to break all the rules, even the rule that Art has to hit some kind of target. They put up a blank square of canvas, and call it a painting. And by now that is considered staid and boring Modern Art, because a blank square of canvas still hangs on the wall and has a frame. What about a heap of garbage? That can also be Modern Art! Surely, this demonstrates that true creativity knows no rules, and even no goals...
But the rules are still there, though unspoken. I could submit a realistic landscape painting as Modern Art, and this would be rejected because it violates the rule that Modern Art cannot delight the untrained senses of a mere novice.
Or better yet, if a heap of garbage can be Modern Art, then I'll claim that someone else's heap of garbage is my work of Modern Art - boldly defying the convention that I need to produce something for it to count as my artwork. Or what about the pattern of dust particles on my desk? Isn't that Art?
Flushed with triumph, I present to you an even bolder, more convention-defying work of Modern Art - a stunning, outrageous piece of performance art that, in fact, I never performed. I am defying the foolish convention that I need to actually perform my performance art for it to count as Art.
Now, up to this point, you probably could still get a grant from the National Endowment for the Arts, and get sophisticated critics to discuss your shocking, outrageous non-work, which boldly violates the convention that art must be real rather than imaginary.
But now suppose that you go one step further, and refuse to tell anyone that you have performed your work of non-Art. You even refuse to apply for an NEA grant. It is the work of Modern Art that never happened and that no one knows never happened; it exists only as my concept of what I am supposed not to conceptualize. Better yet, I will say that my Modern Art is your non-conception of something that you are not conceptualizing. Here is the ultimate work of Modern Art, that truly defies all rules: It isn't mine, it isn't real, and no one knows it exists...
And this ultimate rulebreaker you could not pass off as Modern Art, even if NEA grant committees knew that no one knew it existed. For one thing, they would realize that you were making fun of them - and that is an unspoken rule of Modern Art that no one dares violate. You must take yourself seriously. You must break the surface rules in a way that allows sophisticated critics to praise your boldness and defiance with a straight face. This is the unwritten real goal, and if it is not achieved, all efforts are for naught. Whatever gets sophisticated critics to praise your rule-breaking is good Modern Art, and whatever fails in this end is poor Modern Art. Within that unalterable constraint, you can use whatever creative means you like.
But doesn't creative engineering sometimes involve altering your goals? First my goal was to try and figure out how to build a vehicle using horses; now my goal is to build a vehicle using fire...
Creativity clearly involves altering my local intentions, my what-I'm-trying-to-do-next. I begin by intending to use horses, to build a vehicle, to drive to the supermarket, to buy food, to eat food, so that I don't starve to death, because I prefer being alive to starving to death. I may creatively use fire, instead of horses; creatively walk, instead of driving; creatively drive to a nearby restaurant, instead of a supermarket; creatively grow my own vegetables, instead of buying them; or even creatively devise a way to run my body on electricity, instead of chemical energy...
But what does not count as "creativity" is creatively preferring to starve to death, rather than eating. This "solution" does not strike me as very impressive; it involves no effort, no intelligence, and no surprise when it comes to looking at the result. If this is someone's idea of how to break all the rules, they would become pretty easy to predict.
Are there cases where you genuinely want to change your preferences? You may look back in your life and find that your moral beliefs have changed over decades, and that you count this as "moral progress". Civilizations also change their morals over time. In the seventeenth century, people used to think it was okay to enslave people with differently colored skin; and now we elect them President.
But you might guess by now, you might somehow intuit, that if these moral changes seem interesting and important and vital and indispensable, then not just any change would suffice. If there's no criterion, no target, no way of choosing - then your current point in state space is just as good as any other point, no more, no less; and you might as well keep your current state, unchanging, forever.
Every surprisingly creative Jump-Out-Of-The-System needs a criterion that makes it surprisingly good, some fitness metric that it matches. This criterion, itself, supplies the order in our beliefs that lets us recognize an act of "creativity" despite our surprise. Just as recognizing intelligence requires at least some belief about that intelligence's goals, however abstract.
One might wish to reconsider, in light of this principle, such notions as "free will that is not constrained by anything"; or the necessary conditions for our discussions of what is "right" to have some kind of meaning.
There is an oft-repeated cliche of Deep Wisdom which says something along the lines of "intelligence is balanced between Order and Chaos", as if cognitive science were a fantasy novel written by Roger Zelazny. Logic as Order, following the rules. Creativity as Chaos, violating the rules. And so you can try to understand logic, but you are blocked when it comes to creativity - and of course you could build a logical computer but not a creative one - and of course 'rationalists' can only use the Order side of the equation, and can never become whole people; because Art requires an element of irrationality, just like e.g. emotion.
And I think that despite its poetic appeal, that whole cosmological mythology is just flat wrong. There's just various forms of regularity, of negentropy, where all the structure and all the beauty live. And on the other side is what's left over - the static on the television screen, the heat bath, the noise.
I shall be developing this startling thesis further in future posts.