Thanks for the insight!
This is actually an incomplete draft that I didn't mean to publish, so I do intend to cover some of your points. It's probably not going to go into the depth you're hoping for since it's pretty much just a synthesis of the bit of information from a segment from a Radiolab episode and three theorems about neural networks.
My goal was to simply use those facts to provide an informal proof that a trade-off exists between latency and optimality* in neural networks and that said trade-off explains why some agents (inclu...
In short, your second paragraph is what I'm after.
Philosophically, I don't think the distinction you make between a design choice and an evolved feature carries much relevance. It's true that some things evolve that have no purpose and it's easy to imagine that emotions are one of things especially since people often conceptualize emotion as the "opposite" of rationality, however; some things evolve that clearly do serve a purpose (in other words there is a justification for their existence), like the eye. Of course nobody sat...
How? The person I'm responding to gets the math of probability wrong and uses it to make a confusing claim that "there's nothing wrong" as though we have no more agency over the development of AI than we do over the chaotic motion of a dice.
It's foolish to liken the development of AI to a roll of the dice. Given the stakes, we must try to study, prepare for, and guide the development of AI as best we can.
This isn't hypothetical. We've already built a machine that's more intelligent than any man alive and which brutal...
This is a pretty lame attitude towards mathematics. If William Rowan Hamilton showed you his discovery of quaternions, you'd probably scoff and say "yeah, but what can that do for ME?".
Occam's razor has been a guiding principal for science for centuries without having any proof for why it's a good policy, Now Solomonoff comes along and provides a proof and you're unimpressed. Great.
After all, a formalization of Occam's razor is supposed to be useful in order to be considered rational.
Declaring a mathematical abstraction useless just because it is not practically applicable to whatever your purpose may be is pretty short-sighted. The concept of infinity isn't useful to engineers, but it's very useful to mathematicians. Does that make it irrational?
Thinking this through some more, I think the real problem is that S.I. is defined in the perspective of an agent modeling an environment, so the assumption that Many Worlds has to put any un-observable on the output tape is incorrect. It's like stating that Copenhagen has to output all the probability amplitudes onto the output tape and maybe whatever dice god rolled to produce the final answer as well. Neither of those are true.
That's a link to somebody complaining about how someone else presented an argument. I have no idea what point you think it makes that's relevant to this discussion.
output of a TM that just runs the SWE doesn't predict your and only your observations. You have to manually perform an extra operation to extract them, and that's extra complexity that isn't part of the "complexity of the programme".
First, can you define "SWE"? I'm not familiar with the acronym.
Second, why is that a problem? You should want a theory that requires as few assumptions as possible to explain as much as possible. The fact that it explains more than just your point of view (POV) is a good thing. It lets yo...
Well, the original comment was about explaining lightning
You're right. I think I see your point more clearly now. I may have to think about this a little deeper. It's very hard to apply Occam's razor to theories about emergent phenomena. Especially those several steps removed from basic particle interactions. There are, of course, other ways to weigh on theory against another. One of which is falsifiability.
If the Thor theory must be constantly modified so to explain why nobody can directly observe Thor, then it gets pushed towards un-falsif...
Thor isn't quite as directly in the theory :-) In Norse mythology...
Tetraspace Grouping's original post clearly invokes Thor as an alternate hypothesis to Maxwell's equations to explain the phenomenon of electromagnetism. They're using Thor as a generic stand-in for the God hypothesis.
Norse mythology he's a creature born to a father and mother, a consequence of initial conditions just like you.
Now you're calling them "initial conditions". This is very different from "conditions" which are directly observabl...
You're trying to conflate theory, conditions, and what they entail in a not so subtle way. Occam's razor is about the complexity of a theory, not conditions, not what the theory and conditions entail. Just the theory. The Thor hypothesis puts Thor directly in the theory. It's not derived from the theory under certain conditions. In the case of the Thor theory, you have to assume more to arrive at the same conclusion.
It's really not that complicated.
That's not how rolling a die works. Each roll is completely independent. The expected value of rolling a 20 sided die is 10.5 but there's no logical way to assign an expected outcome of any given roll. You can calculate how many times you'd have to roll before you're more likely than not to have rolled a specific value (1-P(specific value))^n < 0.5 so log(0.5)/log(1-P(specific_value)) < n. In this case P(specific_value) is 1/20 = 0.05. So n > log(0.5)/log(0.95) = 13.513. So you're more likely than not to have rolled a &quo...
The telos of life is to collect and preserve information. That is to say: this is the defining behavior of a living system, so it is an inherent goal. The beginning of life must have involved some replicating medium for storing information. At first, life actively preserved information by replicating, and passively collected information through the process of evolution by natural selection. Now life forms have several ways of collecting and storing information. Genetics, epigenetic, brains, immune systems, gut biomes, etc.
Obviously a system that collects a...
I think you're example of interpreting quantum mechanics gets pretty close to the heart of the matter. It's one thing to point at solomonoff induction and say, "there's your formalization". It's quite another to understand how Occam's Razor is used in practice.
Nobody actually tries to convert the Standard Model to the shortest possible computer program, count the bits, and compare it to the shortest possible computer program for string theory or whatever.
What you'll find, however; is that some theories amount to othe...
if you cast SI on terms of a linear string of bits, as is standard, you are building in a kind of single universe assumption.
First, I assume you mean a sequential string of bits. "Linear" has a well defined meaning in math that doesn't make sense in the context you used it.
Second, can you explain what you mean by that? It doesn't sound correct. I mean, an agent can only make predictions about its observable universe, but that's true of humans too. We can speculate about multiverses and how they may shape our observations (e.g. the many worlds interpretation of QFT), but so could an SI agent.
That's not how algorithmic information theory works. The output tape is not a factor in the complexity of the program. Just the length of the program.
The size of the universe is not a postulate of the QFT or General Relativity. One could derive what a universe containing only two particles would look like using QFT or GR. It's not a fault of the theory that the universe actually contains ~ 10^80 particles†.
People used to think the solar system was the extent of the universe. Just over a century ago, the Milky Way Galaxy was thought to be the exte...
Once you've observed a chunk of binary tape that has at least one humanlike brain (you), it shouldn't take that many bits to describe another (Thor).
Maxwell's Equations don't contain any such chunk of tape. In current physical theories (the Standard Model and General Relativity), the brains are not described in the math, rather brains are a consequence of the theories carried out under specific conditions.
Theories are based on postulates which are equivalent to axioms in mathematics. They are the statements from which everything else is...
The idea of counting postulates is attractive, but it harbours a problem...
...we'd still find that each postulate encapsulates many concepts, and that a fair comparison between competing theories should consider the relative complexity of the concepts as well.
Yes, I agree. A simple postulate count is not sufficient. That's why I said complexity is *related* to it rather than the number itself. If you want a mathematical formalization of Occam's Razor, you should read up on Solomonoff's Inductive Inference.
To address your point about the...
The Many Worlds interpretation of Quantum Mechanics is considered simple because it takes the math at face value and adds nothing more. There is no phenomenon of wave-function collapse. There is no special perspective of some observer. There is no pilot wave. There are no additional phenomena or special frames of reference imposed on the math to tell a story. You just look at the equations and that's what they say is happening.
The complexity of a theory is related to the number of postulates you have to make. For instance: Special Relativity is actual...
According to the standard model of physics: information can't be created or destroyed. I don't know if science can be said to "generate" information rather than capturing it. It seems like you might be referring to a less formal notion of information, maybe "knowledge".
Are short-forms really about information and knowledge? It's my understanding that they're about short thoughts and ideas.
I've been contemplating the value alignment problem and have come to the idea that the "telos" of life is to captur...
A flaw in the Gödel Machine may provide a formal justification for evolution
I've never been a fan of the concept of evolutionary computation. Evolution isn't fundamentally different than other forms of engineering, rather it's the most basic concept in engineering. The idea slightly modifying an existing solution to arrive at a better solution is a fundamental part of engineering. When you take away all of an engineer's other tools, like modeling, analysis, heuristics, etc. You're left with evolution.
Designing something can be mode...
Rough is easy to find and not worth much.
Diamonds are much harder to find and worth a lot more.
I once read a post by someone who was unimpressed with the paper that introduced Generative Adversarial Networks (GANs). They pointed out some sloppy math and other such problems and were confused why such a paper had garnered so much praise.
Someone replied that, in her decades of reading research papers, she learned that finding flaws is easy and uninteresting. The real trick is being able to find the rare glint of insight that a paper brings to the table. Under...
Drop the "A"
Flight is a phenomenon exhibited by many creatures and machines alike. We don't say mosquitos are capable of flight and helicopters are capable of "artificial flight" as though the word means something fundamentally different for man-made devices. Flight is flight: the process by which an object moves through an atmosphere (or beyond it, as in the case of spaceflight) without contact with the surface.
So why do we feel the need to discuss intelligence as though it wasn't a phenomenon in its own right, but something ...
I was arguing that a specific type of fully general theory lacks a specific type of practical value
In that case, your argument lacks value in its own right because it is vague and confusing. I don't know any theories that fall in the "specific type" of general theory you tried to describe. You used Solomonoff as an example when it doesn't match your description.
one which people sometimes expect that type of theory to have.
When someone develops a formalization, they have to explicitly state its context and any assumptions. If someone exp...
The reason it's untrue is because the concept of "I/O channels" does not exist within physics as we know it.
Yes. They most certainly do. The only truly consistent interpretation I know of current physics is information theoretic anyway, but I'm not interested in debating any of that. The fact is I'm communicating to you with physical I/O channels right now so I/O channels certainly exist in the real world.
the true laws of physics make no reference to inputs, outputs, or indeed any kind of agents at all.
Agents are emergent phenomeno...
I'll probably have a lot more to say on this entire post later, but for now I wanted to address one point. Some problems, like wire-heading, may not be deal-breakers or reasons to write anything off. Humans are capable of hijacking their own reward centers and "wireheading" in many different ways (the most obvious being something like shooting heroin), yet that doesn't mean humans aren't intelligent. Things like wireheading, bad priors, or the possibility of "trolling"[https://www.lesswrong.com/posts/CvKnhXTu9BPcdKE4W/an-...
This tends to assume that we can detangle things enough to see outcomes as a function of our actions.
No. The assumption is that an agent has *agency* over some degrees of freedom of the environment. It's not even an assumption, really; it's part of the definition of an agent. What is an agent with no agency?
If the agent's actions have no influence on the state of the environment, then it can't drive the state of the environment to satisfy any objective. The whole point of building an internal model of the environment is to understand ho...
I don't see the point in adding so much complexity to such a simple matter. AIXI is an incomputable agent who's proofs of optimality require a computable environment. It requires a specific configuration of the classic agent-environment-loop where the agent and the environment are independent machines. That specific configuration is only applicable to a sub-set of real-world problems in which the environment can be assumed to be much "smaller" than the agent operating upon it. Problems that don't involve other agents and have very ...
I think that grappling with embeddedness properly will inevitably make theories of this general type irrelevant or useless
I disagree. This is like saying, "we don't need fluid dynamics, we just need airplanes!". General mathematical formalizations like AIXI are just as important as special theories that apply more directly to real-world problems, like embedded agents. Without a grounded formal theory, we're stumbling in the dark. You simply need to understand it for what it is: a generalized theory, then most of the apparent paradoxes e...
I think you're confusing behavior with implementation.
I'm definitely not treating these as interchangeable -- my argument is about how, in a certain set of cases, they are importantly not interchangeable.
Specifically, I'm arguing that certain characterizations of ideal behavior cannot help us explain why any given implementation approximates that behavior well or poorly.
I don't understand how the rest of your points engage with my argument. Yes, there is a good reason Solomonoff does a weighted average and not an argmax; I don't s...
Hey, G Gordon Worley III!
I just finished reading this post because Steve2152 was one of the two people (you being the other) to comment on my (accidentally published) post on formalizing and justifying the concept of emotions.
It's interesting to hear that you're looking for a foundational grounding of human values because I'm planning a post on that subject as well. I think you're close with the concept of error minimization. My theory reaches back to the origins of life and what sets living systems apart from non-living systems. Living... (read more)