All of Robert Kralisch's Comments + Replies

Yeah, I am not super familiar with PCA, but my understanding is that while both PCA and referential containment can be used to extract lower-dimensional or more compact representations, they operate on different types of data structures (feature vectors vs. graphs/hypergraphs) and have different objectives (capturing maximum variance vs. identifying self-contained conceptual chunks). Referential containment is more focused on finding semantically meaningful and contextually relevant substructures within a causal or relational knowledge representation. It a... (read more)

I think that "epistemic rationality" matches very well with what I am thinking of as level 3, which is my notion of intelligence. It is indeed applicable to non-agentic systems.
I am still thinking about whether to include meta-learning (referring to updating level 3 algorithms based on experience) and meta-processes above that in my concept of intelligence.

Would this layer of meta-learning be part of epistemic rationality, do you think? It becomes particularly relevant if the system is resource constrained and has to prioritize what to learn about, and/or ... (read more)

Thanks a lot for the encouragement :)

Yes, I am trying to understand a generalized (which also means simplified) and formalizable parallel to human cognition. Some of my thinking on this is inspired by predictive coding and adaptive resonance theory (although prettly loosely by the latter), and I am trying to figure out the implications of our most updated understanding of neurobiological principles, together with a notion of the "riverbeds of cognition". 

In other words, how can we design an architecture such that it is not pressured to take shortcuts ... (read more)

Hm, I'll give some thought to how to integrate different types of data with this picture, but I think that the "useful" classification of data ultimately depends on whether the agent possesses the right "key" to interpret it, and by extension, how difficult that "key" is to produce from concepts that the agent is already proficient with. 

At the end of the day, the agent can only "understand" any data in terms of internalized concepts, so there will often be some uncertainty whether the difficulty is in translating sensible data into that internal repr... (read more)

Yeah, I wish we had some cleaner terminology for that.
Finetuning the "simulation engine" towards a particular task at hand (i.e. to find the best trade-off between breadth and depth search in strategy games, or even know how much "thinking time" or "error allowance" to allocate to a move), given limited cognitive resources, is something that I would associate with level 3 capability.
It certainly seems like learning could go into the direction of making the model of the game more useful by either improving the extent to which this model predicts/ouputs good... (read more)

Thanks! 

In your example, I think it is possible that the hunter-gatherer solves the problem through pure level 2 capability, even if they never encountered this specific problem before. Using causal models compositionally to represent the current scene, and computing it to output a novel solution, does not actually require that the human updates their causal models about the world. 
I am trying to distinguish agents with this sort of compositional world model from ones that just have a bunch of cashed thoughts or habits (which would correspond to ... (read more)

the maximum plan length is only $10\cdot 100=1000$ steps

You mean the maximum length for an efficient/minimal plan, right? Maybe good to clarify (even if obvious in this case). Just a thought.

I believe that it is very sensible to bring this sort of structure into our approach to AGI safety research, but at the same time it seems very clear that we should update that structure to the best of our ability as we make progress in understanding the challenges and potentials of different approaches. 

It is a feedback loop where we make each step according to our best theory of where to make it, and use the understanding gleaned from that step to update the theory (when necessary), which could well mean that we retrace some steps and recalibrate (t... (read more)