Well, complete transparency is only relevant if we can show both of the following
That's overstated. If one is going to be pushing for a singleton, complete transparency is relevant if it makes for a significantly better singleton than otherwise. Especially if the degree of transparency is likely to be stable (or evolve in a way that depends on its initial condition) once the singleton is in place. Similarly for any other properties of the singleton.
The idea here is to binarize the problem via a definition of acceptability. From this perspective (which, it is argued, would facilitate analysis), the question is not relevant.
I'm not sure if we thinking in terms of acceptable/unacceptable is actually very useful, though...
BTW, if anyone wants to go to singinst.org and download the audio, you'll note that the actual event did not occur the exact way I remembered it, which should surprise no one here who knows anything about human memory. In particular, Cascio spontaneously provided the Genome Project example, rather than needing to be asked for it.
Generally, the reason I avoid identifying the characters in my examples is that it feels to me like I'm dumping all the sins of humankind upon their undeserving heads - I'm presenting one error, out of context, as exemplar for all the errors of this kind that have ever been committed, and showing none of the good qualities of the speaker - it would be like caricaturing them, if I called them by name.
That said, the reason why I picked this example is that, in fact, I was thinking of Orwell's "Politics and the English Language" while writing this post. And as Orwell said:
In the case of a word like democracy, not only is there no agreed definition, but the attempt to make one is resisted from all sides. It is almost universally felt that when we call a country democratic we are praising it: consequently the defenders of every kind of regime claim that it is a democracy, and fear that they might have to stop using that word if it were tied down to any one meaning.
If you simply issue a call for "democracy", why, no one can disagree with that - it would be like disagreeing with a call for apple pie. As soon as you propose a specific mechanism of democracy, whether it is Congress passing a law, or an AI polling people by phone, or government funding of a large research project whose final authority belongs to an appointed committee of eminent scientists, et cetera, people can disagree with that, because they can actually visualize the probable consequences.
So there is a tremendous motive to avoid criticism, to keep to the safely vague areas where people will applaud you, and not to make the concrete proposals where people might - gasp! - disagree.
Now I do not accuse you too much of this, because you did say "Genome Project" when challenged instead of squirting out an immense cloud of ink. But it is why I challenged you to define "democracy". I think that the real value in these discussions comes from people willing to make concrete proposals and expose themselves to criticism.
Really bad example...
My impression is that democracy is seeing a sharp uptick in attacks from elites and intellectuals. There are many who now believe, e.g., that the US should be more like China (see: the success of Trump).
As the speaker noted, he expected his speech to be controversial in that crowd, and in a way, it was, as evidenced by this blog post :)
"Every set of numbers has a least element" - along with the other, non-inductive axioms of Peano arithmetic.
You should say "replace THEM", in that case, to refer to the infinite set of axioms, as opposed to Peano Arithmetic.
If the brain is efficient, and it is, then you shouldn't try to cargo-cult copy the brain, any more than we cargo-culted feathery wings to make airplanes.
The wright brothers copied wings for lift and wing warping for 3D control both from birds. Only the forward propulsion was different.
make an engine based on a clear theory of which natural forces govern the phenomenon in question -- here, thought.
We already have that - it's called a computer. AGI is much more specific and anthropocentric because it is relative to our specific society/culture/economy. It requires predicting and modelling human minds - and the structure of efficient software that can predict a human mind is itself a human mind.
"the structure of efficient software that can predict a human mind is itself a human mind." - I doubt that. Why do you think this is the case? I think there are already many examples where simple statistical models (e.g. linear regression) can do a better job of predicting some things about a human than an expert human can.
Also, although I don't think there is "one true definition" of AGI, I think there is a meaningful one which is not particularly anthropocentric, see Chapter 1 of Shane Legg's thesis: http://www.vetta.org/documents/Machine_Super_Intelligence.pdf.
"Intelligence measures an agent’s ability to achieve goals in a wide range of environments."
So, arguably that should include environments with humans in them. But to succeed, an AI would not necessarily have to predict or model human minds; it could instead, e.g. kill all humans, and/or create safeguards that would prevent its own destruction by any existing technology.
Thanks for the clarifications - I'll make this short.
Judea Pearl (and a whole host of others) showed up, formalized probabilistic graphical models, related them to Bayesian inference, and suddenly a whole class of ad-hoc solutions were superseded.
Probabilistic graphical models were definitely a key theoretical development, but they hardly swept the field of expert systems. From what I remember, in terms of practical applications, they immediately replaced or supplemented expert systems in only a few domains - such as medical diagnostic systems. Complex ad hoc expert systems continued to dominate unchallenged in most fields for decades: in robotics, computer vision, speech recognition, game AI, fighter jets, etc etc basically everything important. As far as I am aware the current ANN revolution is truly unique in that it is finally replacing expert systems across most of the board - although there are still holdouts (as far as I know most robotic controllers are still expert systems, as are fighter jets, and most Go AI systems).
The ANN solutions are more complex than the manually crafted expert systems they replace - but the complexity is automatically generated. The code the developers actually need to implement and manage is vastly simpler - this is the great power and promise of machine learning.
Here is a simple general truth - the Occam simplicity prior does imply that simpler hypotheses/models are more likely, but for any simple model there are an infinite family of approximations to that model of escalating complexity. Thus more efficient approximations naturally tend to have greater code complexity, even though they approximate a much simpler model.
My claim is that there are other steps such as those that haven't been made yet, that there are tools on the order of "causal graphical models" that we are missing.
Well, that would be interesting.
I'm not sure whether your view is of the form "actually the programmer of the future would say "I don't know how it's building a model of the world either, it's just a big neural net that I trained for a long time"" or whether it's of the form "actually we do know how to set up that system [multi-level model] already", or whether it's something else entirely. But if it's the second one, then by all means, please tell :-)
Anyone who has spent serious time working in graphics has also spent serious time thinking about how to create the matrix - if given enough computer power. If you got say a thousand of the various brightest engineers in different simulation related fields, from physics to graphics, and got them all working on a large mega project with huge funds it could probably be implemented today. You'd start with a hierarchical/multi-resolution modelling graph - using say octrees or kdtrees over voxel cells, and a general set of hierarchical bidirectional inference operators for tracing paths and interactions.
To make it efficient, you need a huge army of local approximation models for different phenomena at different scales - low level quantum codes just in case, particle level codes, molecular bio codes, fluid dynamics, rigid body, etc etc. It's a sea of codes with decision tree like code to decide which models to use where and when.
Of course with machine learning we could automatically learn most of those codes - which suddenly makes it more tractable. And then you could use that big engine as your predictive world model, once it was trained.
The problem is to plan anything worthwhile you need to simulate human minds reasonably well, which means to be useful the sim engine would basically need to infer copies of everyone's minds . . ..
And if you can do that, then you already have brain based AGI!
So I expect that the programmer from the future will say - yes at the low level we use various brain-like neural nets, and various non-brain like neural nets or learned virtual circuits, some operating over explicit space-time graphs. In all cases we have pretty detailed knowledge of what the circuits are doing - here take a look at that last goal update that just propagated in your left anterior prefrontal cortex . ..
While the methods for finding a solution to a well-formed problem currently used in Machine Learning are relatively well understood, the solutions found are not.
And that is what really matters from a safety perspective. We can and do make some headway in understanding the solutions, as well, but the trend is towards more autonomy for the learning algorithm, and correspondingly more opaqueness.
As you mentioned, the solutions found are extremely complex. So I don't think it makes sense to view them only in terms of approximations to some conceptually simple (but expensive) ideal solution.
If we want to understand their behaviour, which is what actually matters for safety, we will have to grapple with this complexity somehow.
Personally, I'm not optimistic about experimentation (as it is currently practiced in the ML community) being a good enough solution. There is, at least, the problem of the treacherous turn. If we're lucky, the AI jumps the gun, and society wakes up to the possibility of an AI trying to take over. If we're unlucky, we don't get any warning, and the AI only behaves for long enough to gain our trust and discover a nearly fail-proof strategy. VR could help here, but I think it's rather far from a complete solution.
BTW, SOTA for Computer Go uses ConvNets (before that, it was Monte-Carlo Tree Search, IIRC): http://machinelearning.wustl.edu/mlpapers/paper_files/icml2015_clark15.pdf ;)
"Every set of numbers has a least element" clearly does NOT define the natural numbers. Consider N U {-1}.
I'm afraid I won't have time to give you more help. There's a short summary of each sequence under the link at the top of the page, so it won't take you forever to see the relevance.
EDIT: you're wondering elsewhere in the thread why you're not being well received. It's because your post doesn't make contact with what other people have thought on the topic.
I put "enjoy itself" in quotes, because I don't mean it literally. The questions that that sequence addresses according to the summary don't seem relevant to what I am trying to get at.
I guess I need to be more precise. I just mean how can we maximize the integral of experience through time (whether we let experience take negative values is a detail). This was one of Tegmark's proposals in that paper, already, except he is writing in terms of a final goal instead of a process, which was the point of my post...
"The amount of consciousness in our Universe, which Giulio Tononi has argued corresponds to integrated information"
While I don't have too much experience to back this up, I think it is probably a lot of things I'm familiar with, elaborated at length, with perhaps a few insights sprinkled in.
Yes, I don't particularly like the way the sequences are written either :/ But I think the kind of thing you're talking about in this post is the sort of topic they address. LW Wiki pages are often better, e.g. see this one:
if a p-zombie is atom-by-atom identical to a human being in our universe, then our speech can be explained by the same mechanisms as the zombie's, and yet it would seem awfully peculiar that our words and actions would have one entirely materialistic explanation, but also, furthermore, our universe happens to contain exactly the right bridging law such that our experiences are meaningful and our consciousness syncs up with what our merely physical bodies do. It's too much of a stretch: Occam's razor dictates that we favor a monistic universe with one uniform set of laws.
I see this as compatible with my reply to skeptical_lurker above.
My point is: how do you evaluate if something has preferences? How do you disambiguate preferences from statements like "I prefer __"? Clearly we DO distinguish between these.
Agreed. I don't have any easy answer to this question. It's kind of like asking the question "if someone is ill or injured, how do you fix them?" It's an important question worthy of extensive study (at least insofar as it's relevant to whatever ethical question you're currently being presented with).
And it's possible that you and I would disagree on how to carve reality in to that which has preferences we consider meaningful vs that which doesn't. Occam's Razor only applies to the territory, not the map, so there's no penalty for us drawing our boundaries in as complicated & intricate a way as we like (kind of like the human-drawn country boundaries on real maps).
I know all about philosophical zombies.
Agreed. I don't have any easy answer to this question.
Do you have any answer at all? Or anything to say on the matter? Would you at least agree that it is of critical ethical importance, and hence worthy of discussion?
And it's possible that you and I would disagree on how to carve reality in to that which has preferences we consider meaningful vs that which doesn't.
Of course, but I assume you agree with me about the program I wrote?
In any case, I think it would be nice to try and forge some agreement and/or understanding on this matter (as opposed to ignoring it on the basis of our disagreement).
Suppose all information processing is inextricably linked to qualia. Now I suppose there is information processing in rocks of a form, in the equations of themodynamics, motion etc that govern the rocks behaviour. But qualia does not imply self-awareness (1), and there's no way you can communicate with the rock. Qualia also doesn't imply emotions (2), and if there is neither self-awareness nor emotions then I don't see why there need be any moral considerations.
As to determining the truth of Panpsychism and categorising which things have emotions, self awareness etc, I shall defer this problem to future superintelligences. Additionally, a CEV AI should devote a lot of resources to humans regardless of whether panpsychism is true, because most people don't believe in Peter Singer style altruism.
1 because (a) people who meditate for many years or take a large does of dissociative drugs can experience ego-death, where they stop conceptualising a self, but they still experience qualia. (b) most animals are not self-aware, yet intuitions and occam's razor tell me that they still experience qualia
2 some people experience emotional blunting, but while the world may seem grey emotionally, yet they still experience qualia. Additionally, squid do not have emotions, and again I believe they still have qualia.
EDIT: as well as lacking self-awareness and emotions, rocks also lack agency. The question of what to do with a human, who due to various incurable diseases, lacks self-awareness, emotions and agency is left as an excersize for the reader.
How do you know what a CEV AI should do?
How do you know that squids don't have emotions?
Define agency.
You could have at least stepped up to the challenge you left to the reader.
= f037147d6e6c911a85753b9abdedda8d){kind=logo}
= 27d567bc2d48d9f40e9ccc20ea370b15)
= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
I think an important first step should be to try to get a sense of the distribution over possible singletons.
Only then can we have a good idea of where a line of "acceptableness" should be drawn.