All of JohnDavidBustard's Comments + Replies

Thanks for the comment, I think it is very interesting to think about the minimum complexity algorithm that could plausibly be able to have each conscious experience. The fact that we remember events and talk about them and can describe how they are similar e.g. blue is cold and sad, implies that our internal mental representations and the connections we can make between them must be structured in a certain way. It is fascinating to think about what the simplest 'feeling' algorithm might be, and exciting to think that we may someday be able to create new conscious sensations by integrating our minds with new algorithms.

Thanks it is very handy to get something that is compatible with SUMO.

Thank you for the thoughtful comments. I am not certain that the approach that I am suggesting will be successful but I am hoping that more complex experiences may be explainable from simpler essences, similar to the behaviour of fluids from simpler atomic rules. I am currently focused on the assumption that the brain is similar to a modern reinforcement learning algorithm where there is a one or more large learnt structures and a relatively simple learning algorithm. The first thing I am hoping to look at is if all the concious experiences could be explai... (read more)

Great links, thank you, I hadn't considered the drug effects before that is an interesting perspective on positive sensations. Also I wanted to say I am a big fan of your work, particularly your media synthesis stuff. I use it in teaching of deep learning to show examples of how to use academic source code to explore cutting edge techniques.

Perfect, thank you

A high level post on its use would be very interesting.

I think my main criticism of the Bayes approach is that it leads to the kind of work you are suggesting i.e. have a person construct a model and then have a machine calculate its parameters.

I think that much of what we value in intelligent people is their ability to form the model themselves. By focusing on parameter updating we aren't developing the AI techniques necessary for intelligent behavior. In addition, because correct updating does not guarantee good performance (because the model properties... (read more)

Eh not impossible... just very improbable (in a given world) and certain across all worlds.

I would have thought the more conventional explanation is that the other versions are not actually you (just very like you). This sounds like the issue of only economists acting in the way that economists model people. I would suspect that only people who fixate on such matters would confuse a copy with themselves.

I suspect that people who are vulnerable to these ideas leading to suicide are in fact generally vulnerable to suicide. There are lots of better reasons to... (read more)

Thanks for your reference it is good to get down to some more specific examples.

Most AI techniques are model based by necessity: it is not possible to generalise from samples unless the sample is used to inform the shape of a model which then determines the properties of other samples. In effect, AI is model fitting. Bayesian techniques are one scheme for updating a model from data. I call them incomplete because they leave a lot of the intelligence in the hands of the user.

For example, in the thesis reference the author designs a model of transformations... (read more)

From what I understand, in order to apply Bayesian approaches in practical situations it is necessary to make assumptions which have no formal justification, such as the distribution of priors or the local similarity of analogue measures (so that similar but not exact predictions can be informative). This changes the problem without necessarily solving it. In addition, it doesn't address the issue of AI problems not based on repeated experience, e.g. automated theorem proving. The advantage of statistical approaches such as SVMs is that they produce practi... (read more)

Thank you very much for your great reply. I'll look into all of the links. Your comments have really inspired me in my exploration of mathematics. They remind me of the aspect of academia I find most surprising. How it can so often be ideological, defensive and secretive whilst also supporting those who sincerely, openly and fearlessly pursue the truth.

Thank you, my main goal at the moment is to get a handle on statistical learning approaches and probability. I hope to read Jaynes's book and the nature of statistical learning theory once I have some time to devote to them. however I would love to find an overview of mathematics. Particularly one which focuses on practical applications or problems. One of the other posts mentioned the Princeton companion to Mathematics and that sounds like a good start. I think what I would like is to read something that could explain why different fields of mathematics w... (read more)

So, assuming survival is important, a solution that maximises survival plus wireheading would seem to solve that problem. Of course it may well just delay the inevitable heat death ending but if we choose to make that important, then sure, we can optimise for survival as well. I'm not sure that gets around the issue that any solution we produce (with or without optimisation for survival) is merely an elaborate way of satisfying our desires (in this case including the desire to continue to exist) and thus all FAI solutions are a form of wireheading.

One frustration I find with mathematics is that it is rarely presented like other ideas. For example, few books seem to explain why something is being explained prior to the explanation. They don't start with a problem, outline its solution provide the solution and then summarise this process at the end. They present one 'interesting' proof after another requiring a lot of faith and patience from the reader. Likewise they rarely include grounded examples within the proofs so that the underlying meaning of the terms can be maintained. It is as if the field ... (read more)

I'm not sure I understand the distinction between an answer that we would want and a wireheading solution. Are not all solutions wireheading with an elaborate process to satisfy our status concerns. I.e. is there a real difference between a world that satisfies what we want and directly altering what we want? If the wire in question happens to be an elaborate social order rather than a direct connection why is that different? What possible goal could we want pursued other than the one which we want?

Ok, so how about this work around.

The current approach is to have a number of human intelligences continue to explore this problem until they enter a mental state C (for convinced they have the answer to FAI). The next stage is to implement it.

We have no other route to knowledge other than to use our internal sense of being convinced. I.e. no oracle to tell us if we are right or not.

So what if we formally define what this mental state C consists of and then construct a GAI which provably pursues only the objective of creating this state. The advantage bein... (read more)

Interesting, if I understand correctly the idea is to find a theoretically correct basis for deciding on a course of action given existing knowledge and then to make this calculation efficient and then direct towards a formally defined objective.

As distinct from a system which potentially sub optimally, attempts solutions and tries to learn improved strategies. i.e. one in which the theoretical basis for decision making is ultimately discovered by the agent over time (e.g. as we have done with the development of probability theory). I think the perspective... (read more)

If there is an answer to the problem of creating an FAI, it will result from a number of discussions and ideas that lead a set of people to agreeing that a particular course of action is a good one. By modelling psychology it will be possible to determine all the ways this can be done. The question then is why choose one over any of the others? As soon as one is chosen it will work and everyone will go along with it. How could we rate each one? (they would all be convincing by definition). Is it meaningful to compare them? Is the idea that there is some transcendent answer that is correct or important that doesn't boil down to what is convincing to people?

When I say feel, I include:

I feel that is correct. I feel that is proved etc.

Regardless of the answer, it will ultimately involve our minds expressing a preference. We cannot escape our psychology. If our minds are deterministic computational machines within a universe without any objective value, all our goals are merely elaborate ways to make us feel content with our choices and a possibly inconsistent set of mental motivations. Attempting to model our psychology seems like the most efficient way to solve this problem. Is the idea that there is some othe... (read more)

Ok, I certainly agree that defining the goal is important. Although I think there is a definite need for a balance between investigation of the problem and attempts at its solution (as each feed into one another). Much as how academia currently functions. For example, any AI will need a model of human and social behaviour in order to make predictions. Solving how an AI might learn this would represent a huge step towards solving FAI and a huge step in understanding the problem of being friendly. I.e. whatever the solution is will involve some configuration... (read more)

I suggest just getting some casual exercise or watching some good films and tv shows. They're full of emotionally motivating experiences.

I think there is a worrying tendency to promote puritan values on LW. I personally see no moral problem with procrastination, or even feeling bad every so often. I feel worried that I might not hit deadlines or experience some practical consequence from not working on a task but I wouldn't want to add moral guilt. I think if people lose sight of the pleasures in life they become nihilistic which in turn leads them to be ... (read more)

I am sure these are interesting references for studying pure mathematics but do they contribute significantly to solving AI?

In particular, it is interesting that none of your references mention any existing research on AI. Are there any practical artificial intelligence problems that these mathematical ideas have directly contributed towards solving?

E.g. Vision, control, natural language processing, automated theorem proving?

While there is a lot of focus on specific, mathematically defined problems on LessWrong (usually based on some form of gambling), the... (read more)

I'm not sure of the merit of studying philosophy as opposed to just personally thinking about philosophical ideas. For me, the most profound pragmatic benefit has been to deeply alter my own psychology as a result of examining ideas like free-will and morality. I had a lot of unexamined assumptions and strongly felt conventions and taboos that I managed to overcome through examining my own feelings in a philosophical way. This is very different from the kind of learnt understanding that can be obtained by reading other peoples ideas. I think it is very com... (read more)

Each time a question like this comes up it seems to get down voted as a bad question. I think it's a great question, just one for which there are no obviously satisfactory answers. Dennet's approach seems to be to say, if you just word things differently its all fine, nothing to see here. But to me this is a weird avoiding of the question.

We feel there is a difference between living things and inanimate ones. We believe that other people and some animals are feeling things that are similar to the feelings we have. Many people would find it absurd to think... (read more)

There have been (and continue to be) many approaches to this, in fact the term Good old fashioned AI basically refers to this. It is very interesting that significant progress has not been made with this approach. This has led to greater use of statistical techniques, such as support vector machines or Bayesian networks. A basic difficulty with any approach to AI is that many techniques merely change the problem of learning, generalisation and problem solving into another form rather than solving it. E.g. Formal methods for software development move the pr... (read more)

Has anyone encountered a formal version of this? I.e. a site for the creation of formal logical arguments. Users can create axioms, assign their confidence to them and structure arguments using them. Users can then see the logical consequences of their beliefs. I think it would make a very interesting format for turning debate into a competitive game, whose results are rigorous, machine readable, arguments.

I think this comment highlights the distinction between popular and good.

High ranked posts are popular, good may or may not have anything to do with it.

Personally I find all this kowtowing to the old guard a bit distasteful. One of my favorite virtues of academia is the double blind submissions process. Perhaps there are similar approaches that could be taken here.

Interesting points.

I suspect that predicting the economy with economics is like predicting a persons behaviour from studying their biology. My desire for wisdom is in the form of perspective, I want to know the rough landscape of the economy (like the internal workings of a body).

For example I have little grasp of the industries contributing most to GDP or the taxes within my (or any other) country. In terms of government spending this site provides a nice overview for the UK, but it is only the start. I would love to know the chain of businesses and sys... (read more)

I fear this would reduce LessWrong to referencing research papers. Perhaps there is more value in applying rigor as disagreements emerge. I.e. a process of going from two people flatly disagreeing to establishing criteria to choose between them. I.e. a norm concerning a process for reaching reasonable conclusions on a controversial topic. In this way there would be greater emphasis on turning ambiguous issues into reasonable ones. Which I view as one of the main benefits of rationality.

Thank you, I also agree with your comments on your posting. I generally prefer a balance of pragmatic action with theory. In fact, I view the 'have a go' approach to theoretical understanding to be very useful as well. I think just roughly listing ones thoughts on a topic and then categorising them can be very revealing and really help provide perspective. I recently had a go at my priorities (utility function) and came up with the following:

• To be loved
• To be wise
• To create things that I am proud of
• To be entertained
• To be respected
• To be independent (id

True, in fact despite my comments I am optimistic of the potential for progress in some of these areas. I think one significant problem is the inability to collaborate on improving them. For example, research projects in robotics are hard to build on because replicating them requires building an equivalent robot, which is often impractical. The robocup is a start as at least it has a common criteria to measure progress with. I think a standardised simulator would help (with challenges that can be solved and shared within it) but even more useful would be t... (read more)

The real difficulty with both these control problems is that we lack a theory for how to ensure the stability of learning based control systems. Systems that appear stable can self destruct after a number of iterations. A number of engineering projects have attempted to incorporate learning. However, because of a few high profile disasters, such systems are generally avoided.

In terms of emulation, the resolution is currently good enough to identify molecules communicating across synapses. This enables an estimate of synapse strengths as well as a full wiring diagram of physical nerve shape. There are emulators for the electrical interactions of these systems. Also our brains are robust enough that significant brain damage and major chemical alteration (ecstasy etc.) are recoverable from, so if anything brains are much more robust than electronics. AI, in contrast, has real difficulty achieving anything but very specific proble... (read more)

I really like this post. It touches on two topics that I am very interested in:

How society shapes our values (domesticates us)

and

What should we value (what is the meaning of life?)

I find the majority of discussions extremely narrow, focusing on details while rarely attempting to provide perspective. Like doing science without a theory, just performing lots of specific experiments without context or purpose.

1 Why are things the way they are and why do we value the things we value? A social and psychological focus, Less Wrong touches on these issues but ap... (read more)

I think this section of your post is part of what makes me feel bad about your comment. The reason I said I like it, is because I think it's important that people can talk about these things and the fact that your comments affect me in that way highlights that they are important to me.

I would have worded this more strongly, myself. In my experience, people who are themselves inclined towards reasoned debate, even civilly, drastically overestimate how much other people are also inclined towards debate and argument.

I can't speak for anyone else, but pers... (read more)

Thank you, that's very interesting, and comforting.

Thank you, this is very interesting. I'm not sure of the etiquette, but I'm reposting a question from an old article, that I would really appreciate your thoughts on.

Is it correct, to say that the entropy prior is a consequence of creating an internally consistent formalisation of the aesthetic heuristic of preferring simpler structures to complex ones?

If so I was wondering if it could be extended to reflect other aesthetics. For example, if an experiment produces a single result that is inconsistent with an existing simple physics theory, it may be that t... (read more)

Is there a bound on the amount of data that is necessary to adjust a prior of a given error magnitude? Likewise, if the probability is the result of a changing system I presume it could well be the case that the pdf estimates will be consistently inaccurate as they are constantly adjusting to events whose local probability is changing. Does the Bayesian approach help, over say, model fitting to arbitrary samples? Is it, in effect, an example of a model fitting strategy no more reasonable than any other?

I suppose the question is, how to calculate the priors so they do make sense. In particular, how can an AI estimate priors. I'm sure there is a lot of existing work on this. The problem with making statements about priors that don't have a formal process for their calculation is that there is no basis for comparing two predictions. In the worst case, by adjusting the prior the resulting probabilities can be adjusted to any value. Making the approach a formal technique which is potentially just hiding the unknowns in the priors. In effect being no more reasonable because the priors are a guess.

I like your post because it makes me feel bad.

What I mean by that is that it gets at something really important that I don't like. The problem is that I get more pleasure from debates than almost anything else. I search for people who don't react in the intensely negative way you describe, and I find it hard to empathise with those that do. I don't do this because I think one method is 'right' and the other 'wrong' I just don't enjoy trying to conform to others expectations and prefer to find others who can behave in the same way. I think for most people ... (read more)

Wow, this really brings home the arbitrary nature of the Bayesian approach. If we're trying to get an AI to determine what to do, it can't guess meaningful priors (and neither can we come to that). I presume when it is applied there is a load of theoretical approaches to prior model estimation or is a uniform prior just used as default? In which case are there other occasions when a frequentist and bayesians probability estimates differ?

Thank you for your reply. It really highlights the difficulty of making an appropriate choice. There is also the difficulty that a lot of professions require specialised training before they can be experienced.

I did not find any of the careers guidance information at school or university to be particularly helpful. However after working in games for a number of years it was clear that there were a number of types with very similar backgrounds. I think it would be very valuable to read honest autobiographical accounts of different professions and ideally s... (read more)

I think your very first step Identify is the key to all this.

Is it rational to pursue an irrational goal rationally?

Our culture focuses on external validation, achievement and winning. My concern is that this is a form of manipulation focused on improving a societies economic measures of value over an individual's personal satisfaction.

In contrast, the science of happiness seems like a good start. This work seems to focus on developing techniques to come to feel satisfaction with ones current state. Perhaps a next step is to look at how communities and or... (read more)

Thank you! That's a great link I'll look into it.

For about 3 years now I've been giving to a number of charities through a monthly standing order. Initially setting it up was very satisfying and choosing the charities was a little like purchasing a new gadget, assuming a hands on experience is not available, and there are no trusted reviewers, I look at the various options and go with the ones whose advertising most closely reflected my personality and who looked the least like charlatans. With gadget purchases I find these indirect signals much more informative of the experience with the product than an... (read more)

Thanks for the link.

You make a good point about the lack of a clear distinction, and at a fundamental level I believe that our genes and external environment determine our behaviour (I am a determinist, i.e. I don't believe in free will). However, I think that it is also possible to be highly motivated about different things which can cause a lot of mental stress and conflict. I think this occurs because we have a number of distinct evolved motivations which can drive us in opposing ways (e.g. the desire to eat, the status desire of being thin, the moral ... (read more)

Thanks for the link, very interesting.

I've wrestled with this disparity myself, the distance between my goals and my actions. I'm quite emotional and when my goals and my emotions are aligned I'm capable of rapid and tireless productivity. At the same time my passions are fickle and frequently fail to match what I might reason out. Over the years I've tried to exert my will over them, developing emotionally powerful personal stories and habits to try and control them. But every time I have done so it tends to cause more problems that it fixes. I experience a lot of stress fighting with myself ... (read more)

Thank you, it's such a pleasure to find so many interesting discussions of these ideas.

I’m glad you like it : )

I suppose the question is, to what extent can ideas be separated from social dynamics, such as status and legitimacy, and therefore not carry with them the risk of causing anger and fear.

Well ideas can certainly create positive as well as negative responses. For example, more accurate understanding and the communication of practically useful approaches are often intrinsically enjoyable. As is the communication of experience that might help determine the correct course of action or help avoid problems (i.e. personal stories, news).... (read more)

Thank you for the link.

I think the discussion distinguishing like and want is the beginning of the answer. My view is that there are a number of distinct complementary motivations which can cause subtly different emotions, each of which can be referred to as happiness. With each motivation having evolved because it contributes towards our survival. These distinctions become clearer when trying to create enjoyment experiences and I'll elaborate on my take on them in the next article.

What I think is so interesting (and important) about this is that without ... (read more)