Original post: http://bearlamp.com.au/wicked-problems/

Nothing is a wicked problem.

When I started researching problems and problem solving and solutions and meta-solving processes I stumbled across a wicked problem. This is from Wikipedia:

Rittel and Webber's 1973 formulation of wicked problems in social policy planning specified ten characteristics:[3][4]

1. There is no definitive formulation of a wicked problem.
2. Wicked problems have no stopping rule.
3. Solutions to wicked problems are not true-or-false, but good or bad.
4. There is no immediate and no ultimate test of a solution to a wicked problem.
5. Every solution to a wicked problem is a "one-shot operation"; because there is no opportunity to learn by trial and error, every attempt counts significantly.
6. Wicked problems do not have an enumerable (or an exhaustively describable) set of potential solutions, nor is there a well-described set of permissible operations that may be incorporated into the plan.
7. Every wicked problem is essentially unique.
8. Every wicked problem can be considered to be a symptom of another problem.
9. The existence of a discrepancy representing a wicked problem can be explained in numerous ways. The choice of explanation determines the nature of the problem's resolution.
10. The social planner has no right to be wrong (i.e., planners are liable for the consequences of the actions they generate).

Conklin later generalized the concept of problem wickedness to areas other than planning and policy.  The defining characteristics are:

1. The problem is not understood until after the formulation of a solution.
2. Wicked problems have no stopping rule.
3. Solutions to wicked problems are not right or wrong.
4. Every wicked problem is essentially novel and unique.
5. Every solution to a wicked problem is a 'one shot operation.'
6. Wicked problems have no given alternative solutions.

Defeating a wicked problem

It took me a while to realise what a wicked problem was.  It is evil.  It's a challenge.  It's a one-shot task that you don't really understand until you are attempting to solve it, and then you influence it by trying to solve it.  It's wicked.  And then I started paying attention to everything around me.  And suddenly being a social human was a wicked problem.  Every new interaction is not like the last ones, as soon as you enter the interaction it's too late; and then you only have one shot.  Any action towards the problem adds more complexity to the problem.

Then I looked to time management.  Time management is a wicked problem.  You start out knowing nothing.  It takes time to work out what takes time.  And by the time you think you have a system in place you are already burning more time.  Just catching up on a bad system is failing at the wicked problem.

Then I looked to cooking.  No two ingredients are the same.  Even if you are cooking a thing for the 100th time, the factors of the day, the humidity, temperature, it's going to be different.  You can't know what's going to happen.

Then I looked at politics.  And that's what wicked problems were invented around, social problems where trying to solve the problem changes the problem.  And nothing makes it easier.

Then I took my man-with-a-hammer syndrome and I whacked myself on the head with it.

Okay so not everything is a hammer-nail wicked problem.  Even wicked problems are not a wicked problem.  There are problems out there that are really wicked problems, but it would be rare that you find one.

There is a trick to solving a wicked problem.  The trick is to work out how it's not a wicked problem.  Sure if it's wicked by design so be it.  But real problems in the real world are only pretending to be wicked problems.

1. The problem is not understood until after the formulation of a solution.
   

Yeah, okay.  So you don't really get the problem.  That's cool.  You have done problems before.  And done problems like this before too.  The worst thing to do in the case of being presented with a problem which is not understood is to never attempt it.  If you don't understand - it's time to quantify what you do understand and quantify what you don't understand. After that it's time to look at how much uncertainty you can get away with and how to solve that.  If in doubt refer to the book How to measure anything.

2. Wicked problems have no stopping rule.

Real wicked problems don't have a stopping rule but real world problems do.  Or you can give them one anyway.  How many years is enough years of life.  "I don't know I will decide when I get there".  How much money is enough money? "I will first earn my next 10 million dollarydoos and then decide what to do next".  Yes.  A wicked problem has no stopping rule.  But that's not the real world.  In the real world even a fake stopping rule is good enough for your purposes.

3. Solutions to wicked problems are not right or wrong.

Okay.  Maybe a tricky one.  Lots of things are not right or wrong.  "should I earn to give, or should I bring around FAI sooner?".  Who knows?  Right now people are arguing about it but we don't really know.  If you are making decisions based on right or wrong you probably want to do the right thing.  We know already that if you can't decide that makes all options equally good and irrelevant what you choose.  If you can make one more right than the other - do that.  It's probably not a real wicked problem.  "How should I format this word document" is not a right or wrong, but it's also irrelevant.

4.  Every wicked problem is essentially novel and unique.

Yes.  If you are facing a truly novel and unique problem there is nothing I can say that can help you.  But if you are not, there are many options.  You can:

• build a model scenario and test solutions
• look for existing examples of similar problems and find similar solutions
• try to break the problem into smaller known parts
• consider doing nothing about the problem and see if it solves itself

IF a problem is truly unique, then you really have no reason to fear the unknown because it was not possible to be prepared.  If it's not unique - be prepared (we are all always being prepared for problems all the time)

5. Every solution to a wicked problem is a 'one shot operation.'

Yea, these are hard.  Maybe some of the solutions to 4 will help.  Build models, try search or create similar scenarios (why do trolley problems exist other than to test one-shot problems with pre-thought-out examples).  You only get one shot to launch a nuclear missile the first time (and we are very glad that we didn't ignite the atmosphere that time).  Now days we have computer modelling.  We have prediction markets, we have Bayes.  We can know what we don't know.  And we can make it significantly less dangerous to launch into space - risking the lives of astronauts when we do.

6.  Wicked problems have no given alternative solutions.

Yes.  Wicked problems don't, but real world problems could, and often do.  Find those solutions, or the degrees of freedom in your problem.  Search and try to confirm possible options, find friend scenarios, and use everything you have.

Nothing is a wicked problem.

Meta: This took 1 hour to write and has been on my mind for months.  Coming soon: Defining what is a problem

Original post: http://bearlamp.com.au/addendum-to-applicable-advice/
(part 1: http://bearlamp.com.au/applicable-advice/)

If you see advice in the wild and think somethings along the lines of "that can't work for me", that's a cached thought.  It could be a true cached thought or it could be a false one.  Some of these thoughts should be examined thoroughly and defeated.

If you can be any kind of person - being the kind of person that advice works for - is an amazing skill to have.  This is hard.  You need to examine the advice and decide how that advice happened to work, and then you need to modify yourself to make that advice applicable to you.

All too often in this life we think of ourselves as immutable.  And our problems fixed, with the only hope of solving them to find a solution that works for the problem.  I propose it's the other way around.  All too often the solutions are immutable, we are malleable and the problems can be solved by applying known advice and known knowledge in ways that we need to think of and decide on.

Is it really the same problem if the problem isn't actually the problem any more, but rather the problem is a new method of applying a known solution to a known problem?

(what does this mean) Example: Dieting - is an easy example.

This week we have been talking about Calories in/Calories out.  It's pretty obvious that CI/CO is true on a black-box system level.  If food goes (calories in) in and work goes out (calories out - BMR, incidental exercise, purposeful exercise), that is what determines your weight.  Ignoring the fact that drinking a litre of water is a faster way to gain weight than any other way I know of.  And we know that weight is not literally health but a representation of what we consider healthy because it's the easiest way to track how much fat we store on our body (for a normal human who doesn't have massive bulk muscle mass).

CICO makes for terrible advice.  On one level, yes.  To modify the weight of our black box, we need to modify the weight going in and the weight going out so that it's not in the same feedback loop as it was (the one that caused the box to be fat).  On one level CICO is exactly all the advice you need to change the weight of a black box (or a spherical cow in a vacuum).  

On the level of human systems: People are not spherical cows in a vacuum.  Where did spherical cows in a vacuum come from?  It's a parody of what we do in physics.  We simplify a system down to it's basic of parts and generate rules that make sense.  Then we build up to a complicated model and try to find how to apply that rule.  It's why we can work out where projectiles are going to land because we have projectile motion physics (even though often air resistance and wind direction end up changing where our projectile lands, we still have a good guess.  And we later build estimation systems based on using those details for prediction too).  

So CICO is a black-box system, a spherical cow system.  It's wrong.  It's so wrong when you try to apply it to the real world.  But that doesn't matter!  It's significantly better than nothing.  Or the blueberry diet.

The applicable advice of CICO

The point of applicable advice is to look at spherical cows and not say, "I'm no spherical cow!".  Instead think of ways in which you are a spherical cow.  Ways in which the advice is applicable.  Places where - actually if I do eat less, that will improve the progress of my weight loss in cases where my problem is that I eat too much (which I guarantee is relevant for lots of people).  CICO might not be your silver bullet for whatever reason.  It might be grandma, it might be Chocolate bars, It might be really really really delicious steak.  Or dinner with friends.  Or "looking like you are able to eat forever in front of other people".  If you take your problem.  Add in a bit of CICO, and ask, "how can I make this advice applicable to me?".  Today you might make progress on your problem.

And now for some fun from Grognor:  Have you tried solving the problem?

Meta: this took 30mins to write.  All my thoughts were still clear after recently writing part 1, and didn't need any longer to process.

Part 1: http://bearlamp.com.au/applicable-advice/
(part 1 on lesswrong: http://lesswrong.com/r/discussion/lw/nu3/applicable_advice/)

Original post: http://bearlamp.com.au/applicable-advice/
Part 2: http://bearlamp.com.au/addendum-to-applicable-advice/
Part 2 on lesswrong: http://lesswrong.com/r/discussion/lw/nuf/addendum_to_applicable_advice/

Einstein said, "If I had an hour to solve a problem I'd spend 55 minutes thinking about the problem and 5 minutes thinking about solutions."

The Feynman Algorithm:
Write down the problem.
Think real hard.
Write down the solution.

There is a lot of advice out there on the internet. Any topic you can consider; there is probably advice about. The trouble with advice is that it can be just as often wrong as it is right. Often when people write advice; they are writing about what has worked for them in a very specific set of circumstances. I'm going to be lazy and use an easy example several times here - weight loss, but the overarching concept applies to any type of advice.

Generic: "eat less and exercise more" (obvious example is obvious)

Dieting as a problem is a big and complicated one. But the advice is probably effective to someone. Take any person who is looking to lose weight and this advice is probably applicable. Does this make it good advice? Heck no! It's atrocious. If that's all the dieting advice we needed we wouldn't invent diets like Atkins, Grapefruit, 2&5, low carb and more.

So what does, "starve for two days a week", have to it that "eat less and exercise more" doesn't? Why does the damn advice exist?

Advice like, "eat less and exercise more", is likely to work on someone in the situation of:
1. Eating too much
and
2. not exercising enough.
and
3. having those behaviours for no reason
and
4. having the willpower and desire to change those behaviours 
and 
5. never do them again.
and
6. the introspection to identify the problem as that, and start now.

With this understanding of the advice, you can say that this advice applies to some situations and not others. Hence the concept of "applicable advice".

Given that the advice, "eat less and exercise more" exists, if you take the time to understand why it exists and how it works; you can better take advantage of what it offers.

Understand that if this advice worked for someone there was a way that it worked for that someone. And considering if there is a way to make it work for someone, you can maybe find a way to make it work for you too.

How not to use Applicable advice

When you consider that some advice will be able to be adapted, and some will not, you will sometimes end up in a failure mode of using an understanding of why advice worked to explain away the possibility of it working for you.

Example: "you need to speak your mind more often".  Is advice.  If I decide that this advice is targeted at introverted people who like to be confident before they share what they have to say, but who often say nothing at all because of this lack of confidence.  I then assume that if I am not an introverted person then this advice is not applicable to me and should be ignored.

This is the wrong way to apply applicable advice.  First; the model of "why this advice worked", could be wrong.  Second, this way of applying applicable advice is looking at the scientific process wrong.

Briefly the scientific method:

1. Observe
2. Hypothesis/prediction
3. test
4. analyse
5. iterate
6. conclude

Compared to the failure mode:

1. You noticed the advice worked for someone else
2. You came up with an explanation about why that advice worked and why it won't work for you
3. You decided not to test it because you already concluded it won't work for you
4. you never analyse
5. you never iterate
6. you never confirm your conclusion but still concluded the advice won't work.

How to use applicable advice

Use the scientific method*.  As above:

1. Observe
2. Hypothesis/prediction
3. test
4. analyse
5. iterate
6. conclude

Method:

1. Observe advice working
2. Come up with an explanation for why it worked.  What world-state conditions are needed for successfully executing said advice, search for how it can be applicable to you.
3. Try to make the world into a state such that this advice is applicable
4. Evaluate if it worked
5. Repeat a few times
6. Decide if you can make it work.

*yes I realise this is a greatly simplified form of the scientific method.

Map and territory

Our observable difference - in how you should and should not be using applicable advice - comes from an understanding of what you are trying to change.  The map is what we carry around in our head to explain how the world works.  The territory is the real world.  Just by believing the sky is green I can't change the sky.  But if I believed the sky is green, I could change my belief to be more in line with reality.

If you assume the advice you encounter is applicable to someone, AKA the advice suited their map and how it applied to their territory to successfully be useful.  Then when you compare your territory and their territory - they do not match.  Instead of concluding that your territory is immune - that the advice does not apply, you can try to modify your own map to make the advice work for your territory.

Questions:

• Where have you concluded that advice will not; or does not work for you?
• Is that true?  And can you change yourself to make that advice apply?
  
• Have other people ever failed to take your advice?  What was the advice? and why do you think they didn't take the advice?
• Have you recently not taken advice given to you?  (What was it? and) Why?  Is there a way to make that advice more useful?

Epistemic status: trying not to do it wrong.

Meta: I have been trying to write this for months and months.  Owing to my new writing processes, I am seeing a lot more success.  Writing this out has only taken 2 hours today, but that doesn't count the 5 hours I had put into earlier versions that I nearly entirely deleted.  It also doesn't count that passive time of thinking about how to explain this over the months and months that I have had this idea floating around in my head.  Including explaining it at a local Dojo and having a few conversations about it.  For this reason I would put the total time spent on this post at 22 hours.

• Why I Grew Skeptical of Transhumanism
• Why I Grew Skeptical of Immortalism
• Why I Grew Skeptical of Effective Altruism
• Only Game in Town

Wonderland’s rabbit said it best: The hurrier I go, the behinder I get.

We approach 2016, and the more I see light, the more I see brilliance popping everywhere, the Effective Altruism movement growing, TEDs and Elons spreading the word, the more we switch our heroes in the right direction, the behinder I get. But why? - you say.

Clarity, precision, I am tempted to reply. I have left the intellectual suburbs of Brazil, straight into the strongest hub of production of things that matter, The Bay Area, via Oxford’s FHI office, I now split my time between UC Berkeley, and the CFAR/MIRI office. In the process, I have navigated an ocean of information, read hundreds of books, papers, saw thousands of classes, became proficient in a handful of languages and a handful of intellectual disciplines. I’ve visited the Olympus and I met our living demigods in person as well.

Against the overwhelming forces of an extremely upbeat personality surfing a hyper base-level happiness, these three forces: approaching the center, learning voraciously, and meeting the so-called heroes, have brought me to the current state of pessimism.

I was a transhumanist, an immortalist, and an effective altruist.

Why I Grew Skeptical of Transhumanism

The transhumanist in me is skeptical of technological development fast enough for improving the human condition to be worth it now, he sees most technologies as fancy toys that don’t get us there. Our technologies can’t and won’t for a while lead our minds to peaks anywhere near the peaks we found by simply introducing weirdly shaped molecules into our brains. The strangeness of Salvia, the beauty of LSD, the love of MDMA are orders and orders of magnitude beyond what we know how to change from an engineering perspective. We can induce a rainbow, but we don’t even have the concept of force yet. Our knowledge about the brain, given our goals about the brain, is at the level of knowledge of physics of someone who found out that spraying water on a sunny day causes the rainbow. It’s not even physics yet.

Believe me, I have read thousands of pages of papers in the most advanced topics in cognitive neuroscience, my advisor spent his entire career, from Harvard to Tenure, doing neuroscience, and was the first person to implant neurons that actually healed a brain to the point of recovering functionality by using non-human neurons. As Marvin Minsky, who invented the multi-agent computational theory of mind, told me: I don’t recommend entering a field where every four years all knowledge is obsolete, they just don’t know it yet.

Why I Grew Skeptical of Immortalism

The immortalist in me is skeptical because he understands the complexity of biology from conversations with the centimillionaires and with the chief scientists of anti-aging research facilities worldwide, he met the bio-startup founders and gets that the structure of incentives does not look good for bio-startups anyway, so although he was once very excited about the prospect of defeating the mechanisms of ageing, back when less than 300 thousand dollars were directly invested in it, he is now, with billions pledged against ageing, confident that the problem is substantially harder to surmount than the number of man-hours left to be invested in the problem, at least during my lifetime, or before the Intelligence Explosion.

Believe me, I was the first cryonicist among the 200 million people striding my country, won a prize for anti-ageing research at the bright young age of 17, and hang out on a regular basis with all the people in this world who want to beat death that still share in our privilege of living, just in case some new insight comes that changes the tides, but none has come in the last ten years, as our friend Aubrey will be keen to tell you in detail.

Why I Grew Skeptical of Effective Altruism

The Effective Altruist is skeptical too, although less so, I’m still founding an EA research institute, keeping a loving eye on the one I left behind, living with EAs, working at EA offices and mostly broadcasting ideas and researching with EAs. Here are some problems with EA which make me skeptical after being shook around by the three forces:

1. The Status Games: Signalling, countersignalling, going one more meta-level up, outsmarting your opponent, seeing others as opponents, my cause is the only true cause, zero-sum mating scarcity, pretending that poly eliminates mating scarcity, founders X joiners, researchers X executives, us institutions versus them institutions, cheap individuals versus expensive institutional salaries, it's gore all the way up and down.

2. Reasoning by Analogy: Few EAs are able to and doing their due intellectual diligence. I don’t blame them, the space of Crucial Considerations is not only very large, but extremely uncomfortable to look at, who wants to know our species has not even found the stepping stones to make sure that what matters is preserved and guaranteed at the end of the day? It is a hefty ordeal. Nevertheless, it is problematic that fewer than 20 EAs (one in 300?) are actually reasoning from first principles, thinking all things through from the very beginning. Most of us are looking away from at least some philosophical assumption or technological prediction. Most of us are cooks and not yet chefs. Some of us have not even waken up yet.

3. Babies with a Detonator: Most EAs still carry their transitional objects around, clinging desperately to an idea or a person they think more guaranteed to be true, be it hardcore patternism about philosophy of mind, global aggregative utilitarianism, veganism, or the expectation of immortality.

4. The Size of the Problem: No matter if you are fighting suffering, Nature, Chronos (death), Azathoth (evolutionary forces) or Moloch (deranged emergent structures of incentives), the size of the problem is just tremendous. One completely ordinary reason to not want to face the problem, or to be in denial, is the problem’s enormity.

5. The Complexity of The Solution: Let me spell this out, the nature of the solution is not simple in the least. It’s possible that we luck out and it turns out the Orthogonality Thesis and the Doomsday Argument and Mind Crime are just philosophical curiosities that have no practical bearing in our earthly engineering efforts, that the AGI or Emulation will by default fall into an attractor basin which implements some form of MaxiPok with details that it only grasps after CEV or the Crypto, and we will be Ok. It is possible, and it is more likely than that our efforts will end up being the decisive factor. We need to focus our actions in the branches where they matter though.

6. The Nature of the Solution: So let’s sit down side by side and stare at the void together for a bit. The nature of the solution is getting a group of apes who just invented the internet from everywhere around the world, and get them to coordinate an effort that fills in the entire box of Crucial Considerations yet unknown - this is the goal of Convergence Analysis, by the way - find every single last one of them to the point where the box is filled, then, once we have all the Crucial Considerations available, develop, faster than anyone else trying, a translation scheme that translates our values to a machine or emulation, in a physically sound and technically robust way (that’s if we don’t find a Crucial Consideration otherwise which, say, steers our course towards Mars). Then we need to develop the engineering prerequisites to implement a thinking being smarter than all our scientists together who can reflect philosophically better than the last two thousand years of effort while becoming the most powerful entity in the universe’s history, that will fall into the right attractor basin within mindspace. That’s if Superintelligences are even possible technically. Add to that we or it have to guess correctly all the philosophical problems that are A)Relevant B)Unsolvable within physics (if any) or by computers, all of this has to happen while the most powerful corporations, States, armies and individuals attempt to seize control of the smart systems themselves. without being curtailed by the hindrance counter incentive of not destroying the world either because they don’t realize it, or because the first mover advantage seems worth the risk, or because they are about to die anyway so there’s not much to lose.

7. How Large an Uncertainty: Our uncertainties loom large. We have some technical but not much philosophical understanding of suffering, and our technical understanding is insufficient to confidently assign moral status to other entities, specially if they diverge in more dimensions than brain size and architecture. We’ve barely scratched the surface of technical understanding on happiness increase, and philosophical understanding is also in its first steps.

8. Macrostrategy is Hard: A Chess Grandmaster usually takes many years to acquire sufficient strategic skill to command the title. It takes a deep and profound understanding of unfolding structures to grasp how to beam a message or a change into the future. We are attempting to beam a complete value lock-in in the right basin, which is proportionally harder.

9. Probabilistic Reasoning = Reasoning by Analogy: We need a community that at once understands probability theory, doesn’t play reference class tennis, and doesn’t lose motivation by considering the base rates of other people trying to do something, because the other people were cooks, not chefs, and also because sometimes you actually need to try a one in ten thousand chance. But people are too proud of their command of Bayes to let go of the easy chance of showing off their ability to find mathematically sound reasons not to try.

10. Excessive Trust in Institutions: Very often people go through a simplifying set of assumptions that collapses a brilliant idea into an awful donation, when they reason:
    I have concluded that cause X is the most relevant
    Institution A is an EA organization fighting for cause X
    Therefore I donate to institution A to fight for cause X.
    To begin with, this is very expensive compared to donating to any of the three P’s: projects, people or prizes. Furthermore, the crucial points to fund institutions are when they are about to die, just starting, or building a type of momentum that has a narrow window of opportunity where the derivative gains are particularly large or you have private information about their current value. To agree with you about a cause being important is far from sufficient to assess the expected value of your donation.

11. Delusional Optimism: Everyone who like past-me moves in with delusional optimism will always have a blind spot in the feature of reality about which they are in denial. It is not a problem to have some individuals with a blind spot, as long as the rate doesn’t surpass some group sanity threshold, yet, on an individual level, it is often the case that those who can gaze into the void a little longer than the rest end up being the ones who accomplish things. Staring into the void makes people show up.

12. Convergence of opinions may strengthen separation within EA:  Thus far, the longer someone is an EA for, the more likely they are to transition to an opinion in the subsequent boxes in this flowchart from whichever box they are at at the time. There are still people in all the opinion boxes, but the trend has been to move in that flow. Institutions however have a harder time escaping being locked into a specific opinion. As FHI moves deeper into AI, and GWWC into poverty, 80k into career selection etc… they become more congealed. People’s opinions are still changing, and some of the money follows, but institutions are crystallizing into some opinions, and in the future they might prevent transition between opinion clusters and free mobility of individuals, like national frontiers already do. Once institutions, which in theory are commanded by people who agree with institutional values, notice that their rate of loss towards the EA movement is higher than their rate of gain, they will have incentives to prevent the flow of talent, ideas and resources that has so far been a hallmark of Effective Altruism and why many of us find it impressive, it’s being an intensional movement. Any part that congeals or becomes extensional will drift off behind, and this may create unsurmountable separation between groups that want to claim ‘EA’ for themselves.

Only Game in Town

The reasons above have transformed a pathological optimist into a wary skeptical about our future, and the value of our plans to get there. And yet, I don’t see other option than to continue the battle. I wake up in the morning and consider my alternatives: Hedonism, well, that is fun for a while, and I could try a quantitative approach to guarantee maximal happiness over the course of the 300 000 hours I have left. But all things considered, anyone reading this is already too close to the epicenter of something that can become extremely important and change the world to have the affordance to wander off indeterminately. I look at my high base-happiness and don’t feel justified in maximizing it up to the point of no marginal return, there clearly is value elsewhere than here (points inwards), clearly the self of which I am made has strong altruistic urges anyway, so at least above a threshold of happiness, has reason to purchase the extremely good deals in expected value happiness of others that seem to be on the market. Other alternatives? Existentialism? Well, yes, we always have a fundamental choice and I feel the thrownness into this world as much as any Kierkegaard does. Power? When we read Nietzsche it gives that fantasy impression that power is really interesting and worth fighting for, but at the end of the day we still live in a universe where the wealthy are often reduced to having to spend their power in pathetic signalling games and zero sum disputes or coercing minds to act against their will. Nihilism and Moral Fictionalism, like Existentialism all collapse into having a choice, and if I have a choice my choice is always going to be the choice to, most of the time, care, try and do.

Ideally, I am still a transhumanist and an immortalist. But in practice, I have abandoned those noble ideals, and pragmatically only continue to be an EA.

It is the only game in town.

Edit: Please note that this write-up is poor quality, having the style of a hastily written personal note.

It has been mentioned that there should be a better write-up of open problems in FAI, and as I understand it there is an ongoing effort to explain such open problems. My feeling has been that the recent effort has tended too hold off proposing solutions for too long. I prefer the approach in the Tiling Agents paper, which explained problems through example systems which fail in various respects. What follows is an outline of what I'd write if I spent significant time; I think it is enough to be of some use. This list very much reflects my personal interests & beliefs.

- Tarski's Undefinability Theorem

  - We would like a system to be able to reason about itself (in a few critical ways), and Tarski's Theorem is one of the important obstacles. Kripke provided the first hope of progress in this area by showing that we can embed a partial truth predicate in a language if we accept a "gap" (statements which cannot be assessed as true or false within the system's self-theory). Work over the decades has "reduced the gap" (capturing an increasing number of the self-judgements we want, while always leaving a gap). There are also "glut" theories (which must assess some things as both true and false), which typically mirror gap theories. Paul Christiano provided a theory of probabilistic self-reference which intuitively reduces the "gap" to infinitesimal size: the system's knowledge about its own probabilities can be wrong, but only by an infinitesimal. (For example, if it believes X, then it may fail to believe P(X)=1, but it will still believe P(X)>c for all c<1.) (Note, this feels a bit like a "glut" theory since the system solves the problem by saying too much rather than remaining silent.)

- First Incompleteness Theorem

  - Logical Uncertainty:

    - First-Order ("easy"): assigning probabilities to eventual results of (halting) computations we don't have time to make. I claim this is mostly solved by the FOL prior: we can prefer simpler hypotheses about the behavior of systems, treating computations as black boxes which we use universal induction to predict, while allowing us to incorporate logical reasoning about the function's behavior via bayesian updates. It also solves somewhat more; I claim it will have better properties than Solomonoff if the environment contains objects like halting oracles. (Some deficiencies with respect to reasoning about halting will be mentioned in the next section, however.)

    - Second-Order ("impossible"): If we want to assign probabilities to programs halting, facts of number theory, or facts of set theory, we're in serious trouble. Using the FOL prior admits nonstandard models. It's not yet clear what qualities such a probability distribution should have. It seems reasonable to want universal statements to approach probability 1 as the set of positive examples approaches all the examples; this turns out to be as difficult to compute as all the bits in the arithmetic hierarchy. I thought it would be reasonable to restrict this to just the universal statements about computable predicates; ie, halting facts & equivalent. Will Sawin proved that it is not possible for our beliefs about halting to approach arbitrarily close to the correct values without some false Sigma_2 statements approaching arbitrarily close to 1. It remains an open problem to construct such a prior. My proposal is to (focusing on sentences in the regular forms Pi_n or Sigma_n) require that we only introduce a quantified statement into a theory if a statement of the same form already exists; so sentences at a given level in the arithmetic hierarchy must wait for a sentence at the next lowest level to be introduced. This does not block any true sentences from being produced, but causes halting facts to converge as we eliminate possible halting times. It is an open problem whether this proposed distribution converges. If this distribution exists, call it the bad arithmetic prior (BAP).

- Second Incompleteness Theorem

  - Lobian Obstacle:

    For a machine to plan its actions in the future, it needs to trust itself. The second incompleteness theorem (and, more generally, Lob's theorem) makes this difficult. (All this is insufficiently formal, but the tiling agents paper gives a good explanation.) Several partial solutions have been proposed in a deterministic setting. It is an open problem whether one of Dan Willard's several self-verifying systems solves this problem. (He had multiple proposals...) In case there is no purely logical solution, it seems intuitively promising to look for probabilistic self-trust. Difficulties are already presented by the previous section. If the BAP converges, then we can show that it has self-knowledge of that convergence of the form Paul Christiano described! This makes the false Sigma_2 beliefs feel more acceptable, because it is a necessary feature of the system's self-reference. However, I think it's the case that BAP ends up converging to 1 for *all* sigma_2 statements, which is really terrible.

  - Anti-Lobian Obstacle:

    In case the Lobian obstacle is solved, the anti-lobian obstacle may be a concern.

EDIT: No one was doing what the post suggests, so I accepted an idea from one of the comments, and embedded my response in a comment, not the post itself

I'd like to ask this question to you, and I'll respond it myself as well.

What Is The Worst Problem You've Ever Encountered and Solved? And the One You Didn't, Yet!

Some prior considerations:

1) I mean "problem" in a very general sense, it could be a math problem, an existential problem, a social problem, an akrasia problem,  a disease problem etc...

2) I'd like people to give informative/didactic responses.  Try not only to state the facts, but also to help someone who'd encounter similar situations to be able to deal with them.

3) When talking about the one you didn't, give enough specifics that someone would actually be able to help you.

The general idea is to teach people how to Win by example, taking in consideration all the shortcomings of biases etc...

Well, that is all. One solved, one not yet solved. State your own issues and help others here. Someone else's rationality is always welcome.

Hi, I've been trying to publish an article to less wrong for about a week, but I'm unable to copy the text to the post area and submit it.

It says: Submitting, then it stops and nothing happened.

When I sliced the entire text in small parts (all copied from Open Office) I manage to publish drafts, some with errors such as missing spaces.

When I re-copy from the drafts to another, bigger draft, then many spaces become missing, and part of the text is aligned to the right border, part isnt.

The text was mostly written in open office. I would like help from someone who has a normal Office. If you post me a message with your e-mail in the private message area, I can send it to you, so you see if you can either publish it in drafts, and then copy it again to me in a publishable form, or edit somethign in office that I'm unable in open office, and send the file back to me so I can publish.

I know this is asking a lot, and I would be thankful for anyone who helps me out of this conundrum.