LESSWRONG
LW

Comment Permalink

I think I’ve found a somewhat easy-to-make error that could pose a significant existential risk when making an AGI. This error can potentially be found in hierarchical planning agents, where each high-level action (HLA) is essentially its own intelligent agent that determines what lower-level actions to do. Each higher-level action agent would treat determining what lower-level action to do as a planning problem and would try to take the action that maximizes its own utility function (if its a utility-based agent) or (if it’s a goal-based agent) probability of accomplishing its goal while minimizing its cost function (UOCF).

For these agents, it is absolutely vital that each HLA’s UOCF prevents the HLA from doing anything to interfere with the highest-level action maximizing its utility function, for example by rewriting the utility function of higher-level actions or sending the higher-level actions deliberately false information. Failing to do so would result in an error that could significantly increase existential risk. To explain why, consider an agent whose highest-level action wants to maximize the number of fulfilling lives lived. In order to do this, the agent has a lower-level action whose goal is to go to a warehouse to get supplies. The cost function of this lower-level action is simply a function of, say, the amount of time it takes for the agent to reach the warehouse and the amount of money spent or money in damages done. In this situation, the lower-level action agent might realize that there is a chance that the higher-level action agent changes its mind and decides to do something other than go to the warehouse. This would cause the lower-level action to fail to accomplish its goal. To prevent this, this lower-level action may try to modify the utility function of the higher-level action to make it certain to continue trying to go to the warehouse. If this is done repeatedly by different lower-level actions, the resultant utility function could be quite different from the highest-level action’s original utility function and may pose a large existential risk. Even if the lower-level action can’t rewrite the utility function of higher-level actions, it may still sabotage the higher-level action in some other way to further its own goals, for example by sending false information to higher-level actions.

To prevent this, the utility function of the lower-level action can simply be to maximize the highest-level action’s utility, and it can see the UOCF it was provided with as a rough method of maximizing the highest-level action’s utility function. In order to make the UOCF accurately represent the highest-level action’s utility function, it would (obviously) need to place high cost on interfering with the highest-level action’s attempts to maximize its utility. Some basic ideas on how to do this is for there to be very high cost in changing the utility functions of higher-level actions or giving them deliberately false information. Additionally, the cost of this would need to increase when the agent is more powerful, as the more powerful the agent is, the greater damage a changed utility function could do. Note that although higher-level actions could learn through experience what the UOCFs of lower-level actions should be, great care would need to be taken to prevent the AGI from, when still inexperienced, accidentally creating a lower-level action that tries to sabotage higher-level actions.

Gunnar_Zarncke10y20

I don't know why this is downvoted so much without an explanation. The problem from the interaction with sub-agents is real even if already known, but G0W51 may not know that.

11TrE10y

Please insert some line-breaks at suitable points to make your comment be more readable. At the moment it's figuratively a wall of text. Edit: Thank you.

See in context

7 Open Thread, Apr. 06 - Apr. 12, 2015

by philh

6th Apr 2015

1 min read

128

7

If it's worth saying, but not worth its own post (even in Discussion), then it goes here.

Notes for future OT posters:

1. Please add the 'open_thread' tag.

2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)

3. Open Threads should be posted in Discussion, and not Main.

4. Open Threads should start on Monday, and end on Sunday.

Open Threads

Personal Blog

7

Open Thread, Apr. 06 - Apr. 12, 2015

0Good_Burning_Plastic

New Comment

128 comments, sorted by

top scoring

Click to highlight new comments since: Today at 8:48 AM

Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

[-]artemium10y230

Interesting talk at BOAO forum : Elon Musk, Bill Gates and Robin Li (Baidu CEO). They talk about Superintelligence at around 17:00 minute.

https://www.youtube.com/watch?v=NG0ZjUfOBUs&feature=youtu.be&t=17m

Elon is critical of Andrew Ng remark that 'we should worry about AI like we should worry about Mars overpopulation' ("I know something about mars" LOL)
Bill Gates mentioned Nick Bostrom and his book 'Superintelligence'. His seems to have read the book. Cool.
Later, Robin Li mentions China Brain projects, which appears to be Chinese government AGI project (anyone knows something about it? Sounds interesting...hopefully it won't end like Japans 'fifth-generation computing' in the 80s)

[-]ChristianKl10y200

Bill Gates mentioned Nick Bostrom and his book 'Superintelligence'. His seems to have read the book. Cool.

He not only mentions it. He recommends it to a room of influential people.

[-]Capla10y150

I'm leading a rationality training group. We're working through the most recent CFAR curriculum, but I also want to work from parts of the sequences.

Which posts in the sequences were particularly impactfull for you? Not just ones that you found interesting, but ideas that you actually implemented in your thinking about object-level stuff.

I'm particularly interested in posts that we could spin out into techniques to practice, like noticing confusion or leaving a line of retreat.

7polymathwannabe10y

Think the thought that hurts the most.

3wadavis10y

“That which can be destroyed by the truth should be.” I couldn't find the sequence that covers it directly, but going through my old journals, this one came up repeatedly while facing hard decisions.

1Viliam10y

"Feeling Rational" mentions it.

[-]So8res10y140

It's been more than a year since I wrote the mechanics of my recent productivity (shortly before being hired as a full-time MIRI researcher), and, as promised about a year ago, I have now posted a short update. It's been a wild year.

6skilesare10y

This was a really helpful and encouraging read. Thanks for taking the time to post it. I'm doing some research in the are of economics and have had trouble finding people willing to discuss my ideas. I realize I'm going to have to get much more familiar with the math. This makes it seem possible.

[-]advancedatheist10y110

A lot of really stupid comments follow this article:

Tech titans’ latest project: Defy death

http://www.washingtonpost.com/sf/national/2015/04/04/tech-titans-latest-project-defy-death/

[-][anonymous]10y130

...as with every other article on the internet.

[-]iarwain110y100

I recently stumbled across the work of Helen De Cruz. I think many of her articles will be of interest to readers of LW. Here are a few of particular interest:

[-]philh10y90

I have joined the ranks of LWers with tumblrs.

Also, interested in discussion on conversational tops and bottoms.

7MathiasZaman10y

In case no-one has mentioned this to you yet, we have a list. If you want on, just send me (Yxoque) a message.

[-]iarwain110y70

Is there a way to view a user's submitted articles from oldest to newest? For example, if I want to read Yvain's articles in chronological order, how would I go about doing that?

0D_Malik10y

1. Go to his first article, then in the "Article Navigation" menu use the "by author" arrows. 2. Go to lesswrong.com/user/Yvain, go to the last page (by clicking "next" or by changing the URL in some way), then go back one page at a time. Haven't tested either of those, but they should work.

[-]G0W5110y60

[-]TrE10y110

Please insert some line-breaks at suitable points to make your comment be more readable. At the moment it's figuratively a wall of text.

Edit: Thank you.

2Gunnar_Zarncke10y

I don't know why this is downvoted so much without an explanation. The problem from the interaction with sub-agents is real even if already known, but G0W51 may not know that.

4gjm10y

I suspect because the Big Wall Of Text writing style is offputting. Perhaps, more specifically, because it's annoying to slog one's way through a Big Wall Of Text without an exciting new insight or something as payoff. (I didn't downvote it. I'm just speculating.)

1G0W5110y

Do you happen to know of any good places to read about this issue? I have yet to look into what others have thought of it.

1NancyLebovitz10y

I've seen not-especially-theoretical discussion about this problem for human organizations, though mostly from the point of view of lower status people complaining that they're being given impossible, incomprehensible, and/or destructive commands. This points at another serious problem-- you need communication (even if mostly not forceful communication) to flow up the hierarchy as well as down the hierarchy. You're pointing at a failure mode (I have a JOB! I wanna do my JOB!) which is quite real, but there are equal and opposite failure modes. For example, obsessive compulsive disorder would be an example of lower level functions insisting on taking charge. Eating disorders are (at least in some cases) examples of higher level functions overriding competent lower level functions. What I'm concluding from this is that if Friendliness is to work, it has to pervade the hierarchy of agents.

0G0W5110y

Remember that making humans in organizations cooperate is a rather different from making the many parts of an AI cooperate with the other parts, because people in organizations can't (feasibly) be reprogrammed to have their values be aligned, but AI HLAs can. True. The real issue is that if you give a lower-level action the same goals and utility functions as the higher ones, you've lost all benefit of having HLAs!.

[-]DataPacRat10y60

Variable-score Iterated Prisoner's Dilemma?

There are a number of variants of the Iterated Prisoner's Dilemma - some with noise, some with random mutations, and so on. Different variants lead to somewhat different most-successful strategies. Does anyone know of any analysis done on IPD tourneys where each round's scoring isn't necessarily the same?

[-]Vaniver10y50

Has anyone seen a "group debate," where a group will decide on one point to put forward, and then another group will spend time coming up with a response, and then the first group will decide a response to that, and so on?

I was thinking of a conversational pattern I've seen, where one person will put forward an odd claim to a group, and then multiple members of the group will ask questions / make counterarguments to try to knock down the claim, but often in an uncoordinated way. That made me think of Kasparov Versus the World, where one player wo... (read more)

[-]mkf10y40

Why exactly did Eliezer stop writing here and started writing his new articles (on intelligent characters in fiction, angel investing etc.) on Facebook or Tumblr instead?

1ChristianKl10y

There are probably multiple reasons. One that I remember him giving was that on Facebook he can simply ban people he doesn't like in a way that he can't here.

1Viliam10y

The nature of feedback was his explanation if I remember correctly. More friendly reactions on other places. Also, on his private pages he can simply remove any reactions he dislikes without worrying and debating about whether he is abusing his powers. I think that LessWrong does not feel for Eliezer anymore like "just me and my friends" (unlike e.g. Facebook), but more like a public place where he is judged, and where people are waiting like hungry sharks (my metaphor, not his) to get some contrarian points by finding some flaw in something he wrote. (Also the online bullying on RationalWiki about "basilisk", where he will be forever criticized for removing some comments years ago. If he removes comments from his Facebook page, there will probably be much less drama; maybe.) Simply, people are behaving less friendly towards Eliezer on LessWrong, and more friendly at other places... so, you know.

-5hairyfigment10y

[-]Adam Zerner10y40

I don't see much brainstorming on LW. Brainstorming seems like a useful and fun thing to do.

2Emily10y

What makes brainstorming specifically different from, say, any given discussion in a comment thread or on an Open Thread?

9Adam Zerner10y

I think the main thing is the confidence level it takes to post something. In most conversations, people only seem to post when they're relatively confident. In brainstorming, you could propose ideas that you're not as confident about. Another thing that is sort of different is the intention. In brainstorming, the intention is to explore different possibilities in search of finding a good solution. In normal discussions, it seems to be mostly about sharing information and discussing the logical inferences people have made.

3NancyLebovitz10y

It might be worth starting a brainstorming thread in discussion, with some rules about lowering the criticism level. It would be like the stupid questions threads.

2ChristianKl10y

We do you think that LW is a good place for brainstorming? What kind of brainstorming discussion would you consider to be useful on LW?

5Adam Zerner10y

Because there are smart people that like to think about (important) things. A lot of stuff. What comes to my mind immediately: * Startup ideas. * Ideas to promote rationality. * Ways to make this site better.

0Elo10y

While I recognise anyone's right to believe that brainstorming is fun; I have never held that belief, never enjoyed brainstorming and never found it useful for anything at all. Perhaps there are others that also do not find it effective (like me) or perhaps no one has championed a thread of it before. All the same - give it a go. Good luck.

[-]MrMind10y30

I'm trying to wrap my mind around Hay's "Universal semimeasures", where he proves some very interesting equivalences.
In order to follow all the derivations, it would be very useful if there was a tool to draw binary trees and do some outlining of the branches, sub-trees, etc. Do you know of anything like that?

[-]NancyLebovitz10y20

Does anyone know why a lot of work has gone in to vegetable-based imitation beef and chicken, but not into good imitation fish?

4pcm10y

In Chinese grocery stores and restaurants, I see about as much veggie fish/shrimp as veggie beef/chicken, and it tastes about as good. But the veggie fish and shrimp take less like real fish/shrimp than veggie beef/chicken taste like real beef/chicken. So it may be that similar effort went into each, and many cultures were less satisfied with the results for fish.

0NancyLebovitz10y

It may be possible to do better vegetarian "fish" with modern technology, but I haven't heard of anyone working on it.

0Elo10y

imitation bacon is common, immitation squid is too. (Although maybe Australia is different) (as well as beef and chicken as you mentioned)

0Viliam10y

I don't know if this could be relevant, but some cultures have a religion that allows them to eat vegetables and fish. Some branches of Buddhism, if I remember correctly. Thus less people motivated to work on fish replacements.

0Elo10y

if you consider ranking living things by their capacity to feel or understand pain, you end up with fish being lower than large land animals. Also humans more often can develop relationships with land animals than fish, while we can develop relationships i.e. with dolfins; we don't eat them often.

0ChristianKl10y

Wikipedia seems to describe it that way: http://en.wikipedia.org/wiki/Buddhist_vegetarianism

[-]advancedatheist10y20

The Age of Adaline, a film about a mysteriously negligibly senescent woman:

https://www.youtube.com/watch?v=clbSd2JzAqc

Let me guess: The unaging woman finds fulfillment by pulling an Arwen or some similar nonsense.

Because Hollywood can't (yet) make films about the utter coolness where ordinary people discover this woman's power, reverse engineer it and become negligibly senescent themselves.

6Izeinwinter10y

Honestly, I think almost all media treatments of this entire topic will be extremely problematic in hindsight once an actual cure for senescence is found. In this particular case, I'd expect her to become... very interested in biochemistry. That would be a much better plot, wouldn't it? One woman's fight to cure ageing because she knows for a fact it can be done, but at the same time trying to not end up strapped to a lab table. Heck, for the first period, the fact that women were massively overlooked in science would be outright helpful. - Getting hired at a place which does whatever she was currently investigating would be fairly simple, and then just have some random dude steal credit for whatever results she manages. Paper trail? What paper trail?

0advancedatheist10y

If this happened to someone in an undeveloped non-Western country that didn't have much contact with the rest of the world, or in a premodern society several centuries back, the character wouldn't have the ideas to think about his or her situation as a scientific problem. But a reasonably intelligent woman who grew up in the U.S. in the early 20th Century would at least know of the existence of a culture of science that could shed light on her condition. This raises the question of whether a nonaging person encountering science after several centuries would have the abiltiy to absorb the implications of this relatively new and unintuitive way of thinking.

2Izeinwinter10y

First option doesn't exist. The third world is well and truely aware that science is a thing. As for the second.. Writing someone who is old, but not impaired by decay is very, very difficult, due to lack of examples, but I think this might be less of a leap than it seems. Necessity will force mobility upon our protag, and contact with various cultures will immunize against believing received wisdom without proof. Going from there to "reality is the final arbiter" isn't much of a leap. .. Now I am trying to think what applicable skills someone really old might have to bring to the project of science, assuming she didn't win the cosmic lottery trice over and is both a genius and highly creative on top of unageing.. "Social-Fu, ninth dan"? Hypercompetency at organizing a group of people into working smoothly together is something which she could with very high plausibility have picked up simply from endless practice. Setting up a carpentry shop in Milan one decade, a china production in venice the next and so on conferring skills that do tranfer quite well to running a lab within budget and with abnormally low social frictions.

0Richard_Kennaway10y

George Bernard Shaw wrote some in his play "Back To Methuselah". It is long; search for "Lutestring" (the name of one of the characters) and read forwards from there. Context: Mrs Lutestring and The Archbishop are, covertly, over 250 years old, and their secret (previously not even known to each other) has just come out. The others in the scene are of ordinary ages and, as far as they know, short-lived. Then search onward for the subtitle of act 5, "As Far as Thought can Reach", set in the year 31,920 A.D. (Shaw's speculative mechanism for life extension can be ignored.) And of course there is Lazarus Long.

0hairyfigment10y

That last part is interesting. But just to note, you're using "several" to mean 'probably more than three, and definitely more than two.'

[-]Calvin10y10

Self Help, CBT and quantified self Android applications

A lot of people on LW seem to hold The Feeling Good Handbook, of Dr. Burns in high regard when it comes to effective self-help. I am through the process of browsing a PDF copy, and it indeed seems like a good resource, as it is not only written in an engaging way, but also packed with various exercises, such as writing your day plan and reviewing it later while assigning Pleasure and Purpose scores to various tasks.

The problem I have with this, and any other self-help-exercise style of books is that I ... (read more)

2ChristianKl10y

I think the problem is a good problem to work on. The potential benefit is huge. The core reason to recommend Burns book over other resources is that he actually run a study to show that the book works. If you on the other hand want to create a completely new product I don't think it makes sense to copy the exercises directly. A book is a different medium than an app and your goal is to optimise for the App medium. The book was written 25 years ago. It feels dated. That's before we had Martin Seligman campaigning for positive psychology. If I remember right Burns book lacks gratitude exercises. I also consider it very helpful to locate emotions in one's own body and be aware of them on a kinesthetic level and I don't think that thought was in Burns book. Newer CBT books might also provide good input.

[-][anonymous]10y10

Cold/flu, I hate having to spend a week in bed every 8 weeks or so. Any less-known tips?

For example I thought that dealing with ear pain with a cup of pan heated salt poured into a sock and held against it (increases blood circulation and somehow that helps) is pretty universal, but just yesterday it turned out an Austrian pharmacist never heard about it. It may be new to you as well. As a traditional folk remedy, it provides very quick symptomatic belief, works in about 10 mins, but if the pain returns the next day, doctor.

For the record, the known tip... (read more)

6NancyLebovitz10y

I'm inclined to think the two month cycle is unusual. Are there other people in your area who are also getting colds every two months? Is there anything else in your life that runs on a two month cycle?

2[anonymous]10y

Agreed. Especially if it’s the flu. “Is it flu or cold symptoms?”… http://www.webmd.com/cold-and-flu/cold-guide/flu-cold-symptoms?page=2 Bedridden and unhappy 11% of life is an extremely high cost - I would spend the money to see a doctor.

0[anonymous]10y

No idea and no.

5[anonymous]10y

When I was a child, I had this. A 6 week cycle of cold -> sinus infection which required antibiotics to normalize, 5 weeks of normalcy, then another one. I didn't grow at all during 1st grade due to constantly being sick. I had bad adenoids - they're like tonsils but in the back of your nose. They are supposed to shrink away to nothing as you grow but mine were hypertrophic and blocking part of the drainage and thus causing constant infections. I had them removed in a quick and easy surgery in second grade, the cycle ended instantly, and I caught up with my classmates quite quickly.

2hyporational10y

Don't get it in the first place. Take care of good hand hygiene and don't pick your nose or rub your eyes and watch where you put your hands in public places. Don't get sleep deprived or stressed and don't exercise excessively so that you don't compromise your immune system. Avoid people who you know are sick, avoid shaking hands or wash your hands afterwards. If you get a cold very frequently or it is always prolonged despite of taking precautions check for asthma and allergic rhinitis and get those treated.

1[anonymous]10y

Hm, I thought people get it mostly from the air, via other people sneezing? I must admit I am guilty of picking my nose as I hate it when it is full, and doing that with not even having washed my hand after grabbing everywhere on the public transport. But I did not think cold/flu virus comes from the hand, I thought it only works with droplet infection from the air, such as people sneezing. I would rule asthma out - it would pretty much make exercise impossible, wouldn't it? My cardio is not too bad for my weight and with that kind of problem it should be. Allergy - good point, I actually told the doc that one of the medicine I took (probably silver-protein + ephedrin) resulted in allergic skin rash, and then the doc said if I am unlucky, the symptoms of medicine allergy and the cold can just as well add up, as they are similar.

2hyporational10y

Nope. You can use normal saline and oil sprays to get your nose clean so you don't have to pick it. Or pick it with a clean tissue. I doubt drug allergies and environmental allergies correlate in a meaningful way.

2ChristianKl10y

If you get cold every 8 weeks the obvious thing is to get vaccinated against the common cold. The second thing is to get enough Vitamin D3. Either naturally through spending time in the sun or through a supplement. The fact that it provides quick symptomatic relief doesn't mean that there isn't risk involved by putting something very warm near your ear.

5NancyLebovitz10y

There is no vaccine against the common cold. I'm not sure that the heated salt is hot enough to be worrisome. What ill effects do you think are likely?

2ChristianKl10y

Ah, you are right. Still there's a flu vaccine. I don't have specific concerns, but in general heating up a region under pain seems to be an idea that could be within standard medicine and it doesn't seem to be standard practice. If it would be useful, why is there no big pharma company that sells a creme that heats up ears and markets it?

1Nornagest10y

I've never heard of the salt/ear thing, but warm compresses are fairly standard advice for certain types of eye infections. I believe the main goal there is less to increase blood circulation and more to soften oils that might be blocking the glands in the area, though. Physical heating is something that pharma companies aren't really equipped to monopolize, but I have seen microwavable hot/cold packs. They don't seem to offer much advantage over a warm towel.

1[anonymous]10y

Because a cup of salt costs something like cents, it would be very difficult to compete with it.

1ChristianKl10y

Nobody runs clinical trials to show that the cup of salts has ideal properties. Running clinical trials that a certain creme helps the ear on the other hand puts the treatment into the 'evidence-based' medicine bucket. See Scott Alexanders discussion of Melatonin vs. Ramelteon.

0[anonymous]10y

Of course nobody does it, there is no business opportunity in it.

0hyporational10y

Nasal irrigation seems to have been pretty successfully commercialized, so I suppose you could commercialize heated salt and run trials with some inventive marketing.

0ChristianKl10y

But then you would likely sell your heated salt at a higher price point and not for the price of ordinary salt.

0NancyLebovitz10y

Businesses don't necessary notice every opportunity quickly, or even at all.

0ChristianKl10y

There a lot of money in big pharma and there are solutions to heating up areas of the body developed for people with tense backs. It's no absolute heuristic but it looks to me like a topic that's open to big pharma intervention. That means there likely research out there that gives a better idea about whether it's something worthwhile to do then simply following folk remedies.

0Good_Burning_Plastic10y

Well, there are these: https://en.wikipedia.org/wiki/Heating_pad

0ChristianKl10y

It's my understanding that the prime use of heating pad's is treating tense muscles. Mostly for people with tense backs.

0NancyLebovitz10y

Heat and cold also get applied to sprains.

1Adele_L10y

Worth noting that you can easily obtain N-Acetyl Cysteine (e.g. from Amazon), which is an effective antidote to paracetamol toxicity (and the mechanism suggests it can be taken preventatively).

2Dorikka10y

Perhaps good to have on hand as first aid, but are you suggesting taking more acetaminophen and trying to protectt your liver with the cysteine? Seems high risk, low reward if so.

0Adele_L10y

Not suggesting anything like that at all, just a good and interesting thing to be aware of. Especially good to have on hand if you have kids, and I think it's probably good to take it with normal doses of paracetamol.

0ChristianKl10y

I don't think a mechanism suggesting that it can be taken preventatively is enough to suggest that it should be taken. It's a separate drug with it's own side effects. Before taking it preventatively I would like to see studies that suggest that it's benefitial to take it in that fashion.

1polymathwannabe10y

Actually, vitamin C does nothing for the flu.

0NancyLebovitz10y

The recommendation was for D3, not C. More generally, working on improving health all the time is a somewhat different project than trying to get better when you're sick.

2polymathwannabe10y

I wrote in reply to DeVliegendeHollander's vitamin C comment, not to ChristianKI's D3 comment.

0Elo10y

regular exercise. Should help keep your immune system stronger. (sorry I don't have a source, but I am sure there are papers out there)

0Sjcs10y

This is incorrect. Normal paracetamol dosing is less than half the toxic dose of paracetamol, and it is an incredibly safe drug at these levels. Ibuprofen however has rare but well know side effects of gastric irritation, ulceration and life-threatening haemorrhage

0[anonymous]10y

Yes, that is the point. It is incredibly easy to go over double the normal, if you are irritated with symptoms. I know people who take three aspirin pills at once because they really want to throw a nuke on that hangover. A typical coldrex pill is 4 pills a day. It is far too easy to not even read it (it is OTC so "safe" right???? they would make it prescription if it was dangerous right???? <- this is a common logic used) and pop a pill every hour.

0NancyLebovitz10y

Unfortunately, paracetamol is included in some multi-drug combinations, so it's relatively easy to overdose by accident.

[-]Ersatz10y10

A quote for those of you who are interested in decision theory, taken from Counterfactuals and Newcomb's Problem (sorry, though: gated link):

My second observation is closely related to the first. It is true that the state of box 2 is not causally dependent upon my act, in Newcomb's problem. But another kind of dependence relation obtains which is at least as important as causal independence and which seems to take clear precedence in the limit case. If I am completely certain that the being has predicted correctly, then the state of box 2 depends logical

... (read more)

[-]cleonid10y10

Recently several people suggested opening a separate website for rational (or Less Irrational) discussion of political issues. If such a website is created, will you be interested in participating?

[pollid:845]

[-]Vive-ut-Vivas10y110

Slate Star Codex fulfills this niche for me.

[-]MathiasZaman10y170

The lack of up- and down-voting and the limited threading kills it value for me, personally.

4Artaxerxes10y

Cross post the ones you want to discuss here.

4closeness10y

Yes, who's going to go through hundreds of comments to find the good ones. And based on the content they're replying to, there's probably some really good stuff in the comment section that's relatively unseen because of this.

9Dahlen10y

1) Will its name be another pun on "Less Wrong", like it happened with More Right? 2) I still don't understand why it wouldn't simply be easier to create more subreddits for LW on different discussion topics, like it has been proposed a billion times in the past, as opposed to more and more websites springing up.

7Viliam10y

Doing any changes in LW software (based on Reddit software) is very hard. The software has a nice functionality, but it is a maintenance nightmare. But we could just create a new subreddit on Reddit, and link it from here. (My preference is that it should not contain "lesswrong" or "rational" in the title. Just in case it goes very wrong.)

0Dahlen10y

I understand. That's bound to happen anytime people request major features; coming up with cool new ideas is one thing, implementing them is quite another. However, it's among the most popular feature requests I've noticed so far. Having themed subreddits would improve functionality for many users, and I'm not even talking about the specific suggestion in cleonid's comment. How many hours/days of coding or testing are we talking about? (BTW, new account?)

[-]Viliam10y120

How many hours/days of coding or testing are we talking about?

I tried it once. In theory, it should have been super easy. In practice, after a few days of trying to make the damned software run on my computer, I gave up. Of course if I can't make it run, I cannot test any changes I make.

I think the code is freely available, so if you feel heroic, try it at home. Or use the Reddit code, which is almost the same thing.

I believe it would be easier to rewrite the whole thing from scratch. Give me a year's salary, and I would be happy to rewrite the whole thing in Java, and then to maintain in. Any other way just seems more difficult. :(

new account?

Yep, I realized I write too much about my work. I do not suspect my colleagues to read LW, but they could still put my name on google and find something work-related in top results. So I am slightly reducing the risk in the future.

1NancyLebovitz10y

Could this be worth a kickstarter? My feeling is maybe yes for LW, and obviously possible if redditors would want a modifiable version of their code.

3Viliam10y

I would love to try it. The problem is, I never did a project of this size, so I don't know if I am overconfident. I think that when I wrote "year", that already compensated for the planning fallacy (my first reaction was something like "three months"). Or, maybe I spend the whole year doing this, and at the end we find out that the project is ready and works, but maybe it is much slower than the original version. Etc. What would be a good solution? Maybe first build a smaller prototype, measure the speed, and decide whether to move on. But this careful iterative programming does not seem compatible with the kickstarter approach. So, uhm, maybe the small prototype first, and then the kickstarter?

1Kaj_Sotala10y

Is rewriting everything from scratch obviously better than forking some other existing forum software (e.g. Discourse) and adding features?

5Viliam10y

That of course depends on how much that "other existing forum software" is complicated, and how many features need to be changed to fit our needs. To answer your question, it is not "obviously better"; you just have to make a probabilistic estimate. Sometime people make the mistake of reinventing the wheel, when a good (or even superior) version already exists. But sometimes also people spend a lot of time investigating existing options, only to find out that all of them have some major problem. (And this process can also cost a lot of time, and when it fails, you are left empty-handed at the end.) I have seen both of these situations in real life. The "don't reinvent the wheel" part gets very popular on internet, but people underestimate that even when a solution is already available, it can take a lot of time to learn how to use it properly (that means, not just use the software as an ordinary user, but also as an administrator, and even to be able to customize it; those are an order of magnitude more complex tasks). I guess a reasonable first step would be to set a deadline, for example a month or two, to explore the existing alternatives. (But you have to set a deadline, because there will always be yet another obscure alternative you have not thoroughly examined yet.) Maybe a month would be enough, if we consider the fact that LW already runs for a few years and it was not replaced by an existing superior alternative.

1NancyLebovitz10y

I know even less about this sort of project than you do. My level of ignorance is shown by taking your claim that you could do it in a year at face value and only thinking about whether you could raise the money. A test project sounds very reasonable. So does thinking about where you could go for competent advice about the difficulty and the risks of the project turning out badly.

0Dahlen10y

I see. Too bad though; maybe future attempts, by you if you wish to try again or by others, would go better. I wish I could reasonably undertake this myself, but 1) I never learned much coding beside the basics; 2) I remember maybe 5% of what I did learn; 3) these days I'm less motivated than a baby panda on Valium.

0gjm10y

What about a baby panda on Viliam?

1[anonymous]10y

I don't think lesswrong has enough volume for it actually... it would spread out the discussion too much.

7gwillen10y

Easier for whom? The people making the proposals are not the same people who have the power to edit the LW site code, and my sense is that the people who have that power are generally no longer interested in (and/or do not have time for) making any significant use of it.

3cleonid10y

2) We are thinking of creating a new format for the discussions (based on the principles of collaborative filtering) that will be less vulnerable to various mind-killing mechanisms.

1Dahlen10y

How would it differ from LW-style upvotes & downvotes?

0cleonid10y

Among other things, it will include an individualized recommendation system. In a political debate, the regular upvotes & downvotes system has many problems (for instance, it encourages partisanship).

0polymathwannabe10y

Maybe "Frozen Mind," both as an awkward synonym for "cold head" and as opposed to "Mind-Killed?"

4gjm10y

For me this (1) suggests killed rather than unkilled minds and (2) also suggests something to do with cryonics. Too distracting, I think. [EDITED to fix an obvious typo.]

7Vaniver10y

There was, at one point, an invite-only mailing list to discuss mind-killing issues. It did not see much use, and I am fundamentally pessimistic about forums that are open to the public that discuss politics.

0Elo10y

as a definite no voter, the usefulness is not aimed at me, thats fine. you won't see me there. But it looks like there exists some interest.

[-]skilesare10y00

I've been working through the sequences and have been trying to apply things to my current project that has to do with remaking money. I'd appreciate any feedback. Did I miss anything major? Am I on the right track to fix my thinking?

http://www.hypercapital.info/news/2015/4/9/hypercapitalism-what-you-think-about-money-is-wrong

2NancyLebovitz10y

I found the article fairly irritating because it's just a claim that you might have a good idea, but it doesn't describe the idea. I followed links to find out what your idea is, and I'm quite dubious about it-- I think the problem you're trying to solve is probably the wrong problem. I think your issue is capitalists getting too much money from employees and customers, and then doing something profitable with the money and not sharing the profits. You don't (I may have missed something) cover the case of capitalists doing something unprofitable with the money, which may be an honest experiment or a stupid mistake, but in any case, there are no profits to share. Excuse me if I've missed things you've covered, but what will enforcement look like?

0skilesare10y

Great point. I need to flesh out the exact process. The high level solution is that we can 'fold the blockchain.' Since we have a record of where money flowed, when an entity fails, we can fold the inputs and connect them to the outputs. N1 sends cash to N2. N2 wastes it on a bad idea spending money at N3,N4, and N5. The great thing about 'money' is that N3,N4, or N5 have a new chance to do something 'valuable' with it. If N2 fails, we can 'fold the blockchain' and pass through the benefits from 3,4,5 back to 1. Money doesn't disappear...it generally flows some where else where the next person in the economy has a chance to create a recursive value engine. In the short run, the system is not very different than today. Short term profit motives are virtually unaffected. In the long run there is a value incentive and reward for proactively finding value. The theory is that it is possible to do both.

[-]skilesare10y00

I'd love some feedback on the latest update on my kickstarter. Would anyone be willing to make sure I haven't misstated the math and/or concepts?

Hypercapitalism: Information Theory and Bayes

https://www.kickstarter.com/projects/hypercapital/hypercapital-experiment-testnet-apis/posts/1190876

[-]Punoxysm10y140

I think you have failed to consider the requirements of Kickstarter. You need to really think about your marketing, deliver as clear and slick a statement of your project as possible, and imitate as much as you can of successful similar kickstarter projects.

Your video is neither clear nor slick (verbal explanation with random camera angles is a poor way for most people to absorb information), and needs visual aids at least.

I watched a couple minutes of explanation, and then zoned out. Sorry.

I'm guessing from your low funds that you've also done little to evangelize your concept, or have been unsuccessful in doing so.

I also think Kickstarter for coding projects is a high barrier, since so much programming is volunteer open-source projects, people wonder why they'd donate extra. And you offer no rewards for backers, a key element of Kickstarter's concept.

You also say in the first paragraph "the project isn't going well".

0skilesare10y

Yes...the kickstarter portion is a bit of a mess. I threw it together at the last minute when I was invited to pitch my ideas. I'm more to net rested in feed back on the math and concepts behind it. I think I understand bayes and am trying to apply it. This seemed like the right place to seek guidence.

5Punoxysm10y

After looking at the website, it's still a very complicated idea, it's totally unclear what a small-scale implementation is supposed to look like (i.e. what impact does this have in a small community of enthusiasts, what incentive does anyone have to adopt it, etc.). I can't evaluate your math because I don't see any math in your website or kickstarter. I see references to Bayes and Information theory, but no actual math.

0skilesare10y

Yes , this is the issue. I'm deficienct in the math area. I'm working on an out line to lay out the math, but it has been a long time since I took diffeq and it is coming back slowly. Thanks for the feed back.

[+]Xerographica10y-170

[+]Xerographica10y-200

Moderation Log