LESSWRONG
LW

Comment Permalink
dkirmani20

Could you please list the likely points of failure in the narrative I posted? I'd like to debug it.

The failure mode that stands out to me is "the American voter simply doesn't give a shit about what the prediction markets say". In case of this failure mode, then thank God for China. (and El Salvador, and Vietnam, and Singapore, and ...)

But I don't think the median voter would be so indifferent. "Put your money where your mouth is" is common folk wisdom that everyone understands. Besides, if anyone can create a market for like $10, people could ask the markets questions like "Is my wife cheating on me?" and "What are the odds of my daughter getting into Harvard?", which I expect people would, in fact, care about.

Reply
See in context
by Yair Halberstadt
1 min read
A
529

35

I sometimes get a little bit melancholic when I read about Dath Ilan. Humanity could be so much better off, but due to economic illiteracy, we can't have nice things.

Better to light a small flame than to curse the darkness. So if you had to build a model econoliterate city on earth, how would you go about it?

There's lots of micronations projects out there. What I'm interested in is:

  1. How do you keep the city econoliterate, immediately and a hundred years in the future.
  2. How do you make this practical to achieve - most micronations projects dont go anywhere, and are actively thwarted by governments.

I will not cease from mental fight,
Nor shall my pen sleep in my hand:
Till we have built Dath Ilan,
In this earths green & pleasant land.

Dath IlanWorld Optimization
Frontpage

35

How would you build Dath Ilan on earth?
New Answer
New Comment

5 Answers sorted by
top scoring

dkirmani

*

370

Tl;dr: Legalize prediction markets, make civilization adequate.

  1. Spend a lot of money on ad campaigns and lobbying, and get {New Hampshire/Nevada/Wyoming/Florida} to nullify whatever federal anti-gambling laws exist, and carve out a safe haven for a serious prediction market (which does not currently exist).

  2. You could alternatively just fund the development of a serious prediction market on the Ethereum blockchain, but I'm not as sure about this path, as the gains one could get might be considered "illegal" in some jurisdictions. Also, a fully legalized prediction market could rely on courts to arbitrate market resolution criteria.

  3. In order for a prediction market to be "serious", it has to allow epistemically rational people to get very very rich (in fiat currency) without going to jail, and it has to allow anyone to create and arbitrate a binary prediction market for a small fee. Such a platform does not currently exist.

  4. If you offer a large monetary incentive for people to be epistemically rational, they will become more epistemically rational.

  5. Regular people will be able to Google questions like "what are the odds that Donald Trump will bring gas prices below $3/gallon in his first year if he's elected" and "what are the odds that Proposition 23, if passed, will reduce unemployment in the next six months". Normal people will actually be able to get real answers to these questions.

  6. Political incentives shift. Legislation is now more likely to pass if it achieves its stated goals, so effective legislation now actually gets written. Presidential candidates that the markets say will improve {GDP/unemployment/poverty/violent crime} have an advantage over their opponents. Prediction markets let people call politicians on their bullshit, so they can't afford to bullshit as much anymore.

  7. A prediction market hosted in the USA can also predict foreign elections. Decision markets on China's economic growth over Xi Jinping's successor emerge, and the Central Committee pays attention to them. American decision markets are a global public good, providing priceless intelligence to government officials of all nations. The honest conditional forecasts provided by prediction markets make world governments more effective and less Mazelike.

  8. Institutions become adequate. American healthcare costs 10x less and works much more smoothly. Land value taxes are phased in, income taxes are phased out, a voucher system replaces public schools, government research grants actually fund innovation, the economy booms, the median salary grows, millions are lifted out of poverty, et cetera. It is very hard for me to forecast what an actually effective government would look like (because it's hard to predict something smarter than me), but I'm sure it would be good. Not exactly Dath Ilan, but in the right basin of attraction.

[-]MondSemmel100

I'm highly dubious that prediction markets would fix all our problems by themselves. At the very least, we'd need to go through several generations of prediction markets to work out all the inevitable problems that would come up.

Which we can't do, as you've pointed out, when they're currently effectively illegal to experiment with.

Reply
[-]dkirmani100

Oh, for sure the first generation of prediction markets won't be enough. However, market forces will automatically take care of iteratively improving prediction market platforms for us (where "us" = "people concerned about x-risk"), and all "we" need to do is push to make them legal.

As long as entrepreneurs are allowed to make money by building better prediction markets (+ adjacent services), they will.

Reply
2dkirmani
Could you please list the likely points of failure in the narrative I posted? I'd like to debug it. The failure mode that stands out to me is "the American voter simply doesn't give a shit about what the prediction markets say". In case of this failure mode, then thank God for China. (and El Salvador, and Vietnam, and Singapore, and ...) But I don't think the median voter would be so indifferent. "Put your money where your mouth is" is common folk wisdom that everyone understands. Besides, if anyone can create a market for like $10, people could ask the markets questions like "Is my wife cheating on me?" and "What are the odds of my daughter getting into Harvard?", which I expect people would, in fact, care about.
3MondSemmel
Regarding issues with your narrative (split into a separate comment): * Institutions are inherently conservative and don't like to change. Any particular innovation is unlikely to transform society, and any particular story for how the future will turn out a particular way is unlikely to come to pass. * For correct forecasters to get very rich on prediction markets, either someone has to provide liquidity, or there have to be "suckers at the table", i.e. people who invest money for irrational reasons. If it's the former case, where is this money coming from? And if it's the latter, why are those people spending money on your prediction market rather than a lottery or something? * Also, it takes a lot of time for the money to go from bad forecasters to good ones. * And the losers have incentives to lobby against prediction markets. Anti-gambling laws didn't write themselves. * Prediction market forecasts are probabilistic. So what if a politician's policy turns out badly, and the prediction market predicted that outcome at 90%? The politician will ignore the forecast unless forced to confront it (which requires the prediction markets to have already accumulated credibility and trust among voters - how did that happen?), in which case they can criticize it as far too pessimistic ("I lost a 1/10 chance, not a 9/10"), or claim sabotage ("someone bet against me and then sabotaged my policy"), or find one of a myriad other excuses. * Also, prediction markets will inevitably get caught in partisan crossfires. When the markets predict good outcomes for their side, a voter might consider themselves a truthseeking advocate of prediction markets, and if the markets predict bad outcomes for their pet policies, they'll suddenly accuse the markets of bias or manipulation or favoring the other side. * And so on.
9dkirmani
True. There are plenty of suckers at the table in the realm of securities trading. Because finance bros and tech bros know that Lotteries Are For Poor People, while Prediction Markets, much like day trading crypto, are the kind of thing that sigma males like them use to get rich at the expense of irrational suckers (who totally aren't other guys exactly like them). If I recall correctly, high frequency trading is almost entirely zero-sum, and there's plenty of energy behind it anyway. The same criticisms apply to regular securities trading. To be fair, there are plenty of losers who hate securities trading, and wish it was illegal. This is true. However, voters don't choose policies directly, only their representatives. If the market expectation of a candidate's overall performance is upstream of voting patterns, then those candidates are forced to optimize for those expectations. Yes, they will. The cool thing about prediction markets is that if you disagree with them, you can just bet against them and win money. Put up or shut up.
1servy
Suppose, there is a prediction market for a question like: "If Alice is elected president, will GDP grow more than 20% by the end of the next 4 years?" Current bets are 10 to 1 against Alice succeeding if elected. I strongly disagree, so I would like to bet $5000 on Alice and win a lot of money. Alice does not end up being elected, the prediction market probably being largely responsible for this outcome. So, the outcome is unresolved, I'm angry and frustrated since I was not able to make money, nor got my preferred politician got elected, and I lost some opportunity cost due to those $5000 being committed to this bet for some time. So I vote against prediction markets next time since those are obviously an evil plot to keep Alice from becoming president and fixing things. Maybe, this can be somehow amended if I could instead bet on the effects of some policy that Alice endorses and I find compelling, and it is possible to test this policy on a smaller scale, and we could allocate some money from this presumably large betting pool for actually testing it, and we could somehow guarantee that those who bet lots of money won't be able to influence this test much. There are a lot of "ifs" here, and this seems to require way more than just legalized prediction markets. I don't immediately see how legalizing prediction markets necessarily leads to incentives to legalize and make it possible to easily resolve something like that.
2dkirmani
I expect future iterations of prediction market platforms to allow your money to accrue interest over the course of longer-term bets. You don't have to convince the whole country. Not even close. As long as you get New Hampshire to legalize prediction markets, New Hampshire can tell the federal government to go fuck itself. Weed is still federally illegal, remember?
3MondSemmel
It hasn't escaped the notice of forecasters that prediction markets aren't free of issues. For instance, check Scott's Mantic Mondays posts on ACX. For one thing, while there are many issues (like legal ones) which currently impede popularity of prediction markets, it's nevertheless the case that so far none of them has particularly caught on on a larger scale. Then there's stuff like transaction costs making them less useful (if there are transaction costs of 5%, markets won't drop below 5% or rise above 95%), resolution risk being a big problem (questions you can easily arbitrate are often not the ones you care about; but if you can't arbitrate the ones you care about, you can't make a prediction market for them), lack of liquidity (another way for a prediction market to flop is for it to experience no uptake by users), and so on. None of these issues seem fundamentally unresolvable. But conversely, we cannot currently point at any existing prediction market and say that "if we legalize this market and scale it up, it will transform society", or something. As for market questions like "is my wife cheating on me", I'm extremely dubious that even if you managed to get prediction markets legalized, those kinds of questions would be allowed.
2Peter Hroššo
This is actually already possible right now on https://manifold.markets/ Even though, the market uses play, instead of real money, but you get at least something.. Otherwise I completely agree with your critique of current prediction markets, and I agree that none of the issues seem fundamentally unresolvable. Actually, I'm currently starting a new project (funded by FTX Future Fund) which aims to do just that! Sorry for the shameless plug, but here is a description of the project, and while I'm at it, I'm looking for a cofounder ;)
2MondSemmel
I'm aware of Manifold Markets. The only reason the site can be lax with regards to which markets and resolution criteria it allows is precisely because it only uses play money.
[-]Jackson Wagner90

For some more detail on what this plan might look like, my 2nd-place-winning entry in the Future of Life Institute's "A.I. world-building" competition, was all about how humanity uses prediction markets and other new institutional designs to increase its level of civilizational adequacy, becoming strong/wise enough to manage the safe development of transformative AI.

See my lesswrong post here (which focuses on the details of how AI development is controlled in my team's fictional scenario), or the whole entry here (which includes two great short stories by... (read more)

Reply
[-]Rochambeau60

This seems like the kind of thing that could be hosted in a place like Próspera. I believe the ZEDE law was very recently repealed, but I have yet to hear that the Próspera project is stopping and the constitutional amendment could be repealed next year. Próspera claims it will be proceeding and international law/treaties and it's 50 year stability agreements protect it. https://prospera.hn/news/press-releases/built-to-last-legal-stability-in-the-zede-framework

It seems too early, but if Próspera survives this most recent political shift in Honduras, then i... (read more)

Reply

iceman

300

Mu.

The unpopular answer is that Dath Ilan is a fantasy setting. It treats economics as central, when economics is really downstream of power. Your first question implies you understand that whatever "econoliteracy" is, it isn't a stable equilibrium. Your second question notices that governments are powerful enough to stop these experiments which are a threat to their power.

My background assumption is that any attempt at building prediction markets would either:

a) ...have little effect because it becomes another mechanism for actual power to manipulate procedural outcomes, most likely through selective subsidies, manipulation of the monetary supply, or education or social pressure resulting in all right minded people voting the way power centers want (ie, how things work today).

b) ...be used as a coordination points for a Point Deer Call Horse style coup (see also: how publicly betting in cockfights can be more about signaling alliances, not predictions).

c) ...devolves into Jim Bell's Assassination Markets because there actually isn't a way for power elites to prevent some markets from being made (and we should expect any general way to prevent some markets being made to go back to (a)).

Quintin Pope

140

Honestly, I think the easiest way to do what you’re asking is to solve alignment, build an aligned AGI, and ask for it to help you.

Partially, I think this because I think alignment is actually a lot more tractable than most people assume. The other reason is that “build real Dath Ilan, but real and stable” seems like a strictly harder version of the alignment problem.

To build a stable version of Dath Ilan, I think you (roughly) need to establish a environment of microeconomic incentive gradients that robustly select for agents that are cooperative, powerful, friendly and rational. I think this is a similar problem to establishing literal gradients that select for the same in the circuits of an ML system.

However, doing so in a real economy is much harder because you have less control over a real economy and because other people can interfere with your efforts.

[-]dkirmani*80

My inner simulation of Yudkowsky says, roughly:

  • Alignment is Very Hard and will not be solved in our world, so we must Die With Dignity.
  • Alignment probably would end up being solved in Dath Ilan (Yudkowsky has claimed this is >97% likely).
  • We don't live in Dath Ilan, boo hoo.

So maybe the easiest way to solve alignment is to fix civilizational inadequacy first, and then let our adequate civilization figure out the rest.

And maybe the easiest way to solve civilizational inadequacy is by legalizing prediction markets.

Reply
8gjm
My understanding is that dath ilan hasn't solved the alignment problem but has solved the problem of getting people not to try to build AIs powerful enough to be problematic if unaligned. Apparently mostly by keeping the idea of powerful AIs secret from most of the population, which seems absurd to me. I am not sure why Eliezer thinks dath ilan could do that more effectively than it could have them know that it's possible and dangerous and then Do Something That Is Not That; both are obviously impractical on earth but I think the claimed superior abilities of dath ilan help much more with the latter than with the former.
9dkirmani
* Prediction markets would lead to good AI timeline forecasts. * It is easy to bet on p(doom), as long as doing so is legally supported. If Alice believes that the world will end in five years, and Bob does not: * Alice and Bob sign a contract. * Bob gives Alice $50k. * If the world is still around in five years, Alice must give Bob $200k. * Therefore, prediction markets will forecast the apocalypse. * Decision markets will evaluate {legislation/executive orders/candidates} with respect to their effects on the apocalypse-forecasts. * Therefore, prediction markets will enable coordination against building AGI.
2Victor Novikov
My understandings is that dath ilan has the cilivizational cooperation to not build AGI or increase its civilizational technological capacities until the alignment problem is solved, which means it can take as long as it needs to solve the alignment problem. No "race against time" issue. I think it's fairly obvious there is a secret goverment project working on AI alignment.
1JBlack
It has been explicitly written into one of the dath ilan isekai stories that there is indeed a project in dath ilan to both solve AI alignment and to suppress further computer technology progress until after alignment is fully solved. Edit: More than that: it is essentially the central project that defines much of the course of civilization.
0dkirmani
This is not obvious to me, although I wish it were the case. How do you think you know this?
7Victor Novikov
Sorry, I meant in that is the case in dath ilan.
1AprilSR
I think the number of dath ilani who think of the concept of AGI is plausibly pretty substantial, but they just go talk to a Keeper about it.

Dagon

90

I think you'd need to start with different humans.  Or a filter to remove a whole lot (billions) of the least-capable-of-cooperation existing humans.  It's not JUST economic illiteracy, it's actual misalignment and short-term limited-moral-circle goals.  

I don't know if this necessary condition (better people) is sufficient.  It may still be impossible without significant baseline changes in the individuality and very deep-seated psychological drives in most people.  

iceman

30

[The comment this was a response to has disappeared and left this orphaned? Leaving my reply up.]

But there's no reason to believe that it would work out like this. He presents no argument for the above, just pure moral platitudes. It seems like a pure fantasy.

As I pointed out in the essay, if I were running one of the organizations accepting those donations and offering those prizes, I would selectively list only those targets who I am genuinely satisfied are guilty of the violation of the "non-aggression principle." But as a practical matter, there is no way that I could stop a DIFFERENT organization from being set up and operating under DIFFERENT moral and ethical principles, especially if it operated anonymously, as I anticipate the "Assassination Politics"-type systems will be. Thus, I'm forced to accept the reality that I can't dictate a "strongly limited" system that would "guarantee" no "unjustified" deaths: I can merely control my little piece of the earth and not assist in the abuse of others. I genuinely believe, however, that the operation of this system would be a vast improvement over the status quo.

Bell's organization acts as (a) where Bell's organization can dictate who is and is not a valid moral target. If we are talking about purely anonymous uncontrolled markets (and I assume we both are, since I separated them from (a) and you're referring to anonymous markets on Ethereum), then we should instead expect them to be used to usher in hell.

4 comments, sorted by
top scoring
Click to highlight new comments since:
[-]Shmi220
  1. Look into what has been tried (and failed) in the past.
  2. Identify apparent reasons they failed.
  3. Figure out the underlying patterns of failure.
  4. See what can be done to change these patterns.
  5. Check if that idea has been tried already (it most likely has, truly original ideas are rare).
  6. Repeat 1 to 5 until you are satisfied that you have a truly original approach never tried before.
  7. Write several pre-mortem scenarios.
  8. Consider ways to mitigate the failures in 7.
  9. Repeat 7 and 8 until you are out of ideas for failure scenarios.
  10. Repeat 7..9 by engaging other people, highly motivated to poke holes in what you come up with.
  11. Write a multi-player simulation of the latest and greatest and watch it fail in new and unexpected ways. 
  12. Repeat 11 until something resembling the Utopia you hope for emerges. 
Reply
[-]gjm100

It seems worth pointing out that dath ilan isn't just econoliterate and its advantages over earth aren't supposed to be consequences just of much greater economic literacy. In particular, they depend on 1. a population that is smarter than that of earth, and 2. widespread appreciation among that population of a way of thinking that is mathematically valid and automatically motivates those who understand it to act in better-coordinated ways than we do.

#1 is imaginable but obviously difficult to achieve. I'm not quite sure whether Eliezer thinks the way of thinking in #2 exists (and equals "timeless/updateless/logical/functional decision theory") or not, but to me it is not at all clear that any way of thinking actually has those properties.

Reply
[-]Raemon30

I think part of the deal with Dath Ilan is that there was a deliberate eugenics program to bring about #1 (I'm not sure if the world is supposed to have a single-point-of-departure from earth or be quite different in many ways, and whether the baseline intelligence was higher)

Reply
[-]JBlack20

It's deliberately vague, and there's a lot of authorial leeway due to that screening off of all but the most recent history in the setting. The continents correspond and the humans are indistinguishable on casual observation from Earth humans physically, medically, and behaviourally, so it should be some sort of alternate history Earth. Whether any divergence is 200 or 20k years ago is hard to say (and Eliezer may not yet have decided).

Reply
Moderation Log
More from Yair Halberstadt
View more
Curated and popular this week
A
54