All of Liam Donovan's Comments + Replies

What do they have against AI? Seems like the impact on regular people has been pretty minimal.  Also, if GPT4 level technology ws allowed to fully mature and diffuse to a wide audience without increasing in base capability, it seems like the impact on everyone would be hugely beneficial

4cousin_it
They think (correctly) that AI will take away many jobs, and that AI companies care only about money and aren't doing anything to prevent or mitigate job loss.
3Michael Thiessen
In my non-tech circles people mostly complain about AI stealing jobs from artists, companies making money off of other people's work, etc. People are also just scared of losing their own jobs.
1omegastick
Some people have strong negative priors toward AI in general. When the GPT-3 API first came out, I built a little chatbot program to show my friends/family. Two people (out of maybe 15) flat out refused to put in a message because they just didn't like the idea of talking to an AI. I think it's more of an instinctual reaction than something thought through. There's probably a deeper psychological explanation, but I don't want to speculate.

In an impure sample you would see high residual resistance below Tc

Don't the authors claim to have measured 0 resistivity (modulo measurement noise)?

3Muireall
From the six-author paper: "In the first region below red-arrow C (near 60 °C), equivalent to region F in the inset of Fig. 5, the resistivity with noise signals can be regarded as zero." But by "noise signals" they don't mean measurement noise (and their region C doesn't look measurement-noise limited, unless their measurement apparatus is orders of magnitude less sensitive than it should be) but rather sample physics—later in that paragraph: "The presence of noise in the zero-resistivity region is often attributed to phonon vibrations at higher temperature." The other papers do seem to make that claim, but for example the April paper shows the same data but offset 0.02 Ohm-cm on the y-axis (that is, the April version of the plot [Fig. 6a] goes to zero just below "Tc", but the six-author arXiv version [Fig. 5] doesn't). So whatever's going on there, it doesn't look like they hooked up their probes and saw only the noise floor of their instrument.

In the MIRI dialogues from 2021/2022 I thought you said you would update to 40% of AGI by 2040 if AI got an IMO gold medal by 2025? Did I misunderstand or have you shifted your thinking (if so, how?)

What do you think are the strongest arguments in that list, and why are they weaker than a vague "oh maybe we'll figure it out"?

4Christopher King
Hmm, Where I agree and disagree with Eliezer actually has some pretty decent counter-arguments, at least in the sense of making things less certain. However, I still think that there's a problem of "the NN writes a more traditional AGI that is capable of foom and runs it".

It seems like something has to be going wrong if the model output has higher odds that TAI is already here (~12%) than TAI being developed between now and 2027 (~11%)? Relatedly, I'm confused by the disclaimer that "we are not updating on the fact that TAI has obviously not yet arrived" -- shouldn't that fact be baked into the distributions for each parameter (particularly the number of FLOPs to reach TAI)?

3Matthew Barnett
In the notebook, the number of FLOP to train TAI is deduced a priori. I basically just estimated distributions over the relevant parameters by asking what I'd expect from TAI, rather than taking into consideration whether those values would imply a final distribution that predicts TAI arrived in the past. It may be worth noting that that Bio Anchors also does this initially, but it performs an update by chopping off some probability from the distribution and then renormalizing. I didn't do that yet because I don't know how to best perform the update. Personally, I don't think a 12% chance that TAI already arrived is that bad, given that the model is deduced a priori. Others could reasonably disagree though.

Minor curiosity: What was the context behind Asimov predicting in 1990 that permanent space cities would be built within 10 years? It seems like a much wilder leap than any of his other predictions.

5technicalities
Good catch! The book is generally written as the history of the world leading up to 2000, and most of its predictions are about that year. But this is clearly an exception and the section offers nothing more precise than "By the year 3000, then, it may well be that Earth will be only a small part of the human realm." I've moved it to the "nonresolved" tab. DM me for your bounty ($10)! I added your comment to the changelog. Thanks! 

Would be very curious to hear thoughts from the people that voted "disagree" on this post

Pattern
1715

It's a shame we can't see the disagree number and the agree number, instead of their sum.

Maybe you could measure how effectively people pass e.g. a multiple choice version of an Intellectual Turing Test (on how well they can emulate the viewpoint of people concerned by AI safety) after hearing the proposed explanations. 

[Edit: To be explicit, this would help further John's goals (as I understand them) because it ideally tests whether the AI safety viewpoint is being communicated in such a way that people can understand and operate the underlying mental models. This is better than testing how persuasive the arguments are because it's a) mo... (read more)

I strongly prefer the "dying with dignity" mentality for 3 basic reasons:

  • as other people have mentioned, "playing to your outs" is too easy to misinterpret as conditioning on comfortable improbabilities no matter how much you try to draw the distinctions
  • relatedly, focusing on "playing to your outs" (especially if you do so for emotional reasons) may make it harder to stay grounded in accurate models of reality (that may mostly output "we will die soon")
  • Operating under the mindset that death is likely when AGI is still some ways around the corner and easy t
... (read more)

Wait why are your predictions for Brazil so far from the market? As of right now, there are 180,000 shares of Bolsonaro on the orderbook under 50c on FTX (avg price of 44c if you buy them all).

3Zvi
I explicitly did not look and hadn't previously checked. Given that, this distance is small. With that information I would revise to be moderately above current fair.

Yeah it's definitely against poly's terms of service but not against US law (otherwise they wouldn't be complying with the prohibition on offering their services to US customers)

FWIW it is totally legal for Americans to trade on polymarket via a VPN or similar; it's just not legal for polymarket itself to offer services to people with US IP addresses

3kjz
I'm not a lawyer, but their newest Terms of Service imply otherwise: Not sure how willing and able they would be to enforce such a regulation, but that's a different question. (Not legal advice!)

Is there currently a supply shortage of vaccines? 

9Davidmanheim
Yes, a huge one. "COVAX, the global program for purchasing and distributing COVID-19 vaccines, has struggled to secure enough vaccine doses since its inception.. Nearly 100 low-income nations are relying on the program for vaccines. COVAX was initially aiming to deliver 2 billion doses by the end of 2021, enough to vaccinate only the most high-risk groups in developing countries. However, its delivery forecast was wound back in September to only 1.425 billion doses by the end of the year. And by the end of November, less than 576 million doses had actually been delivered."

Yep, I wanted to experiment with a central example of a comment that should be in the "downvote/agree" quadrant, since that seemed like the least likely to occur naturally. It's nice to see the voting system is working as intended. 

jefftk
232

This is a good example of a comment that is not a contribution to the discussion, but which lots of people here agree with, illustrating the power and flexibility of the two-axis voting system. Except, if you meant it as such, it's a valuable contribution to the discussion, and shouldn't be downvoted. But if it weren't downvoted it wouldn't properly show off the voting system, and would no longer be valuable. You've tied us in knots!

I haven't done much research on this, but from a naive perspective, spending 4 billion dollars to move up vaccine access by a few months sounds incredibly unlikely to be a good idea? Is the idea that it is more effective than standard global health interventions in terms of QALYs or a similar metric, or that there's some other benefit that is incommensurable with other global health interventions? (This feels like asking the wrong question but maybe it will at least help me understand your perspective)

8Davidmanheim
The idea is that the extra production capacity funded with that $4b doesn't just move up access a few months for rich countries, it also means poor countries get enough doses in months not years, and that there is capacity for making boosters, etc. (It's a one-time purchase to increase the speed of vaccines for the medium term future. In other words, it changes the derivative, not the level  or the delivery date.)

Wait, how do you get to 17%-25% chance of a crisis situation if there's only a 2.5% chance of omicron causing severe disease in vaccinated/previously infected people? Isn't that the vast majority of people in the US?

1tkpwaeub
Because there aren't enough hospital beds/nurses etc
4Bucky
My understanding is that a few months after vaccination the vaccines provide ~85% (?) protection against hospitalisation compared to an unvaccinated person. I think the 2.5% prediction is intended to indicate that Zvi thinks it's highly unlikely that this will drop to 20% protection from Omicron (or similarly low number). However even a drop to 70% (which seems more reasonable) would mean a doubling of hospitalisations per case. Even with this, my model of a crisis situation is more to do with a huge increase in infections than with increasing hospitalisation rates per case.
1arunto
We know the vaccination rates (NPR). And we have estimates for the the number of infected persons (Estimated COVID-19 Burden). But do we have a trustworthy estimate for the intersection of those two groups (and with that an estimate for the total number of previously infected and/or vaccinated)? That would be important information, ideally statewise. The other important data point would be the intensive care capacity.

My uninformed impression is that an "adjuvant" is just something that stimulates increased immune response, which has historically been an additional chemical (such as an aluminum salt) added to vaccines. The mRNA vaccines do not contain these chemicals, but some people confusingly refer to the lipid nanoparticles that surround and protect the mRNA as adjuvants (because they also help increase immune response). I haven't seen any evidence that these lipid nanoparticles are the kind of adjuvants that might mitigate OAS, but that doesn't mean much because I'... (read more)

1cistrane
Immune response to the lipid nanoparticles themselves is very bad. It reduces the uptake of protein coding mRNA into the cells and diminishes vaccine efficacy. It also leads to most of the severe allergic reactions.
2Douglas_Knight
Adjuvants are for activating the immune system to respond to free-floating virus particles. If the virus actually infects a cell and produces proteins on the cell's surface, that is a different signal to the immune system and adjuvants are not needed. Thus adjuvants were not needed for traditional live/attenuated vaccines and 21st century mRNA and vector vaccines (AZ, etc), but were needed for 1980s recombinant protein vaccines (Novavax) and traditional dead/inactivated vaccines.
2Pattern
Yes. https://www.goodreads.com/book/show/38884954-detonation works fine.
Answer by Liam Donovan
*30

Detonation -- an entertaining book that tries to flesh out a fast takeoff scenario and explicitly cites Bostrom and Yudkowsky. However, it also makes some extremely dubious choices; for example, the protagonist is a Marine hired to fight against the unfriendly AI, which doesn't seem like a very effective AI alignment strategy.

2Pattern
The link doesn't work.

Hopefully it's not too late to try to keep the focus on the fun puzzles, etc! There really does seem to be an alarming amount of craziness floating around LW, along with the constant weird attempts to explicitly model things that we evolved to understand instinctively (eg most aspects of social interaction). Reading that stuff slightly negatively affected my mental health despite thinking it was mostly silly -- to the extent it's taken seriously it seems like it could have more substantial negative effects. 

If anyone wants to get money onto Polymarket (real-money prediction market with no limits, no fees, and a wide variety of markets), I can facilitate that for free. Send me cash or crypto and I will send money directly to your poly account. As a heavy polymarket user I'm somewhat sad more LW people aren't taking the opportunity to bet on their beliefs, so I'd like to make it as easy as possible for anyone interested. 

What prediction markets are liquid enough for a fund to make sense? (unless you were just referring to the crypto trading part?)

1sharpobject
Hello, You're right. Most of the time there are not many prediction markets I would want to take positions in that also have enough capacity. We would be willing to regularly bet millions of dollars on coups not occuring in the US, but these opportunities are unusual.

Risk neutral pricing is always a danger when trying to make inferences about real world probabilities based on market pricing, but it's usually a negligible one because participants in current prediction markets are generally speculators with no built-in exposure to the underlying asset, or ability to hedge against other markets.

 On the other hand, implied probabilities from options pricing can differ significantly from real world probability, because any participant in the options market can hedge their position against the underlying asset.

"In all t... (read more)

1SimonM
Right, but then the underlying asset is telling you something and if you disagree with that, then you can trade the underlying asset. There's nothing special about options here. The difference comes from the fact that the underlying asset can have a return. (In the same way that a bond have a price different from par doesn't (necessarily) mean that the market is forecasting default - they are discounting the value of a future cash flow). The evidence would be something akin to "the historic sharpe for risk assets is <1" so the order magnitude of risk premia is "small enough" relative to the volatility. I don't think there is anything misleading about taking the market prices, constructing a bet and presenting that as a market probability, any more than taking showing betting odds and saying that's the betting market probability. Sure, there might be some subtleties depending on the market (eg long-shot bias, fees, etc), but fundamentally that's the price the market is offering. If you disagree, BET. I agree with this, all I'm saying is that the degree to which those assets fail to be a martingale is small relative to their volatility. I assume all those people are long crypto, which fundamentally means they disagree with the underling price and are long... I don't see any inconsistency between that and what I'm saying. I would be more interested if you could find me someone who thinks both that  * option prices are wrong  * they shouldn't have a position in options * they shouldn't have a position in the underlying because of some kind of risk-neutral vs real-world probability considerations.

I'm happy to; no commission needed. If anyone else wants to get money from fiat into polymarket easily with no fee, just let me know

2Davidmanheim
Update: we did this, I bought shares, we'll see how it goes.
5Davidmanheim
Want to sell me USDC on there in exchange for paypal, so I can bet? (I'll gladly pay a 2% "commission" for, say, $200 in USDC.)

This isn't really a meaningful explanation for why risk neutral vs real world is meaningless? To me "the credence I have that something happens" is actually a meaningful, important number that is by definition different from the risk neutral price. You can argue that all market probabilities may deviate from real-world probabilities in some way, but that doesn't make real-world probability meaningless! 

1SimonM
I'm now confused as to what you mean by "real world" in this context? Zvi is giving a credence for the event (p_zvi). The market is offering a bet which implies some probability for the event (p_market). All I am noting is p_zvi is different from p_market. I don't think there's anything special about the fact that options are involved here. (Unless I'm the one inferring you were specifically talking about options when you talk about "risk neutral measures". All market probabilities are in some sense in risk neutral probabilities. If you're complaint is about me talking about market probabilities then I guess this post wasn't really for you?) EDIT: To be more concrete about this, the places where "risk neutral" vs "real world" probabilities end up mattering is places where there is a concrete risk premium. (ie what the options market implies about stocks in 1y's time doesn't account for the fact that "stocks tend to go up over time"). In all the examples we're talking about, those risk premiums are tiny relative to the numbers involved so they don't make a significant difference to how we should be calculating the "market implied" odds.

https://www.betonline.ag/sportsbook/futures-and-props/politics-futures has a market for "California Governor on 12/31/2021", which should function as a great proxy for the chances of Newsom being recalled. They're quoting .0625@? for the chance of Newsom not being governor on 12/31 (they accept bets on Newsom remaining governor, but not on Newsom being removed as governor, thus the "?")

1SimonM
Thanks!

You think there's only an 80% chance the olympics happen? This is a bit of a tangent but I'd love to hear why, since I have 100k+ shares of OLY2021 on FTX and haven't heard a convincing argument for the odds being below 90%.

re options, presumably Zvi is using the real world measure, which you can'treally infer from options prices without making a lot of dubious assumptions. Can you elaborate on how/why Zvi is "off the market forecast"?

1SimonM
I think there's ~80-85% chance the Olympics happen on time. I think there's a ~90% chance that the Olympics go ahead this year. I think the case against them going ahead this year is roughly: * Current state of COVID in Japan, potential for it getting worse * "Cancellation is possible" statements from government * Public opinion is against the games I don't think it's very likely, but do I think there's a ~15-20% chance that COVID flares up in Japan in the next 3 months in a particularly bad way? Doesn't seem crazy to me. (FWIW, I don't think betting on the Olympics at the FTX odds is a bad bet, I just don't think it's a sure thing) Risk neutral vs Real world measure isn't really a meaningful distinction the way you think it is. You can construct a binary bet in terms of options, and the price is the market price for that bet and that's the market probability. It's no different than betting on any other event. If you don't like market pricing, then sure, ignore everything I've written here, but don't think "risk neutral measure" is some magic phrase which lets you ignore the options market. If you think the odds are different, you can always place that bet. I'm not saying Zvi is wildly wrong. Indeed he says he wouldn't trade with anything in 40-60% (and the market being at 60% means he's technically not "off" it), but I given it's close to what he'd consider trading, I think that's an interesting difference worth noting.

Cumulative cases are still much lower, right? I assumed that "case count" meant cumulative cases, but it sounds like you're interpreting it as daily cases, which would explain our difference. Given that Scott only gives 50% chance it seems much more likely that he meant cumulative cases too imho

Can you elaborate on why India's cases being higher than the US is an instabuy to 80%? I haven't given it any thought but it seems like that would require a pretty big increase in cases there?

2Zvi
See latest Covid post, it's already much higher.

Because mr co2 guy was clearly making a -EV bet that happened to pay off this time :)

FWIW FTX allows you to bet on its prediction markets on margin with a tokenized version of the S&P 500 as collateral, which accomplishes exactly what you want to accomplish here

markets that easily allow you to make tens of thousands of dollars with nearly no risk. 

Back during November-January, FTX had a contract called TRUMPFEB that gave  ~risk free ~15% returns in 2 months, up to millions of dollars (by betting against Trump to be president in Feburary). Right now the FTX OLY2021 market comes pretty darn close -- you can bet hundreds of thousands of dollars on the Olympics happening at 76c. There is obviously the risk of the Olympics not happening, but I haven't seen a good case for that risk being under 10%, making this a fantastic trade in expectation. 

3sapphire
I agree and recommend this trade here: https://www.lesswrong.com/posts/MSpfFBCQYw3YA8kMC/violating-the-emh-prediction-markets

Just commenting to say this is a great post and I'm surprised it hasn't gotten more engagement (maybe it's so good there's nothing else to say)

The fees are always 2% of the transaction value; i.e. numShares*avgPrice. The trick I described lets you substitute (1-avgPrice) when that would be cheaper . I should've been much clearer about this initially; I forgot that most people here probably don't know the polymarket fee structure. 

You still pay fees, they're just lower. Most sharps on polymarket do indeed do this

2ChristianKl
How high are the fees in that case?

I don't know about Omen, but on Polymarket you can mostly avoid this issue by minting a complete set of shares and selling the cheaper side. In your example this would only cost you $0.0004 in fees to acquire shares at $0.98. The easiest way I know of to mint complete shares is to add and then immediately remove liquidity

2ChristianKl
What are the limits of this technique to avoid fees of polymarket? Why doesn't everyone do it and pay no fees?

But he's been a professional politics gambler for years now, which seems like much stronger evidence for evaluating his calibration than the results of one election cycle?

2sapphire
That does not seem like very strong evidence his bets on the 2020 election were good.  He did better than people who refused to bet at all. But he did worse than people who just bet on Biden. Many people figured out the Biden bet in the poker and gambling communities. You should certainly downgrade your opinion of RJ relative to people who did a lot better.

But that line has an infinitesimal chance of intersecting with any point as significant as the Great Pyramids? Scott's argument about Uganda and Tanzania seems wrong for the same reasons 

2devonsydney
Agreed, there is an infinitesimal chance of intersecting something with such large significance as the pyramids. My point was to show that by leaving 1 of only 2 chosen variables open AND having a variable that can be set to many different values along a continuum, you significantly increase optionality for how you could retrospectively jump through hoops to assign importance to what is just a coincidence. One further thought would be this example chose a single hypothesis a conspiracy theorist may try to prove, when in reality they will be looking for these 'proofs' in many different places. If they don't find the connection in one place they'll just keep looking.

I'm probably too late here but if anyone's wondering, PI will send you a 1099-MISC with $8.57 of income, and that's what you have to pay taxes on. (well, they only send the form if you make over $600, but that doesn't change your tax liability).

 The $85 you deposit definitely does not count as income, regardless of whether you itemize deductions. Regarding the discussion below, the IRS does not treat PI winnings as gambling income, but rather as 1099-MISC (other) income.

(source: have paid taxes on PredictIt winnings)

Shor is very open about the fact that his views are to the left of 90%+ of the electorate, and that his goal is to maximize the power of people that share his views despite their general unpopularity. 

3DanielFilan
Yeah, I think I'm more surprised by Galef's tone than Shor's.

For anyone interested, the keyword to read about things like this in the economics literature is "performativity"

2NunoSempere
Thanks to both; this is a great example; I might add it to the main text
5Davidmanheim
Thanks - this is super-helpful! And after looking briefly, a citation for the above example is here.

FWIW I'm 99% sure RJ made money from this election, and I'm 50% sure he made over 90k. Why would you update against someone who has consistently made enormous amounts of money betting on his beliefs?

 

Edit: looks like I was a bit overoptimistic about his profits but he supposedly did make a decent amount

https://twitter.com/rainbow_jeremy_/status/1336815220061327363

(keep in mind he lies all the time so this is only noisy evidence)

2sapphire
Many people including myself got way better percentage and absolute returns on the election. He was way too optimistic.

reasonable expectation of negative outcomes if you lost your bankroll

One of the necessary conditions for the Kelly Criterion to be optimal is that losing your bankroll is infinitely bad, so a reasonable expectation of negative outcomes isn't enough to deviate from Kelly.

Isn't the point of "bet or update" is that you should be either updating on your counterparty's credences or taking a bet that your counterparty thinks is +EV? Here, the player is updating upon observing the host point to the door, not on the bet itself. 

 After the player has updated on the host pointing to the door, you can require the player to take the offered bet or update as normal. Assuming the host is offering the bet as a function of his credences*, the player should update from P(gold) = 1/3 to P(gold) ~= 0, because the player knows that... (read more)

Can you explain how the leverage system works on FTX for TRUMPFEB? The calculator on the site seems to produce bizzare results. 

You've actually met the first mugger so he's more likely to exist than a hypothetical counter-mugger you've never seen any evidence for

yeah maybe the best framing is fractional kelly is doing 2 things -- updating your beliefs based on fact that the market is offering you these odds, then betting full kelly based on your new posterior belief. 

Load More