All of gabrielrecc's Comments + Replies

Love pieces that manage to be both funny and thought-provoking. And +1 for fitting a solar storm in there. There is now better evidence of very large historical solar storms than there had been during David Roodman's Open Phil review in late 2014, have been meaning to write something up about that but other things have taken priority.

This is cool, although I suspect that you'd get something similar from even very simple models that aren't necessarily "modelling the world" in any deep sense, simply due to first and second order statistical associations between nearby place names. See e.g. https://onlinelibrary.wiley.com/doi/pdfdirect/10.1111/j.1551-6709.2008.01003.x , https://escholarship.org/uc/item/2g6976kg .

Leopold and Pavel were out ("fired for allegedly leaking information") in April. https://www.silicon.co.uk/e-innovation/artificial-intelligence/openai-fires-researchers-558601

Nice job! I'm working on something similar.

> Next, I might get my agent to attempt the last three tasks in the report

I wanted to clarify one thing: Are you building custom prompts for the different tasks? If so, I'd be curious to know how much effort you put into these (I'm generally curious how much of your agent's ability to complete more tasks might be due to task-specific prompting, vs. the use of WebDriverIO and other affordances of your scaffolding). If not, isn't getting the agent to attempt the last three tasks as simple as copy-pasting the task instructions from the ARC Evals task specs linked in the report, and completing the associated setup instructions? 

I'd recommend participating in AGISF. Completely online/virtual, a pretty light commitment (I'd describe it more as a reading group than a course personally), cohorts are typically run by AI alignment researchers or people who are quite well-versed in the field, and you'll be added to a Slack group which is pretty large and active and a reasonable way to try to get feedback.

Nice, thanks for this!

If you want to norm this for your own demographic, you can get a very crude estimate by entering your demographic information in this calculator, dividing your risk of hospitalization by 3 and multiplying the total by 0.4 (which includes the 20% reduction from vaccination and the 50% reduction from Paxlovid)

Anecdotally, I feel like I've heard a number of instances of folks with what pretty clearly seemed to be long Covid coming on despite not having required hospitalization? And in this UK survey of "Estimated number of people (in tho... (read more)

The byte-pair encoding is probably hurting it somewhat here; forcing it to unpack it will likely help. Try using this as a one-shot prompt:
 

How many Xs are there in "KJXKKLJKLJKXXKLJXKJL"? 

Numbering the letters in the string, we have: 1 K, 2 J, 3 X, 4 K, 5 K, 6 L, 7 J, 8 K, 9 L, 10 J, 11 K, 12 X, 13 X, 14 K, 15 L, 16 J, 17 X, 18 K, 19 J, 20 L. There are Xs at positions 3, 12, 13, and 17. So there are 4 Xs in total. 

How many [character of interest]s are there in "[string of interest goes here]"?
 

If it's still getting confused, add more s... (read more)

It seems like you're claiming something along the lines of "absolute power corrupts absolutely" ... that every set of values that could reasonably be described as "human values" to which an AI could be aligned -- your current values, your CEV, [insert especially empathetic, kind, etc. person here]'s current values, their CEV, etc. -- would endorse subjecting huge numbers of beings to astronomical levels of suffering, if the person with that value system had the power to do so. 

I guess I really don't find that claim plausible. For example, here is my r... (read more)

This reminded me of some findings associated with "latent semantic analysis", an old-school information retrieval technique. You build a big matrix where each unique term in a corpus (excluding a stoplist of extremely frequent terms) is assigned to a row, each document is assigned to a column, and each cell holds the number of times that term $t_i$ appeared in document $d_j$ , and with some kind of weighting scheme that downweights frequent terms), and you take the SVD. This also gives you interpretable dimensions, at least if you use varimax... (read more)

Why do you expect Bitcoin to be excepted from being labelled a security along with the rest? 
(Apologies if the answer is obvious to those who know more about the subject than me, am just genuinely curious)

Had a similar medical bill story from when I was a poor student: Medical center told me that insurance would cover an operation. They failed to mention that they were only talking about the surgeon's fee; the hospital at which they arranged the operation was out-of-network and I was stuck with 50% of the facility's costs. I explained my story to the facility. They said I still had to pay but that a payment plan would be possible, and that I could start by paying a small amount each month. I took that literally and just started paying a (very) small amount ... (read more)

I was using medical questions as just one example of the kind of task that's relevant to sandwiching. More generally, what's particularly useful for this research programme are

• tasks where we have "models which have the potential to be superhuman at [the] task", and "for which we have no simple algorithmic-generated or hard-coded training signal that’s adequate"; and
• for which there is some set of reference humans who are currently better at the task than the model;
• and for which there is some set of reference humans for whom the task is difficult enough that

It's worth knowing that there are some categories of data that Surge is not well positioned to provide. For example, while they have a substantial pool of participants with programming expertise, my understanding from speaking with a Surge rep is that they don't really have access to a pool of participants with (say) medical expertise -- although for small projects it sounds like they are willing to try to see who they might already have with relevant experience in their existing pool of 'Surgers'. This kind of more niche expertise does seem likely to beco... (read more)

I did Print to PDF in Word after formatting my Word document to look like a standard LaTeX-exported document, it had no problem going through! But might depend on the particular moderator. 

Sounds a little like StarWeb? Recently read a lovely article about a similar but different game, Monster Island, which was a thing from 1989 to 2017.

But yes, my default assumption would be that the particular conversation you're referring to never resulted in a game that saw the light of day; I've seen many detailed game design discussions among people I've known meet the same fate.

Thanks, I agree that's a better analogy. Though of course, it isn't necessary that none of the employees (participants in a sandwiching project) are unaware of the CEO's (sandwiching project overseer's) goal; I was only highlighting that they need not necessarily be aware of it in order to make it clear that the goals of the human helpers/judges aren't especially relevant to what sandwiching, debate, etc. is really about. But of course if it turns out that having the human helpers know what the ultimate goal is helps, then they're absolutely allowed to be ... (read more)

We start with some ML model which has lots from many different fields, like GPT-n. We also have a human who has a domain-specific problem to solve (like e.g. a coding problem, or a translation to another language) but lacks the relevant domain knowledge (e.g. coding skills, or language fluency). The problem, roughly speaking, is to get the ML model and the human to work as a team, and produce an outcome at-least-as-good as a human expert in the domain. In other words, we want to factorize the “expert knowledge” and the “having a use-case” parts of the prob

I just shared this info with an immune-compromised relative, thanks so much for this.

When I see young healthy people potentially obsessing, turning life into some sort of morbid probability matrix because one particular potential risk (Long Covid) has been made more salient and blameworthy, I sympathize a lot less. 

ONS's latest survey finds 2.8% of the UK population report that they are currently experiencing long COVID symptoms: 67% of that 2.8% report that the symptoms adversely affect their day-to-day activities. Separately, they've estimated that 70% of England has had COVID at least once; weighting their estimates for Engla... (read more)

The poster's concern is with long COVID, which can certainly have effects that a lot of people would consider severe. The "severe" COVID that has a baseline of less than 1% for the young and healthy refers to COVID that requires hospitalization. Long Covid rates are higher.

I was slightly surprised to find that even fine-tuning GPT-Neo-125M  for a long time on many sequences of letters followed by spaces, followed by a colon, followed by the same sequence in reverse, was not enough to get it to pick up the pattern - probably because the positional encoding vectors make the difference between e.g. "18 tokens away" and "19 tokens away" a rather subtle difference. However, I then tried fine-tuning on a similar dataset with numbers in between (e.g. "1 W 2 O 3 R 4 D 5 S : 5 S 4 D 3 R 2 O 1 W") (or similar representation -- can't remember exactly, but something roughly like that) and it picked up the pattern right away. Data representation matters a lot!

Thanks for digging into this a bit, and I should have linked directly to the paper rather than to an article with the headline "Heart-disease risk soars after COVID" with a focus on relative risks, since as you say the absolute risks are very important for putting things into perspective. For what it's worth, I agree with Zvi's final conclusion ("That’s not nothing, but it’s not enough that you shouldn’t live your life").

That said, an additional 1.2 out of every 100 people experiencing heart failure in the first 12 months after COVID-19 infection, if that ... (read more)

Yes, this seems correct. Unfortunately it already sounds difficult to get out:

‘My future is taken away from me’: Russians flee to escape consequences of Moscow’s war | Russia | The Guardian

"Those seeking to leave faced a severe lack of available flights after western countries closed their airspace to Russian airlines. Moscow has also closed its airspace to much of the west in response.

Flights to Yerevan, Istanbul and Belgrade were completely sold out for the coming days while a one-way ticket to Dubai was priced at over £3,000 ($4,006) – compared with £25... (read more)

Just to clarify - given that your first link seems concerned about athlete collapses/deaths following vaccination (supposedly, although the comments there imply insufficient fact-checking), but your second link is about athlete collapses/deaths following COVID-19 infection and your comment is on a post about long COVID, is your concern about heart issues following vaccination or following COVID infection?

If the latter, yes, heart disease and stroke do seem to be more probable following COVID infection according to this recent large study. It should be note... (read more)

The way that data set is presented is infuriating – there are tables that list raw counts without reference to the sample size (maybe it’s an estimated raw number for the whole country, in which case they’re quite small)

This is the UK Office for National Statistics - their usual is to report estimated numbers for the whole country. Easy to miss but it's in thousands -- scroll to the far right of each table with raw numbers and you'll see that stated near the top. So Table 1 estimates 1,332,000 UK residents with Long COVID, which is in line with the 2% figu... (read more)

UK's ONS has a nice comparison with controls which shows a clear difference, see Fig 1. (Note that this release uses laboratory-confirmed COVID-19 only, unlike some of their other releases.)

Given that the early data I've seen suggests that efficacy of 3 doses vs. omicron is similar to that of 2 doses vs. delta -- probably a bit lower, but at least in the same universe -- I've been using it largely as is, multiplying the final output by 2 to 3 based on what I've seen about the household transmission rate of Omicron relative to Delta. I know some other boosted people who have used it in a similar fashion. There's so much uncertainty in the model assumptions that its best use in my view is to get very broad-strokes order-of-magnitude idea of the... (read more)

Thank you for such a comprehensive rundown! I've bookmarked this as I expect/hope to be in a situation in the future when this comes in handy.

I hate to say it, but the images are not coming through for me, as perhaps you've already noticed!

Really, though, shouldn't we be able to do something to protect the elderly or other vulnerable people without causing everyone else six months of financial hardship and lost relationships?"

"Six months..." the man squirms. "I might need you to do this for a year or two."

Not exactly a fair description of what the public health measures have been. What country has been in lockdown for "a year or two" (besides China)?

> The harms caused by COVID suppression were larger than the harms of COVID itself for most people.

Possibly, but I doubt the same can be said... (read more)

The number of experiences I've had of reading an abstract and later finding that the results provided extraordinarily poor evidence for the claims (or alternatively, extraordinarily good evidence -- hard to predict what I will find if I haven't read anything by the authors before...) makes this system suspect. This seems partially conceded in the fictive dialogue ("You don't even have to dig into the methodology a lot") - but it helps to look at it at least a little. I knew a senior academic whose system was as follows: read the abstract (to see if the topic of the paper is of any interest at all) but don't believe any claims in it; then skim the methodology and results and update based on that. This makes a bit more sense to me.

Relevant: From OpenAI's "Training Verifiers To Solve Math Word Problems": "We also note that it is important to allow the model to generate the full natural language solution before outputting a final answer. If we instead finetune a 6B model to directly output the final answer without any intermediate steps, performance drops drastically from 20.6% to 5.2%." Also the "exploration" linked in the post, as well as my own little exploration restricted to modulo operations on many-digit numbers (via step-by-step long division!), on which LMs do very poorly wit... (read more)

https://www.bbc.co.uk/news/health-57667987 - Hard to say what's "likely" with this government, but it's what the Joint Committee on Vaccination and Immunisation has advised

Interesting... although that given that the report you link to is also about the ZOE research, and later seems to elaborate on what they mean by "up to" 30% as by saying that "even if a vaccinated individual goes on to contract Covid-19, that person’s chances of developing Long Covid are reduced by a further 30 per cent in the most at-risk age group", I wonder if this is just a slightly trumped-up way of stating the finding of a significant difference for the older age group alone in the preprint that OP already linked to. (The preprint states: "In the 60+ group, we found lower risk of symptoms lasting for more than 28 days (OR=0.72, 95%CI [0.51-1.00])".)

Worth noting that studying the effect of damage to GPS systems was not considered in the Open Philanthropy report (beyond a mention that it is beyond the scope of the report), but GPS going down could be very bad indeed, given power grids' and supply chains' near-total dependence on it: https://www.nytimes.com/2021/01/23/opinion/gps-vulnerable-alternatives-navigation-critical-infrastructure.html . On the plus side, an emergency backup system for GPS has already been mandated by U.S. law, but it sounds from the NYT article like it's been 3 years and the gov... (read more)

This seems like a slippery slope. Minorities tend to have shorter life expectancies than whites, at least in the U.S. and U.K. Do their votes then count for less?

No; my script only contains the handful of unicode characters I commonly use, and is so idiosyncratic to me that it wouldn't be of much use to anyone else (mine includes autoreplacements for directories, email addresses I commonly type, etc.). But it's easy enough to make your own with whatever characters you use -- the syntax is simply

::text-to-replace::desired-replacement

::alpha::α

::em::—

etc.

I use Autohotkey on Windows for that purpose.

Networks of the Brain by Olaf Sporns certainly doesn't cover all of computational neuroscience, but is a good accessible introduction to using the tools of network theory to gain a better understanding of brain function at many different levels.

One could bury Wikipedia, the Internet Archive, or a bunch of other items suggested by The Long Now Foundation

Since no one's yet included the links to the Long Now Foundation's blog posts in which they discuss suggestions for such items and other projects that are attempts in this direction, here they are:

http://blog.longnow.org/02010/04/06/manual-for-civilization/

http://blog.longnow.org/category/manual-for-civilization/manual-book-lists/

I find that negative visualization in conjunction with Mark Williams' guided meditation "Exploring Difficulties" is useful for getting me in that stoic mindset of being more okay with a worst-case scenario. (Or at least, I hope so - I guess I'll see how well it worked if the worst-case scenario ever comes to pass.)

I've taken the survey.

I keep a daily journal. Beginning of day: Two things that I'm grateful for. End of day: Two things that went well that day, two things that could have gone better. Each "thing" is usually only a sentence or few long. I find that going back through the end-of-day sentences every so often is useful for doing 80-20 analyses to find out what seems to be bringing me the most happiness / dissatisfaction (at least as judged by my end-of-day assessments).

I'm very sorry to hear about your dog. It's a very difficult thing to go through even without any predisposition towards depression.

This is probably an idiosyncratic thing that only helps me, but I find remembering that time is a dimension just like space helps a little bit. In the little slice of time I inhabit, a pet or person who has passed on is gone. From a higher-dimensional perspective, they haven't gone anywhere. If someone were to be capable of observing from a higher dimension, they could see the deceased just as I remember them in life. So in th... (read more)

That's the point of the article: agriculture allowed the Earth to support a vastly larger human population than it could have otherwise, but at a cost.

Personally I'm more optimistic than the author of the article I linked that the median quality of life of a human on Planet Earth will ultimately exceed the median quality of life of a human on an Earth where agriculture had never been developed -- in fact I think there's a good chance that that's already the case. But I don't think it's completely obvious, for reasons the author describes in detail.

I guess whether > 3 mg/kg is a "lot" compared to other food types is relative to the number of food types the study considered.

I haven't dug up the France study to see how many foods they looked at that didn't make the >3 mg/kg cut, but the first study that I clicked on after searching Google scholar just now is a German study that found a median mg/kg of 160 for "cocoa powder" and 39 for "chocolate". Of the 1,431 food samples they tested, "77.8% had an aluminium concentration of less than 10 mg kg-1. Of the sample... (read more)

I think it turned out pretty well.

Well, that remains to be seen.