All of AVoropaev's Comments + Replies

What's the source of that 505 employees letter? I mean the contents aren't too crazy, but isn't it strange that the only thing we have is a screenshot of the first page?

Re: Tik-tok viral videos. I think that the cliff is simply because recent videos had too little time to be watched 10m times. The second graph in the article is not about the same for 0.1m views, but about average views per week (among videos with >0.1m views), which stays stable.

I don't understand the point of questions 1 and 3.

If we forget about details of how model works, the question 1 essentially checks whether the entity in question have a good enough rng. Which doesn't seem to be particularly relevant? Human with a vocabulary and random.org can do that. AutoGTP with access to vocabulary and random.org also have a rather good shot. Superintelligence that for some reason decides to not use rng and answer deterministically will fail. I suppose it would be very interesting to learn that say GPT-6 can do it without external rng, ... (read more)

Yeah, you are right. It seems that it was actually one of the harder ones I tried. This particular problem was solved by 4 of 28 members of a relatively strong group. I distinctly remember also trying some easy problems from a relatively weak group, but I don't have notes and Bing don't save chat.

I guess I should just try again, especially in light of gwillen's comment. (By the way, if somebody with access to actual GPT-4 is willing to help me with testing it on some math problems, I'd really appreacite it .)

That would explain a lot. I've heard this rumor, but when I tried to trace the source, i haven't found anything better than guesses. So I dismissed it, but maybe I shouldn't have. Do you have a better source?

I agree that there are some impressive improvements from GPT-3 to GPT-4. But they seem to me a lot less impressive than jump from GPT-2 producing barely coherent texts to GPT-3 (somewhat) figuring out how to play chess.

I disagree with you take on LLM's math abilities. Wolfram Alpha helps with tasks like SAT -- and GPT-4 is doing well enough on them. But for some reason it (at least in the incarnation of Bing) has trouble with simple logic puzzles like the one I mentioned in other comment.

Can you tell more about success with theoretical physics concepts? I don't think I've seen anybody try that.

I didn't say "it's worse than 12 yo at any math task". I meant nonstandard problems. Perhaps that's wrong English terminology? Sort of easy olympiad problem?

The actual test that I performed was "take several easy problems from a math circle for 12 y/o and try various 'lets think tep-by-step' to make Bing write solutions".

Example of such a problem:

Between 20 poles, several ropes are stretched (each rope connects two different poles; there is no more than one rope between any two poles). It is known that at least 15 ropes are attached to each pole. The poles are divided into groups so that each rope connects poles from different groups. Prove that there are at least four groups.

Two questions about capabilities of GPT-4.

1. The jump in capabilities from GPT-3 to GPT-4 seems like much much less impressive than jump from GPT-2 to GPT-3. Part of that is likely because later version of GPT-3 were noticeably smarter than the first ones, but that reason doesn't seem sufficient to me. So what's up? Should I expect that GPT-4 -> GPT-5 will be barely noticeable?
2. In particular I am rather surprised at apparent lack in ability to solve nonstandard math problems. I didn't expect it to beat IMO, but I did expect that problems for 12 y/o would be

Can it in some way describe itself? Something like "picture of DALL-E 2".

#2: My impression is that something like 2%-10% of Ukrainian population believed that a month ago (would you consider that worrying enough?). My evidence for that is very shacky and it is indeed quite possible that I am overestimating it by an order of magnitude (still kind of worrying, though I might be overestimating even more).

First, my aunt is among them. Second, over last few years I've seen multiple (something like 5-10, concentrated around present date?) discussions on social media where friends of friends (all Russians) said that they believe in na... (read more)

Yes, I think that it is the most likely scenario. Still, it bothered me enough that I mentioned it -- I consider such omission 2-3 times more likely in a world where there are other important (intentional) omissions that I haven't noticed than in a world where he is honest.

I still think that reading Galeev is worth it and that he is trustworthy enough source. But if for example he'll make a thread on modern Russian opposition that doesn't mention Navalny, it'll be a huge red flag for me.

To clarify: this site contains very effective propaganda that makes it a cognitohazard. You are likely underestimating its danger. It is not "just a bunch of fake statements". It is "a bunch of statements optimized for inflicting particular effects on its readers". Such "particular effects" are not limited to believing in what news says. In fact, news regularly contradict what they said a few months ago even in peace time, so believing what they are literally saying is probably not the point.

Before reading propaganda consider that such materials:

1) Convinc... (read more)

As a Russian I confirm that everything that Galeev says seems legit. I haven't been following our politics that much, but Gallev's model of Putin's fits my observations.

The only thing that looked a little suspicious to me was the thread on Russian parliamentarism -- there was an opportunity to say something about Navalny's team there (e.g. as a central example of party that can't be registered or something about them organizing protests), and I expected that he would mention it, but he didn't. In fact, I don't think he ever mentioned Navalny in any of his threads. Why?

I think that if Lesswrong wants to be less wrong, then questions "why do you believe in that?" should not be downvoted.

As for the question itself, I know next to nothing about the situation on this NPP, but just from priors I'd give 70% that if someone shelled it, it was Russian army.

1) It is easier to shoot at NPP if you don't know what you re shooting at. Russian army is much more likely to mistake this target for something else.

2) p(Russian government lies that it wasn't them | it was them) > p(Ukrainian government lies it wasn't them | it was them) ... (read more)

Update: Prosecutor's General Office says that protest will be treated as "participation in radical group" which is up to 6 years. Probably won't be used too massively, at least initially.

Yeah, doesn't seem to be true. There is this law, and general attitude of treating posts on vk/facebook as a mass media -- but it is 'just' 3 years or a huge fine, and it is rarely enforced (yet). (There might be some other relevant laws that I don't know about, but I would be very surprised (and concerned) if they involved 10 year prison terms.) It might be wise to make some minimal precautions though -- like making all posts that are not meant to be read by tovaritch major "friends only".

Thank you for treating it as a "today's lucky 10,000" event. I am aware about quines (though not much more than just 'aware') and what I am worried about is whether people that created FairBot were careful enough.

"Definition" was probably a wrong word to use. Since we are talking in the context of provability, I meant "a short string of text that replaces a longer string of text for ease of human reader, but is parsed as a longer string of text when you actually work with it". Impredicative definitions are indeed quite common, but they go hand in hand with proofs of their consistency, like proof that a functional equation have a solution, or example of a group to prove that group axioms are consistent, or more generally a model of some axiom system.

Sadly I am not f... (read more)

It's been ages since I studied provability logic, but those bots look suspicious to me. Have anybody actually formalized them? Like the definition of FairBot involves itself, so it is not actually a definition. Is it a statement that we consider true? Is it just a statement that we consider provable? Why won't adding something like this to GL result in contradiction?

I'm no programmer, so I have no comment on "how to develop" part. The "safe" part seems extremely unsafe to me though.

1) Your strategy relies on human supervisor's ability to recognize a threat that is disguised by superintelligence. Which is doomed to failure almost by definition.

2) Supervisor himself is not protected from possible threat. He is also one of the main targets that AI would want to affect.

3) >Moreover, the artificial agent won’t be able to change the operational system of the computer, its own code or any offline task that could fundament... (read more)

I'm trying to see what makes those numbers so implausible, and as far as I understand (at least without looking into regional data) the most surprising/suspicious thing is that number of new cases of Delta is dropping too fast.

But why shouldn't it be dropping fast? Odds of people getting Omicron (as opposed to Delta) are growing fast enough -- if we assume that they are (# of Omicron cases)/(# of Delta cases)*(some coefficient like their relative R_0), then due to Omicrons's fast doubling it can go from 1:2 to 4:1 in just a week. That will make new Delta c... (read more)

I don't see why all possible ways for AGI to critically fail to do what we have build it for must involve taking over the lightcone.

That doesn't stop other people from building one.

So let's also blow up the Earth. By that definition the alignment would be solved.

When you say "create an AGI which doesn't do this" do you mean that it has about 0% probability of doing it or one that have less than 100% probability of doing it? 

Edit: my impression was that the point of alignment was producing an AGI that have high probability of good outcomes and low probability of bad outcomes. Creating an AGI that simply have low probability of destroying the universe seems to be trivial. Take a hypothetical AGI before it produced output, throw a coin and if its tails then destroy it. Voila, the probability of destroying the un... (read more)

To teach million kids you need like hundred thousand teachers from Dath Ilani. They don't currently exist.

It can be circumvented by first teaching say a hundred students, 10% of which becomes teachers and help teaching new 'generation'. If each 'generation' takes 5 years, and one teacher can teach 10 students in one generation, the amount of teachers will be multiplied by 2 every 5 years, and you'll get a million Dath Ilanians in like 50 years.

One teacher teaching 10 students and 1 of them becoming a teacher might be more possible than it seems. For exampl... (read more)

How would having AGI that have 50% chance to obliterate lightcone, 40% to obliterate just Earth and 10% to correctly produce 1000000 of paperclips without casualties solve the alignment?

I think that since lady also said something about pharmacy, it's more likely that "lusi"="Luke's".

It' not about relative age (either as in age of one person divided by age of another or one age substracted from another), it's about their month of birth. So it's evidence for relevance of amount of received sunshine during pregnancy, relevance of age of being admitted in school and relevance of astrology.

Since it seems to somewhat align with different kinds of education starting in different times of year, my personal bet is on schools, though I wouldn't completely discount differences of pregnancies in different times of the year (sorry, astrology, but I need a lot more evidence to seriously consider you).

But that's a fix to a global problem that you won't fix anyway. What you can do is allocate some resources to fixing a lesser problem "this guy had nothing to eat today".

It seems to me that your argument proves too much -- when faced with a problem that you can fix you can always say "it is a part of a bigger problem that I can't fix" and do nothing.

What do you mean by 'real fix' here? What if said that real-real fix requires changing human nature and materialization of food and other goods out of nowhere? That might be more effective fix, but it is unlikely to happen in near future and it is unclear how you can make it happen. Donating money now might be less effective, but it is somehow that you can actually do.

Detailed categorizations of mental phenomena sounds useful. Is there a way for me to learn that without reading religious texts?

How can you check proof of any interesting statement about real world using only math? The best you can do is check for mathematical mistakes.

I assume you mean that I assume P(money in Bi | buyer chooses Bi )=0.25? Yes, I assume this, although really I assume that the seller's prediction is accurate with probability 0.75 and that she fills the boxes according to the specified procedure. From this, it then follows that P(money in Bi | buyer chooses Bi )=0.25.

Yes, you are right. Sorry.

Why would it be a logical contradiction? Do you think Newcomb's problem also requires a logical contradiction?

Okay, it probably isn't a contradiction, because the situation "Buyer writes his decision and it is common... (read more)

I've skimmed over A Technical Explanation of Technical Explanation (you can make links end do over stuff by selecting the text you want to edit (as if you want to copy it); if your browser is compatible, toolbar should appear). I think that's the first time in my life when I've found out that I need to know more math to understand non-mathematical text. The text is not about Bayes' Theorem, but it is about application of probability theory to reasoning, which is relevant to my question. As far as I understand, Yudkowski writes about the same algorithm that... (read more)

That's interesting. I've heard about probabilistic modal logics, but didn't know that not only logics are working towards statisticians, but also vice versa. Is there some book or videocourse accessible to mathematical undergraduates?

This formula is not Bayes' Theorem, but it is a similar simple formula from probability theory, so I'm still interested in how you can use it in daily life.

Writing P(x|D) implies that x and D are the same kind of object (data about some physical process?) and there are probably a lot of subtle problems in defining hypothesis as a "set of things that happen if it is true" (especially if you want to have hypotheses that involve probabilities). 

Use of this formula allows you to update probabilities you prescribe to hypotheses, but it is no... (read more)