As AIs don't have the same origin as humans, it is basically inconceivable to me that they will ever share the internal processes underlying their "emotions" no matter how good they get at surface "emoting", in my opinion this makes it impossible to have a true connection/meaningful relationship with them as humans, even if many people in the future will fail to see this, but otherwise I largely agree with your post (a brain implemented in silicon could be a meaningful friend to a human (eg ems), people will prob fully integrate AIs into the social scene (this is a mistake in my view), people will modify their brains in (what now seems like) radical ways).

I just straight up don't believe the codeforces rating. I guess only a small subset of people solve algorithmic problems for fun in their free time, so it's probably opaque to many here, but a rating of 2727 (the one in the table) would be what's called an international grandmaster and is the 176th best rating among all actively competing users on the site. I hope they will soon release details about how they got that performance measure..

Tangential, but my immediate reaction to your example was "ugly kitten? All kittens are cute!", so I searched specifically for "ugly kitten" on Google and it turns out that you were right! There are a lot of ugly kittens even though I never saw them! This probably says something about society..

3 (not so easy) steps to understand consciousness:

epistemic status: layman, so this is unlikely to have any value to those well-versed in philosophy, but their input is ofc appreciated if given

1. Understand what difficult words like consciousness and qualia point to. This is hard because most of our words point to objects/relations in the physical world and "what it is to be like someone/sth"/"the blueness of blue" does not. I've seen people first getting acquainted with these words have trouble disentangling these concepts from things in the physical worl... (read more)

I really enjoyed Brandon Sanderson's Secret Project #3 and I recommend it to everyone. Without spoiling anything, here is a fun fact: In it, people stack pebbles into heaps; similarly to Sorting Pebbles Into Correct Heaps, a text from this community I still think about semi-frequently (another is The Virtue of Silence). So if you take recommendations from random lesswrong users, give it a try!

I didn't believe it, so I tried to reproduce. It actually works. Scary..

https://chat.openai.com/share/312e82f0-cc5e-47f3-b368-b2c0c0f4ad3f

EDIT: Shared link no longer works and I can't access this particular chat log after logging in either.

Thanks and that's fair. I would have liked to bet mostly as a hedge to allow myself to not think about aliens in the next 10 years (I semi-regularly get roped into investigating some claims, but the UFO community's epistemics is the worst I've seen and it is always an awful experience), but the bet wasn't really positive EV for me either, so I don't think I will bet at worse odds, but you can probably find someone on the r/UFOs subreddit if you want, some of them seem to be celebrating the new world order now. 

It's particularly interesting from 5:30: if I'm not misunderstanding him, he implies that they found extra-terrestrial bodies inside the landed/crashed vehicles, which makes me think this is deliberate misinformation. Why would an advanced entity, capable of traveling between stars, separate their body from their spacecraft? Not impossible, but encountering this evidence is much more likely imo if they are crafting a story from common memes about aliens rather than reporting something actually real.

If your offer isn't just to lc, then I accept: My 20 usd against your 20*99=1980 usd, both sides adjusted for inflation and the time value of money using US Treasury Bills, paid either at 2033.06.06. or when you admit a conclusive proof was found. Are these terms acceptable?

Recently, OpenAI implemented a "Share link" feature which is a bit different than the one mentioned in the parent comment (It creates a snapshot of the conversation which is not updated as the user continues to chat, but at any time they can generate a new link if they wish. I especially like that you can switch between sharing anonymously or with your name.); therefore, this feature request can be considered closed: Now the authenticity of chatGPT's output can be proved! Thanks to everyone who supported it and OpenAI for implementing it (even though these events are probably unrelated)!

With some helpful questions and commands (eg. "which was the last stone a given player took to win?" and "write out the full tree of every possible move of the game!") it is able to solve it:

[..]

🧔‍♂️What if the players play to win?

🤖If both players play optimally to win, the game will unfold as follows:

Player 1 takes a stone from any cell (A, B, C, or D):
1.1. Player 1 does NOT take a neighboring stone (3 stones left):

sql
Copy code
 1.1.1. Player 2 takes the stone with two neighbors:

     1.1.1.1. Player 2 does NOT take an additional stone

When I was a child, I literally tried the same ineffective actions like 40 times in similar games, so I felt a bit for gpt given you only let it try the the ineffective actions a few times. Therefore, I tried the same test with ChatGPT-4 and let it use all of my 3hr limit (25): (EDIT:generated +25 moves)
 

🧔‍♂️I want you to play a parser-based interactive fiction game. I'll show you what's displayed on the screen, and you tell me what you would type.
🤖Sure, please go ahead and show me what's displayed on the screen.

🧔‍♂️"Alert. Power outage in Sector 4

Good idea! 

I especially like that your feature does not require active buy-in from the user: anytime they make a screenshot the signature will be there. It is also nice, that the user could keep making screenshots of the conversation which (as a picture is more eye-catching than text) is great for marketing reasons (though this will be imo less and less important as chatGPT (or successor or competitor models) inevitably become household names on par with "Google")

I fear however, that if OpenAI is anything like software companies I knew and there is a ... (read more)

Not relevant to capabilties or safety, but my two favourite parts was 

1. when he copy pasted the letter "Q" from discord to gpt-4 (actual programmer!) 
2. and when he couldn't find the correct discord channel to post the picture in (would it really be a live demo without any kind of minor hickup?)

loved this!

Agreed. I got the weights very quickly after filling out the form, even though I simply wrote "None" in the (required!) "Previous related publications" field. (It still felt great to get it, so thx Meta!)

At this point in their life, Taleuntum did not at all expect that one short, self-referential joke comment will turn out to be the key to humanity's survival and thriving in the long millenias ahead. Fortunately, they commented all the same.

I upvoted, because these are important concerns overall, but this sentence stuck out to me:

The fact that Yudkowsky doesn't even know enough about Chollet to pronounce his name displays a troubling lack of effort to engage seriously with opposing views.

I'm not claiming that Yudkowsky does display a troubling lack of effort to engage seriously with opposing views or he does not display such, but surely this can be decided more accurately by looking at his written output online than at his ability to correctly pronounce names in languages he is not native in.... (read more)

As someone who doesn't know web-development I'm curious what would be the obstacle to letting the user write their own custom comment sorting algorithm? I'm assuming comment sorting is done on the client machine, so it would not be extra burden on the server. I would like to sort top level comments lexicographically first by whether they were written by friends (or at least people whose posts I'm subscribed to) or having descendant replies written by friends, then whether they were written in the last 24h then by total karma and lower level comments lexico... (read more)

I think a key idea related to this topic and not yet mentioned in the comments (maybe because it is elementary?) is the probabilistic chain rule. A basic "theorem" of probability which, in our case, shows that the procedure of always sampling the next word conditioned on the previous words is mathematically equivalent to sampling from the joint of probability distribution of complete human texts. To me this almost fully explains why LLMs' outputs seem to have been generated with global information in mind. What is missing is to see why our intuition of "me... (read more)

Some slightly amusing chats I had: 

me: can you tell me about the most famous controveries involving ea?
ai: As a large language model trained by OpenAI, I don't have any specific knowledge about past controversies involving Electronic Arts (EA). My training is focused on generating human-like text based on the input I receive, and I don't have access to information about past events or controversies that may have involved EA or other organizations. I can provide some general information about EA and its products if you're interested, but I don't have a... (read more)

Unfortunately, I don't have the capacity to deal with this problem right now however serious, because I have much more pressing issues in the form of r/place.

My guess would be that they are upvoting some kind of post/comment which only shows up for those it is shared with. Some sites do allow these, though I'm not sure how to do this on lesswrong. I've already strong downvoted all of their public comments containing no value, so I guess they've won this time :(

EDIT: Looking around the UI I see that you can share drafts with people, maybe they're upvoting each other's drafts? (If that's even possible)

Have you taken electrolytes?

Thanks for the game, I really enjoyed it and finally trying out some things I learned in practice. The solution I submitted was the best possible team according to a XGBClassifier calibrated with sklearn.CalibratedClassifierCV. Calibration performance on a test set and evaluating a solution with a different model (Dense NN) did make me realize that the solution is unlikely to be performant, but it was worth a try.

You can just pretend that $\omega$ is finite and plug it into the formula for the partial sum.$\sum _{i=1}^{n}i=\frac{1}{2}{n}^2+\frac{1}{2}n$, so $\sum _{i=1}^{\omega }i=\frac{1}{2}{\omega }^2+\frac{1}{2}\omega$. If they were to give the ith odd number amount of fish on the ith day (1,3,5,7,9...), then you would have ${\omega }^2$ amount of fish, because $\sum _{i=1}^{n}2i-1=n^2$. The two links I posted about the handling of infinite divergent series go into greater detail (eg. the question of the starting index).

You can absolutely count your fish that way with the help of hyperreals! ("growing promise" stream would be $\frac{1}{2}{\omega }^2+\frac{1}{2}\omega$ though)

I think https://en.wikipedia.org/wiki/Hyperreal_number#The_ultrapower_construction is a good introduction. https://math.stackexchange.com/questions/2649573/how-are-infinite-sums-in-nonstandard-analysis-defined and https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3459243 address the handling of infinite divergent series with hyperreals and https://arxiv.org/pdf/1106.1524v1.pdf talks about uniform probability over $N$&nb... (read more)

My teams:

The Lost Temple of Lemarchand:

• Ranger 1
• Rogue 1
• Cleric 4
• Mage 4

The Infernal Den of Cheliax:

• Mage 1
• Fighter 3
• Druid 4
• Ranger 5

The Goblin Warrens of Khaz-Gorond:

• Fighter 3
• Ranger 3
• Rogue 3
• Cleric 4

One common outcome (though not universal) in my simulations was that the carni/omnivores kill the non invincibles, then the pure carnivores starve to death and then the minimal size invincibles outcompete to extinction the non-minimal invincibles (which also includes the omnivores, the multi-herbivores and the animals able to migrate). I'm curious on how many biomes this outcome will play out.

Furthermore, It is also not inconceivable to me that an adversary might be able to use the hash itself without cracking it. For example, the sha256 hash of some information is commonly used to prove that someone has that information without revealing it, so an adversary, using the hash, could credibly lie that he already possesses a launch code and in a possible counterfactual world where no one found about the client side leaking the hash except this adversary, use this lie to acquire an actual code with some social engineering.

Like:

"Attention Lesswrong! ... (read more)

Counterclockwise, I've never heard anyone use it for clockwise.

Is it that intuitive to you that you should name the rotating object's direction using the movement of the top of the object? I think I would get confused with your words after a while. I just use "positive" and "negative" direction.

I was the one who first shared the notes on reddit. Unfortunately, I did not know that recording the speech was against the wishes of Sam Altman, but I should have checked why there was no recording (at the time I simply thought that no one bothered to do it). I'm sorry, fwiw I deleted the post.

I didn't know about Ramanujan primes and I enjoyed this article about them, so thanks!

One small note: I was a bit confused about whether the magic line's endpoints are included in the magic line. Initially, I thought they both are, but the formal definition says that the lower endpoint is not included. Maybe others were also confused.

This was my guess too, but they later did another study with rats and it did not show significant effects.

https://sciencenordic.com/climate-change-denmark-obesity/four-years-later-is-co2-making-us-fat/1440745

You are right, that is also a possibility. I only considered cases with one intervention, because the examples I've heard given for Goodhart's law only contain one (I'm thinking of UK monetary policy, Soviet nail factory and other cases where some "manager" introduces an incentive toward a proxy to the system). However, multiple intervention cases can also be interesting. Do you know of a real world example where the first intervention on the proxy raised the target value, but the second, more extreme one, did not (or vica versa)? My intuition suggests that in the real world those type of causal influences are rare and also, I don't think we can say that "P causes V" in those cases. Do you think that is too narrow of a definition?

Have P proxy and V value. Based on past observances P is correlated with V.

Increase P! (Either directly or by introducing a reward to the agents inside the system for increasing P, who cares)

Two cases:

P does not cause V

P causes V

Case 1: Wow, Goodhart is a genius! Even though I had a correlation, I increased one variable and the other did not increase!

Case 2: Wow, you are pedantic. Obviously if the relationship between the variables is so special that P causes V, Goodhart's law won't apply. If I increase the amount of weight lifted (proxy), then obviously I... (read more)

Replication Markets is going to start a new project focusing on COVID studies. Infos:

• Surveys open on October 28, 2020.
• Markets open on November 11, 2020.
• A total of $14,520 in prizes will be awarded.
• Contest will forecast (1) publication, (2) citation, (3) replication, and (4) usefulness for the Top-400 claims from COVID-19 research, using both surveys and markets.

Your betrayal of the clique is very nice, hats off to you. I also liked your idea of getting others not that interested in the game to submit bots helping you, It's a pity it did not occur to me.

However, I think you are, too, overconfident in you winning. I've run a simulation of the whole tournament till the 160th round with 8 bots (MatrixCrashingBot, TitforTatBot, PasswordBot1, PasswordBot2, EmptyCloneBot, earlybird, incomprehensiblebot, CliqueZviBot) and in the final equilibrium state there are three bots: earlybird, incomprehensiblebot and CliqueZviBot... (read more)

Disqualifying players for things they obviously wouldn't do if they knew the rules of the game seems pretty cruel. I hope isusr just deletes that line for you.

The links you posted do not work for me. (Nevermind)

Wow, you are really confident in you winning. There are 10 players in the clique, so even if there are no players outside the clique (a dubious assumption) a priori there is 10% chance. If I had money I would bet with you.

I also think there is a good chance that a CloneBot wins. 10 possible member is a good number imo. i would say 80%.

I would say 70% for the (possibly accidental) betrayal.

Without seeing your jailbreak.py I can't say how likely that others are able to simulate you.

What does "act out" mean in this context?

Yes, I feared that some might think my friend is in the clique. However I couldn't just say that they are not in the clique, because that would have been too obvious. (like my other lie: "Yeah, I totally have another method for detecting being in a simulation even if the simulation runs in a separate process, but unfortunately I can't reveal it.") So I tried to imply it by speaking about him as if he is not in the conversation and him not commenting after I mentioned him. I hoped in case someone was planning to submit a simulator outside the clique they would try to sneakily inquire about whether my friend is in the clique or not and then I would have asked a random, not competing lesswronger to play the part of my friend.

Good to know. I'm a C++ guy which has a "one definition rule" not only for the translation unit, but for the whole program, so I incorrectly assumed that python is the same even though the languages are obviously very different.

Maybe it's a little cheap to say this after you've revealed it, but it did actually occur to me that you might have deliberately made this weakness. Had I known that in Python you can redefine methods, I might have reported it, but the exploit with __new__() seemed pretty obscure (even though I didn't know the other way and I did know this). The possibility of this being a test was also the reason I went with the "Oh I'm so busy, I didn't have time to review the code.." excuse. I'm also curious whether Larion calculated with you deliberately planting the m... (read more)

Explanation of my strategy and thought process in chronological order

After seeing Vanilla_Cabs's comment I lied to them about wanting to join the clique. I was undecided, but I figured seeing the code of the clique can be a great advantage if I can exploit some coding mistake and I can still decide to join later anyway if I want.

The first versions of CloneBot (the name of the program for our clique) did actually contain a mistake I could exploit (by defining the __new__() method of the class after the payload) and so this was my plan until Vanilla_Cabs fi... (read more)

In what order do programs get disqualified? For example, if I submit a program with an infinite loop, every other program using simulation will also go into infinite loop when meeting with my program as detecting infinite loops generally isn't theoretically feasible. Is my program disqualified before the others? What is the general principle? 

EDIT: An unrelated question: Do round numbers start from 0 or 1? In the post you write "Unlike Zvi's original game, you do get to know what round it is. Rounds are indexed starting at 0.", but also: "Your class must have an __init__(self, round=1) [..]". Why not have the default initializer also use 0 if the round numbers start from zero?

You should also check whether 'exec' is in the program code string, because someone could call getopponentsource with exec and caesar-encryption, otherwise you will be DQ'd if someone submits a program like that. (However, rebinding getopponentsource is probably more elegant than this type of static analysis.)

I don't have much time, so I've only checked the first study. The numbers come from this one: https://ultrasuninternational.com/wp-content/uploads/raharusun-et-al-2020_patterns_of_covid-19_mortality_and_vitamin_d_an_indonesian_study.pdf

I looked a bit more and found this: https://www.cambridge.org/core/journals/british-journal-of-nutrition/article/covid19-and-misinformation-how-an-infodemic-fueled-the-prominence-of-vitamin-d/8AC1297F0D6F4196938FB13A85A817A3

It seems to be misinformation.

I couldn't find the second study, though I haven't l... (read more)