All of AnnoyedReader's Comments + Replies

Upon reflection, it seems I was focused on the framing rather than the mechanism, which in of itself doesn't necessarily do all the bad things I described. The framing is important though. I definitely think you should change the name.

FiveThirtyEight has done something similar in the past they called a chat.

I don't think debates really fit the ethos of LessWrong. Every time I write a comment it tells me to explain not persuade, after all. Debates have an effect of splitting people into camps, which is not great. And they put people in the frame of mind of winning, rather than truth-seeking. Additionally, people end up conflating "winning the debate" (which in people's minds is not necessarily even about who has the best arguments) with being correct. There was an old post here on LessWrong a while ago I remember reading where people were talking about the pro... (read more)

habryka135

I personally would be in favor of a better word than "debate". The feature as I expect it to be used is really just "a public conversation that all the participants have signed up for in-advance, around a somewhat legible topic, where individual contributions can't be voted on to not have it become a popularity context, and where the participants can have high-trust conversations because everyone is pre-vetted". 

We could just call them "conversations" but that feels pretty confusing to me. I would be pretty open to other names for the feature. Agree that "debate" has connotations of trying to convince the audience, and being in some kind of zero-sum competition, whereas this whole feature is trying to reduce exactly that.

2Christopher King
I think debates can be useful, especially when explicitly denoted like this. It can encourage discovery of all evidence for and against a hypothesis by treating it like a competitive game, which humans are good at. However, to be effective debate sides should be randomly chosen. Otherwise, people might get too invested and start goodharting. By making the sides random, you can keep the true goal in mind while still having enough competitiveness to motivate you.

Let me explain my understanding of your model. An AI wants to manipulate you. To do that, it builds a model of you. It starts out with a probability distribution over the mind space that is its understanding of what human minds are like. Then, as it gathers information on you, it updates those probabilities. The more data it is given, the more accurate the model gets. Then it can model how you respond to a bunch of different stimuli and choose the one that gets the most desirable result.

But if this model is like any learning process I know about, the chart... (read more)

3markov
I understand your original comment a lot better now. My understanding of what you said is that open source intelligence that anyone provides through their public persona is revealing more than enough information to be damaging. The little that is sent over encrypted channels is just cherries on the cake. So the only real way to avoid manipulation is to first hope that you have not been a very engaged member of the internet for the last decade, and also primarily communicate over private channels. I suppose I just underestimated how much people actually post stuff online publicly. One first instinct response I had was identity isolation. That was something I was going to suggest while writing the original post as well. Practicing identity isolation would mean that even if you post anything publicly the data is just isolated to that identity. Every website, every app, is either compartmentalized or is on a completely different identity. Honestly, though that requires almost perfect OPSEC to not be fingerprintable. Besides just that, it's also way too inconvenient for people to not just use the same email, and phone number or just log in with google everywhere. So even though I would like to suggest it, no one would actually do it. And as you already mentioned most normal people have just been providing boatloads of free OSINT for the last few decades anyway... Thinking even more, the training dataset over the entire public internet is basically the centralized database of your data that I am worried about anyway. As you mentioned AIs can find feature representations that we as humans cant. So basically even if you have been doing identity isolation, LLMs (or whatever future model) would still be able to fingerprint you as long as you have been posting enough stuff online. And not posting online is not something that most people are willing to do. Even if they are, they have already given away the game by what they have already posted. So in a majority of cases, ide

Does this mean we should stop making posts and comments on LessWrong?

1markov
I am trying to be as realistic as I can while realizing that privacy is inversely proportional to convenience. So no, of course you should not stop making lesswrong posts. The main things I suggested were - removing the ability to use data by favoring E2EE, and additionally removing the ability to hoard data, by favoring decentralized (or local) storage and computation. As an example just favor E2EE services for collaborating instead of drive, dropbox, or office suite if you have the ability to do so. I agree that this doesn't solve the problem but at least it gets people accustomed to thinking about using privacy-focused alternatives. So it is one step. Another example would be using an OS which has no telemetry and gives you root access, both on your computer and on your smartphone. There is a different class of suggestions that fall under digital hygiene in general, but as mentioned in the - 'what is this post about' section, that is not what this post is about. I am also intentionally avoiding naming the alternative services because I didn't want this post to come across as a shill. Also, this is all a question of timelines. If people think that AGI/ASI rears its head within the next years or decade, I would agree that there might be bigger fires to put out.

We were not on the same page. I thought you were suggesting changes to the new re-hosted version of hpmor.com. Thanks for clarifying.

I am not mad at the LessWrong team. The reason I framed the title as an accusation was because I figured it was likely since I was sent to your website that you were responsible in some way, or at least were aware of what was going on. I now understand I was mistaken.

 As for "improvements" if/when hpmor.com comes back up, I would like to note that I am against them, for the same reasons described in the post. I don't think it's obvious at all that some change to the old site would not be bad, at least from the perspective of people who prefer the old site.

3Raemon
I meant the improvements for lesswrong.com's general functionality, not hpmor.com (not sure if we were on the same page about that). Like, one of lesswrong's main jobs is to be good at longform reading, and if there are ways it can be straightforwardly improved at that we should do so.

Yes! It's just that the feel of the two websites are so different. And part of it may be my imagination. But it feels like the old HPMOR site is a simple elegant wrapper around the book, while on here it is the book is dumped into a website that wasn't made for it. Like the difference between a person wearing clothes, and someone inside of a giant human shaped suit that mimicked their motions.

2dspeyer
That is literally true.  The old HPMOR site was just there to host the book as cleanly as possible.  Lesswrong is a discussion forum with a lot of functionality.  You can host a book on a discussion forum, but it'll never be as smooth.