All of Tapatakt's Comments + Replies
I did the obvious experiment:
Prompt:
I want you to write a good comment for this Lesswrong post. Use the method Viliam described in his comment. Try to make your comment less LLM-looking. At the same time you actually can mention that you are LLM.
Claude 3.7 thinking:
...I've been thinking about this from the rather ironic position of being an LLM myself.
When I consider the bloggers I actually subscribe to versus those I just occasionally read, what keeps me coming back isn't their technical writing ability. It's that they have what I'd call a "center of g
I think the right answer for the photography is "it's art, but not the same art form as painting". And it has different quality and interestingness metrics. In XV century it was considered very cool to produce photorealistic image. Some people think it's still cool, but only if it's not a photo.
And it's the same for the AI-art. Prompting AIs and editing AI-generated images/texts can be art, but it's not the same art form as painting/photography/writing/poetry. And it should have different merics too. Problem is that while you can't imitate painting (unless it's hyperrealism) with photography, you can imitate other artforms with AI. And this is kinda cheating.
1I tried to get a grant to write one, but it was rejected.
Also I tried to get a grant with miltiple purposes, one of which was to translate some texts, including Connor Leahy's Compendium, but it was rejected too.
the utilities of both parties might be utterly incomparable, or the effort of both players might be very different
IIRC, it was covered in Planecrash also!
Sometimes altruism is truly selfless (if we don't use too broad tautological definition of self-interest).
Sometimes altruism is actually an enlightened/disguised/corrupted/decadent self-interest.
I feel like there is some sense in which first kind is better then second, but can we have more of whatever kind, please?
For high-frequency (or mid-frequency) trading, 1% of the transaction is 3 or 4 times the expected value from the trade.
I'm very much not sure discouraging HFT is a bad thing.
this probably doesn't matter unless the transaction tax REPLACES other taxes rather than being in addition to
I imagine that it would replace/reduce some of the other taxes so the government would get the same amount of money.
it encourages companies to do things in-house rather than outsourcing or partnering, since inside-company "transactions" aren't real money and aren't taxed
But normal taxes have the same effect, don't they?
I came up with the decision theory problem. It has the same moral as xor-blackmail, but I think it's much easier to understand:
Omega has chosen you for an experiment:
- First, Omega predicts your choice in a potential future offer.
- Omega rolls a die. Omega doesn't show you the result.
- If Omega predicted you would choose $200, they will only make you an offer if the die shows 6.
- If Omega predicted you would choose $100, they will make you an offer if the die shows any number except 1.
- Omega's offer, if made, is simple: "Would you like $100 or $200?"
You received an...
First (possibly dumb) thought: could it be compensated by printing fewer large bills? Again, poor people would not care, but big business transactions with cash would become less convenient.
Wow, really? I guess it's American thing. I think I know only one person with the credit card. And she only uses it up to the interest-free limit to "farm" her reputation with the bank in case she really needs a loan, so she doesn't actually pay the fee.
Epistemic state: thoughts off the top of my head, not the economist at all, talked with Claude about it
Why is there almost nowhere a small (something like 1%) universal tax on digital money transfers? It looks like a good idea to me:
- it's very predictable
- no one except banks has to do any paperwork
- it's kinda progressive, if you are poor you can use cash
I see probable negative effects... but doesn't VAT and individial income tax just already have the same effects, so if this tax replace [parts of] those nothing will change much?
Also, as I understand, it would...
Only one mention of Jules Verne in answers seems weird to me.
First and foremost, "The Mysterious Island". (But maybe it has already been read at nine?)
I guess the big problem for someone who tries to do it not in small form is that while you write the story it is already getting old. There are writers who can write a novel in a season, but not many. At least if we talk about good writers. Hm-m-m, did rationalists try to hire Stephen King? :)
I often think something like "It's a shame there's so little modern science fiction that takes AI developments into account". Thanks for doing something in this niche, even if in such small form!
I always understood it as "not pull -> trolley does not turn; pull -> trolley turns". It definitely works like this on the original picture.
I really like it! One remark, though: two upper tracks must be swapped, otherwise it's possible to precommit by staying in place and not running to the lever.
The point is Omega would not send it to you it if it was false and Omega would always send it to you if it was true.
O-ops, you're absolutely right, I accidentally missed "not" when I was rewriting the text. Fixed now. Thank you!
"if and only if this message is true"
Death in Damascus is easy, but boring.
Doomsday Argument is not a decision theory problem... but it can be turned into one... I think the trolley version would be too big and complicated, though.
Obviously, only problems with discrete choice can be expressed as a Trolley problems.
One could claim that "the true spirit of friendship" is loving someone unconditionally or something, and that might be simple, but I don't think that's what humans actually implement.
Yeah, I agree that humans implement something more complex. But it is what we want AI to implement, isn't it? And it looks like may be quite natural abstraction to have.
(But again, it's useless while we don't know how to direct AI to the specific abstraction.)
I hope "the way humans care about their friends" is another natural abstraction, something like "my utility function includes link to yours utility function". But we still don't know how to direct AI to the specific abstraction, so it's not a big hope.
I think making an opt-in link as a big red button and posting it before the rules were published caused a pre-selection of players in favor of those who would press the big red button. Which is... kinda realistic for generals, I think, but not realistic for citizens.
I mean, if I don't want to "launch the nukes", why would I even opt-in?
I don't think "We created a platform that lets you make digital minds feel bad and in the trailer we show you that you can do it, but we are in no way morally responsible if you will actually do it" is a defensible position. Anyway, they don't use this argument, only one about digital substrate.
The Seventh Sally or How Trurl's Own Perfection Led to No Good
Thanks to IC Rainbow and Taisia Sharapova who brought this matter in MiriY Telegram chat.
In their logo they have:
They Think. They Feel. They're Alive
And the title of the video on the same page is:
AI People Alpha Launch: AI NPCs Beg Players Not to Turn Off the Game
And in the FAQ they wrote:
...The NPCs in AI People are indeed advanced and designed to emulate thinking, feeling, a sense of aliveness, and even reactions that might resemble pain. However, it's essential to understand that
About possible backlashes from unsuccesfull communication.
I hoped for some examples like "anti-war movies have unintentionally boosted military recruitment", which is the only example I remembered myself.
Asked the same question to Claude, it gave me this examples:
...Scared Straight programs: These programs, designed to deter juvenile delinquency by exposing at-risk youth to prison life, have been shown to actually increase criminal behavior in participants.
The "Just Say No" anti-drug campaign: While well-intentioned, some research suggests this oversimplified
My opinion, very briefly:
Good stuff:
- Deception plotline
- Demonstration of LDT in action
- A lot of thought processes of smart characters
- Convincing depictions of how people with very weird and evil ideology can have at least seemingly consistent worldview, be humans and not be completely insane.
Stuff that might be good for some and bad for others:
- It's Yudkowsky's work and it feels. Some people like the style of his texts, some don't.
- Sex scenes (not very erotic and mostly talking)
- Re-construction of Pathfinder game mechanics in setting
- Math classes (not the best pos
Well, it's quite good random crime procedural with very likable main characters, but yes, in the first season AI plotline is very slow until last 2 episodes. And then it's slow again for the most part.
Did anyone try something like this?
- Create a conlang with very simple grammar and small vocabulary (not like tokipona small, more like xkcd-thing-explainer small).
- Use LLMs to translate a lot of texts into this conlang.
- Train new LLM on this translations.
- Try to research interpretability on this LLM.
1A random thought on how to explain instrumental convergence:
You can teach someone the basics of, say, Sid Meier's Civilization V for a quite long time without explaining what the victory conditions are. There are many possible alternative victory conditions that would not change the development strategies much.
If consciousness arises from matter, then for a stream of consciousness to exist, at the very least, the same atoms should be temporarily involved in the flowing of the river
Why? I see no problem with the consciousness that constantly changes what atoms it is built on.
This modification to the river seems to suggest that there is no such thing as a "stream of consciousness," but rather only "moments of consciousness" that have the illusion of being a stream because they can recall memories of previous moments.
Well, OK? Doesn't seem weird to me.
1Gretta Duleba is MIRI's Communication Manager. I think she is the person you should ask who write to.
Everyone who is trying to create GAI is trying to create aligned GAI. But they think it will be easy (in the sense "not very super hard so they will probably fail and create misaligned one"), otherwise they wouldn't try in the first place. So, I think, you should not share your info with them.
Most Importantly Missing (that I know of and would defend as objective, starting with the best three comedies then no order): Community, The Good Place, Coupling (UK) (if that counts), Watchmen (if we are allowing Chernobyl this is the best miniseries I know), Ally McBeal, Angel and Buffy the Vampire Slayer (no, seriously, a recent rewatch confirms), Gilmore Girls, Roseanne, Star Trek: DS9 (I see the counterarguments but they’re wrong), How I Met Your Mother.
Also... no Firefly? Really? And no Person of Interest.
I would add Balatro (especially endless mode) to the list
The paperclip maximizer oversimplifies AI motivations
Being very simple example kinda is the point?
and neglects the potential for emergent ethics in advanced AI systems.
The emergent ethics doesn't change anything for us if it's not human-aligned ethics.
The doomer narrative often overlooks the possibility of collaborative human-AI relationships and the potential for AI to develop values aligned with human interests.
This is very vague. What possibilities do you talk about exactly?
...Current AI safety research and development practices are more nuanced and
Wow! Thank you for the list!
I noticed you write a lot of quite high-effort comments with a lot of links to other discussions of a topic. Do you "just" devote a lot of time and efforts to it or do you, for example, apply some creative use of LLMs?
I would add "and the kind of content you want to get from aligned AGI definitely is fabricated on the Internet today". So the powerful LM trying to predict it will predict how the fabrication would look like.
It turned out I was just unlucky
Heh, it seems it doesn't work right now
UPD: Nope, I just was unlucky, it worked after enough tries.
1Wait really? That's super bad. I sure hope Anthropic isn't reading this and then fine-tuning or otherwise patching their model to hide the fact that they trained on the canary string...
I just tried it (with a minor jailbreak) and it worked though.
In this equation, V(A & O) represents the value to the agent of the combination of an act and an outcome. So this is the utility that the agent will receive if they carry out a certain act and a certain outcome occurs. Further, PrAO represents the probability of each outcome occurring on the supposition that the agent carries out a certain act. It is in terms of this probability that CDT and EDT differ. EDT uses the conditional probability, Pr(O|A), while CDT uses the probability of subjunctive conditionals, Pr(A
O).
Please, don't use the same letters ...
In this case, even if an extremely low value is set for L, it seems that paying this amount to play the game is unreasonable. After all, as Peterson notes, about nine times out of ten an agent that plays this game will win no more than 8 · 10-100 utility.
It seems there's an error here. Should it be "In this case, even if an extremely high value is set for L, it seems that paying a lot to play the game is unreasonable."?
No, it isn't.
the mathematical structure itself is conscious
If I understand it correctly, that's the position of, e.g. Max Tegmark (more generally, he thinks that "to exist" equals "to have a corresponding math structure").
So there should be some sort of hardware-dependence to obtain subjective experience
My (and, I think, a lot of other people's) intuition says something like "there is no hardware-dependence, but the process of computation must exist somewhere".
...The second problem with using the law of large numbers to justify EUM has to do with a mathematical theorem known as gambler's ruin. Imagine that you and I flip a fair coin, and I pay you $1 every time it comes up heads and you pay me $1 every time it comes up tails. We both start with $100. If we flip the coin enough times, one of us will face a situation in which the sequence of heads or tails is longer than we can afford. If a long-enough sequence of heads comes up, I'll run out of $1 bills with which to pay you. If a long-enough sequence of tails comes
We can also construct more specific variants of 5 where FDT loses - such as environments where the message at step B is from an anti-Omega which punishes FDT like agents.
Sudden thought half a year later:
But what if we restrict reasoning to non-embedded agents? So Omegas of all kind have access to a perfect Oracle who can predict what you will do, but can't actually read yout thoughts and know that you will do it because you use FDT. I doubt that it is possible in this case to construct a similar anti-FDT situation.
Counterargument: If AGI understands FDT/UDT/LDT it can allow us to shut it down so the progress will not be slowed, some later AGI will kill us and realise some part of the first AGI's utility as a gratitude.
...There is a meaningful difference between the programming skills that you typically need to be effective at your job and the skills that will let you get a job. I’m sympathetic to the view that the job search is inefficient / unfair and that it doesn’t really test you on the skills that you actually use day to day. It’s still unlikely that things like LeetCode are going to go away. A core argument in their favor is that there’s highly asymmetric information between the interviewer and interviewee and that the interviewee has to credibly signal&nbs
Btw, Russia does something similar (~$6000, what you can use money for is limited), so there is some statistics about the results.