All of Jalex S's Comments + Replies
possibly an easier entry point to the topic is here
https://en.wikipedia.org/wiki/Chaitin%27s_constant
which is a specific construction that has some relation to the ideas OP has for a construction
I think "read the sequences" is an incredibly unhelpful suggestion. It's an unrealistic high bar for entry. The sequences are absolutely massive. It's like saying "read the whole bible before talking to anyone at church", but even longer. And many newcomers already understand the vast bulk of that content. Even the more helpful selected sequences are two thousand pages.
We need a better introduction to alignment work, LessWrong community standards, and rationality. Until we have it, we need to personally be more helpful to aspiring community members.
See Th...
With regards to subsidizing: all the subsidizer needs to do in order to incentivize work on P is sell shares of P. If they are short P when P is proven, they lose money -- this money in effect goes to the people who worked to prove it.
To be more concrete:
Suppose P is trading at 0.50. I think I can prove P with one hour of work. Then an action available to me is to buy 100 shares of P, prove it, and then sell them back for $50 of profit.
But my going fee is $55/hour, so I don't do it.
Then a grantmaker comes along and offers to sell some shares at $0.40. Now the price is right for me, so I buy and prove and make $60/hr.
This is an example where the true distribution of future prices is bimodal (with the average between the modes). If all you can do is buy or sell stock, then you actually have to disagree with the market about the distribution to make money.
Without having information about the probability of default, there might still be something to do based on the vol curve.
it would be 3 lines
~all of the information is in lines 2 and 3, so you'd get all of the info on the first screen if you nix line 1.~
edit: not sure what I was thinking -- thanks, Slider
My knee-jerk reaction to the argument was negative, but now I'm confused enough to say something.
If the contract is trading for M$p, then the "arbitrage" of "buy a share of yes and cause the contract to settle to 1" nets you M$(1-p) per share. Pushing up the price reduces the incentive for a new player to hit the arb.
If you sell the contract, you are paying someone to press the button and hoping they do not act on their incentive.
An interesting plot twist: after you buy the contract, your incentive has changed -- instead of the M$(1-p) availab...
The US isn't short on places to live, it's short on places to live that are short drives from the people and businesses you most want to interact with. If you want to found a new city, there are cheaper and more desireable places to do it; the difficulty comes from the fact that very few people want to go somewhere that doesn't already have a large critical mass of people, business and infrastructure already in place.
Writing my dating profile was a good use of my time before I shared it with anybody. I had an insufficiently strong sense of what kind of relationship I want and why other people might want to have it with me. The exercise of "make a freeform document capturing all of that" was very helpful for focusing my mind towards figuring it out -- much moreso than the exercise of "fill in dating app textboxes in a way that seems competitive for the swiping game". (This is just a special case of "writing an essay teaches you a lot" -- something I'd like to take advan...
Which trade are you advocating for? "long crypto"? Reversion? (akak "buying the dip") Long ETH vs. short BTC?
All of these are plausible opinions, and it's not crazy to allocate some of your portfolio based on them -- but a trade consists of a price and a size. Do you think you should have 0.1% of your net worth in ETH or 30%? Does that change if ETH goes to 100 or 3000 next week? Do your arguments apply equally well elsewhere? (solana?)
Specifically, the punchline is using the repetition trap as an emotive ending bang. (Which is clever but also something that will be lost on most people who have not used large models personally, because users usually scrub or restart repetition-trap samples while tweaking the repetition penalty & other sampling parameters to minimize it as part of basic prompt engineering.)
So you have a crisp concept called "unbounded utility maximizer" so that some AI systems are, some AI systems aren't, and the ones that aren't are safe. Your plan is to teach everyone where that sharp conceptual boundary is, and then what? Convince them to walk back over the line and stay there?
Do you think your mission is easier or harder than nuclear disarmament?
I think I get what you're saying now; let me try to rephrase. We want to grow the "think good and do good" community. We have a lot of let's say "recruitment material" that appeals to people's sense of do-gooding, so unaligned people that vaguely want to do good might trip over the material and get recruited. But we have less of that on the think-gooding side, so there's a larger gap of unaligned people who want to think good that we could recruit.
Does that seem right?
Where does the Atlas fellowship fall on your scale of "recruits do-gooders" versus "recruits think-gooders"?
I think the most important claim you make here is that trying to fit into a cultural niche called "rationality" makes you a more effective researcher than trying to fit into a cultural niche called "EA". I think this is a plausible claim, (e.g. I feel this way about doing a math or philosophy undergrad degree over doing an economics or computer science undergrad degree) but I don't intuitively agree with it. Do you have any arguments in favor?
Pushing which button? They're deploying systems and competing on how capable those systems are. How do they know the systems they're deploying are safe? How do they define "not-unbounded-utility-maximizers" (and why is it not a solution to the whole alignment problem)? What about your "alignment-pilled" world is different from today's world, wherein large institutions already prefer not to kill themselves?
I think the impact of little bits of "people engage with the problem" is not significantly positive. Maybe it rounds to zero. Maybe it is negative, if people engaging lightly flood serious people with noisy requests.
Hard research problems just don't get solved by people thinking for five minutes. There are some people who can make real contributions [0] by thinking for ~five hours per week for a couple of months, but they are quite rare.
(This is orthogonal to the current discussion, but: I had not heard of stampy.ai before your comment. Probably you should...
you would be able to drop those activities quickly and find new work or hobbies within a few months.
I don't see it. Literally how would I defend myself? Someone who doesn't like me tells you that I'm doing AI research. What questions do you ask them before investigating me? What questions do you ask me? Are there any answers I can give that meaningfully prove that I never did any such research (without you ransacking my house and destroying my computers?)
re q2: If you set up the bounty, then other people can use it to target whoever they want. ...
In this system, how do I defend myself from the accusation of "being an AI researcher"? I know some theorems, write some code, and sometimes talk about recent AI papers. I've never tried to train new AI systems, but how would you know?
Have you heard about McCarthyism?
If you had the goal of maximizing the probability of unaligned AI, you could target only "AI researchers" that might contribute to the alignment problem. Since they're a much smaller target than AI researchers at large, you'll kill a much larger fraction of them and reduce their relative power over the future.
Thanks for all the detail, and for looking past my clumsy questions!
It sounds like one disagreement you're pointing at is about the shape of possible futures. You value "humanity colonizes the universe" far less than some other people do. (maybe rob in particular?) That seems sane to me.
The near-term decision questions that brought us here were about how hard to fight to "solve the alignment problem," whatever that means. For that, the real question is about the difference in total value of the future conditioned on "solving" it and conditioned on "not sol...
I guess by "civilization" I meant "civilization whose main story is still being meaningfully controlled by humans who are individually similar to modern humans". Other than that, I just mean your current expectations about what that civilization is like, conditioned on it existing.
(It seems like you could be disagreeing with "a lot of people here" about what those futures look like or how valuable they are or both -- I'd be happy to get clarification on either front.)
Holden Karnofsky has written some about average quality of life, including talking about that chart.
https://www.cold-takes.com/has-life-gotten-better/
I think he thinks that the zero point was crossed long before 1900, but I'm not sure.
I think the phrase "10,000 years of misery" is perfectly consistent with believing that the changes were net good due to population growth, and "misery" is pretty much equivalent to "average quality of life".
I mostly agree with swarriner, and I want to add that writing out more explicit strategies for making and maintaining friends is a public good.
The "case clinic" idea seems good. This sometimes naturally emerges among my friends, and trying to do it more would probably be net positive in my social circles.
Requiring to be finite is just part of assuming the form a probability distribution over worlds. I think you're confused about the type difference between the and the utility of . (Where in the context of this post, the utility is just represented by an element of a poset.)
I'm not advocating for or making arguments about any fanciness related to infinitesimals or different infinite values or anything like that.
L is not equal to infinity; that's a type error. L is equal to 1/2 A_0 + 1/4 A_1 + 1/8 A_2 ...
is a bona fide vector space -- addition behaves as you expect. The points are infinite sequences (x_i) such that is finite. This sum is a norm and the space is Banach with respect to that norm.
Concretely, our interpretation is that x_i is the probability of being in world A_i.
A utility function is a linear functional, i.e. a map from points to real numbers such that the map commutes with addition. The space of continuous linear functionals...
The sum we're rearranging isn't a sum of real numbers, it's a sum in . Ignoring details of what means... the two rearrangements give the same sum! So I don't understand what your argument is.
Abstracting away the addition and working in an arbitrary topological space, the argument goes like this: . For all Therefore, f is not continuous (else 0 = 1).
that's what the entire post is about?