Comment Permalink

I'd love your feedback on my thoughts on decision theory.

If you're trying to get a sense of my approach in order to determine whether it's interesting enough to be worth your time, I'd suggest starting with this article (3 minute read).

I'm also considering applying for funding to create a conceptual alignment course.

2cousin_it1y

I don't really understand your approach yet. Let's call your decision theory CLDT. You say counterfactuals in CLDT should correspond to consistent universes. For example, the counterfactual "what if a CLDT agent two-boxed in Newcomb's problem" should correspond to a consistent universe where a CLDT agent two-boxes on Newcomb's problem. Can you describe that universe in more detail?

Chris_Leong1y20

I don't want to universally claim that we should always work with consistent universes, but I think we should have a strong bias toward reformulating problems to be consistent when we can. This involves imagining that the past is different in the counterfactual.

2Chris_Leong1y

That's not quite what I wrote in this article: I'll respond to the other component of your question later.

See in context

46 I'm open for projects (sort of)

by cousin_it

18th Apr 2024

1 min read

46

I left Google a month ago, and right now don't work. Writing this post in case anyone has interesting ideas what I could do. This isn't an "urgently need help" kind of thing - I have a little bit of savings, right now planning to relax some more weeks and then go into some solo software work. But I thought I'd write this here anyway, because who knows what'll come up.

Some things about me. My degree was in math. My software skills are okayish: I left Google at L5 ("senior"), and also made a game that went semi-viral. I've also contributed a lot on LW, the most prominent examples being my formalizations of decision theory ideas (Löbian cooperation, modal fixpoints etc) and later the AI Alignment Prize that we ran with Paul and Zvi. Most of that was before the current AI wave; neural networks don't really "click" with my mind, so I haven't done much work on them.

And yeah, this is an invitation to throw at me not necessarily money-paying work, but also stuff you'd like me to look at, criticize, help with your own projects and so on. I find myself with a bit more free time now, so basically drop me a PM if you have something interesting to talk about :-)

Practical

Personal Blog

46

I'm open for projects (sort of)

New Comment

13 comments, sorted by

top scoring

Click to highlight new comments since: Today at 3:22 AM

[-]Adam Zerner1y30

What are your goals? Money? Impact? Meaning? To what extent?

I think it'd also be helpful to elaborate on your skillset. Front end? Back end? Game design? Mobile apps? Design? Product? Data science?

[-]cousin_it1y*20

Yeah, that might be a good idea in case any rich employers stumble on this :-)

In terms of goals, I like making something, having many people use it, and getting paid for it. I'm not as motivated by meaning, probably different from most EAs in that sense.

In terms of skillset, I'd say I'm a frontend-focused generalist. The most fun programming experience in my life was when I built an online map just by myself - the rendering of map data to png tiles, the serving backend, the javascript for dragging and zooming, there weren't many libraries back then - and then it got released and got hundreds of thousands of users. The second most fun was when I made the game - coming up with the idea, iterating on the mechanics, graphic design, audio programming, writing text, packaging for web and mobile, the whole thing - and it got quite popular too. So that's the prototypical good job for me.

[-]Adam Zerner1y20

Hm, gotcha.

It's tough, I think there are a lot of tradeoffs to navigate.

You could join a big company. You'll 1) get paid, 2) work on something that lots of people use, but 3) you'll be a small cog in a large machine, and it sounds like that's not really what you're looking for. It sounds like you enjoy autonomy and having a meaningful and large degree of ownership.
You could work on your own project. That addresses 3. But then 1 and 2 become pretty big risks. It's hard to build something that makes good money and lots of people use.
You could join an open source project that lots of people use and is lacking contributors. But there's often not really a path to getting paid there.
- Something interesting: https://fresh.deno.dev/. I really like what they're doing. I personally think it's the best web framework out there. And there's only one person working on it. He's an incredible developer. Deno is paying him to work on it. I'm not sure if they'd be open to paying a second contributor. And I am not too optimistic that Fresh will become something that many people use.
Working on LessWrong is an interesting possibility. After all, you're a longtime user and have the right skillset. However, 1) I'm not sure how good the prospects are for getting paid, 2) it's a relatively small community so you wouldn't be getting that "tons and tons of people use something I built" feeling, and 3) given that it's later stage and there's a handful of other developers working on it, I'm not sure if it'd provide you enough feeling of ownership.
Joining a small company seems like the most realistic way to get 1, 2 and 3, but the magnitude of each might not be idea: smaller companies tend to pay less, have fewer users, and still have enough employees such that you don't really have that much ownership.

My best guess is that starting your own company would be best. Something closer to an indie hacker-y lifestyle SaaS business than a "swing for the fences" VC-backed business. The latter is probably better if you're earning to give and looking to maximize impact, but since you're leaning more towards designing a good life for yourself, I think the former is better, and I also think most people would agree with that. I've seen a lot of VC's be very open about the fact that the "swing for the fences" approach is frequently not actually in the founder's interest.

I'm looking to do the lifestyle SaaS business thing right now btw. If you're interested in that I'd love to chat: shoot me a DM.

[-]Ben Pace1y31

Oh cool! Um, first thought, register interest in this?

[-]cousin_it1y20

Done! I didn't do it at first because I thought it'd have to be in person only, but then clicked around in the form and found that remote is also possible.

[-]jacquesthibs1y20

I'm hoping to collaborate with some software engineers who can help me build an alignment research assistant. Some (a little bit outdated) info here: Accelerating Alignment. The goal is to augment alignment researchers using AI systems. A relevant talk I gave. Relevant survey post.

What I have in mind also relates to this post by Abram Demski and this post by John Wentworth (with a top comment by me).

Send me a DM if you (or any good engineer) are reading this.

[-]Chris_Leong1y20

[-]cousin_it1y20

[-]Chris_Leong1y20

You say counterfactuals in CLDT should correspond to consistent universes

That's not quite what I wrote in this article:

However, this now seems insufficient as I haven't explained why we should maintain the consistency conditions over comparability after making the ontological shift. In the past, I might have said that these consistency conditions are what define the problem and that if we dropped them it would no longer be Newcomb's Problem... My current approach now tends to put more focus on the evolutionary process that created the intuitions and instincts underlying these incompatible demands as I believe that this will help us figure out the best way to stitch them together.

I'll respond to the other component of your question later.

[-]Viliam1y20

Besides math and programming, what are your other skills and interests?

I have an idea of a puzzle game, not sure if it would be good or bad, I haven't done even a prototype. So if anyone is interested, feel free to try... I hope I can explain it sufficiently clearly in words...

The game plan is divided into squares; I imagine a typical level to be between 10x10 and 30x30 squares large. Each square is either empty, or contains an immovable wall, or contains a movable block. The game consists of moving the blocks. Each move = you click a specific block, and try dragging it in one of the 4 directions, and either it is possible or not.

A block cannot move into a wall. A block can push another block. A block does not pull another block. For example, if there are 3 blocks in a horizontal line, and you click the middle one and try dragging it to the left, two blocks will move and the third one (the one on the right) will stay there. So far, it should be completely obvious, like what you would happen if you moved some actual objects.

In addition, each side of a block (or a wall) may be empty, or may contain a colored "magnet" (or perhaps a "lock" is a better metaphor). These add the following constraints for the movement of blocks:

Magnets of different colors can never touch each other. If one block has a green magnet on the right side, and another has a blue magnet on the left side, you cannot put them next to each other so that the magnets would touch. (If you try to do that, the block refuses to move. Graphically, I imagine that it would move like half the way, and then you would get a visual indicator where is the problem, and when you stop dragging, it will return to its original place.) Though it is okay if the blocks touch on their other sides, where they don't have magnets.
Magnets of the same color cannot be connected or disconnected by a move in a perpendicular direction. If one block has a green magnet on the right side, and another has a green magnet on the left side, if you move them next to each other, then when you try moving one of them up or down, it drags the other block along with it. Either both blocks move (in a direction perpendicular to their magnetic connection) or neither does. In a direction parallel to the magnetic connection, either one block pushes the other, or they disconnect if you pull them apart (i.e. the magnets do nothing when moving in a parallel direction).
A magnet can touch a side without a magnet, doing so has no effect as if the magnet is not there.

Or, to describe it more like a programmer:

You choose a block and a direction to move. Now we create a set of "blocks that will move one step in given direction" like this: At first, the set contains the selected block. For each block in the set, a block next to it in the selected direction is also added to the set (pushed by the previous block). For each block in the set, a block next to it in a perpendicular direction is also added to the set if they are connected by magnets of the same color. We keep applying these two rules until we can add no more blocks to the set.
Now we check what would happen if blocks in the set moved one step in given direction, and all other blocks stayed at their place. If any block would move into a wall, the entire move is cancelled. (A block cannot move into another block, because by the set creation algorithm, that other block would also be in the set, and thus it would also move.) If two blocks -- one that moved, and one that didn't move -- would end up next to each other so that their magnets would touch each other (regardless of their colors), the entire move is cancelled. In both cases, the place that causes the problem is visually indicated to the player. (That is, even if you already know that the move is cancelled, keep checking which other places you also need to highlight. Then move all blocks in the set a few pixels in a given direction, so the player sees which blocks would be pushed along.)
If there is no problem, the blocks in the set all simultaneously move one step the given direction.

I think that these rules are time-reversible; whatever move you make, you can revert it by one or more moves. This is a desirable property, because it means you can never get stuck in the game. (It also means you can automatically generate levels by generating a solution and then making a few hundred random moves.)

A magnet can also be on the side of a wall. (The wall is basically a block that cannot be moved.)

The puzzle is solved when each magnet is connected to a magnet of the same color.

For bonus points, include a visual editor, and maybe an export/import of levels to a text file.

[-]cousin_it1y20

Besides math and programming, what are your other skills and interests?

Playing and composing music is the main one.

I have an idea of a puzzle game, not sure if it would be good or bad, I haven’t done even a prototype. So if anyone is interested, feel free to try

Yeah, you're missing out on all the fun in game-making :-) You must build the prototype yourself, play with it yourself, tweak the mechanics, and at some moment the stars will align and something will just work and you'll know it. There's no way anyone else can do it but you.

[-]Adam Zerner1y20

Kudos for writing this post. I know it's promotional/self-interested, but I think that's fine. It's also pro-social. Having the rule/norm to encourage this type of post seems unlikely to be abused in a net-negative sort of way (assuming some reasonable restrictions are in place).

Moderation Log