All of Vitor's Comments + Replies

Domain: Farming Construction and Craftsmanship

Link: Simple off grid Cabin that anyone can build & afford: https://www.youtube.com/watch?v=bOOXmfkXpkM (and many other builds on his channel)

Person: Dave Whipple

Background: Construction contractor, DIY living off-grid in Alaska and Michigan.

Why: He and his wife bootstrapped themselves building their own cabin, then house, sell at a profit, rinse and repeat a few times. There are many, many videos of people building their own cabins, etc. Dave's are simple, clear, lucid, from a guy who's done it many times and has skin in the game.

I agree that points 12 and 13 are at least mildly controversial. From the PoV of someone adopting these rules, it'd be enough if you changed the "will"s to "may"s.

By and large, the fewer points that are binding for the market creator, the easier it is to adopt the rules. I'm fine with a few big points being strongly binding (e.g. #15), and also fine with the more aspirational points where "Zvi's best judgement" automatically gets replaced with "Vitor's best judgement". But I'd rather not commit to some minutiae I don't really care about.

(It's more about "attack surface" or maybe in this case we should say "decision surface" than actual strong disagreement with the points, if that makes sense?)

Very interesting read, thank you!

How did you end up doing this work? Did you deliberately seek it out? What are teachers, probation officers and so on (everyone who is not a guard) like? What drives them?

This kind of thing has existed (for example optimal hardware layout) for decades. It sounds a lot less impressive when you sub out "AI" for "algorithm".

"for certain aspects of computer science, computer scientists are already worse than even naive sorting algorithms". Yes, we know that machines have a bunch of advantages over humans. Calculation speed and huge, perfect memory being the most notable.

Where on earth are you pulling those predictions about GPT-5 and 6 from? I'd take the other side of that bet.

The original chart is misleading in more ways than one. Facebook, Netflix et al might be household names now, but this has more to do with their staying power and network effects than any sort of exceedingly fast adoption.

I also suspect that chatGPT has a bunch of inactive accounts, as it's essentially a toy without an actual use-case for most people.

Recognise that almost all the Kolmogorov complexity of a particular simulacrum is dedicated to specifying the traits, not the valences. The traits — polite, politically liberal, racist, smart, deceitful — are these massively K-complex concepts, whereas each valence is a single floating point, or maybe even a single bit!

A bit of a side note, but I have to point out that Kolmogorov complexity in this context is basically a fake framework. There are many notions of complexity, and there's nothing in your argument that requires Kolmogorov specifically.

It seems to me that you are attempting to write a timeless, prescriptive reference piece. Then a paragraph sneaks in that is heavily time and culture dependent.

I'm honestly not certain about the intended meaning. I think you intent mask wearing to be an example of a small and reasonable cost. As a non-american, I'm vaguely aware what costco is, but don't know if there's some connotation or reference to current events that I'm missing. And if I'm confused now, imagine someone reading this in 2030...

Without getting into the object-level discussion, I think such references have no place in the kind of post this is supposed to be, and should be cut or made more neutral.

You didn't address the part of my comment that I'm actually more confident about. I regret adding that last sentence, consider it retracted for now (I currently don't think I'm wrong, but I'll have to think/observe some more, and perhaps find better words/framing to pinpoint what bothers me about rationalist discourse).

It's analogous to a customer complaining "if Costco is going to require masks, then I'm boycotting Costco."  All else being equal, it would be nice for customers to not have to wear masks, and all else being equal, it would be nice to lower the barrier to communication such that more thoughts could be more easily included.

Just a small piece of feedback. This paragraph is very unclear, and it brushes on a political topic that tends to get heated and personal.

I think you intended to say that the norms you're proposing are just the basic cost of en... (read more)

One thing I've read somewhere is that people who sign but aren't deaf, tend to use sign language in parallel with spoken language. That's an entire parallel communications channel!

Relatedly, rationalists lean quite heavily towards explicit ask/tell culture. This is sometimes great, but often clunky: "are you asking for advice? I might have some helpful comments but I'm not sure if you actually want peoples' opinions, or if you just wanted to vent."

Combining these two things, I see possible norms evolving where spoken language is used for communicating comp... (read more)

I think you're confusing arrogance concerning the topic itself with communicating my insights arrogantly. I'm absolutely doing the latter, partly as a pushback to your overconfident claims, partly because better writing would require time and energy I don't currently have. But the former? I don't think so.

Re: the Turing test. My apologies, I was overly harsh as well. But none of these examples are remotely failing the Turing test. For starters, you can't fail the test if you're not aware you're taking it. Should we call anyone misreading some text or getti... (read more)

fair enough, I can see that reading. But I didn't mean to say I actually believe that, or that it's a good thing. More like an instinctive reaction.

It's just that certain types of life experiences put a small but noticeable barrier between you and other people. It was a point about alienation, and trying to drive home just how badly typical minding can fail. When I barely recognize my younger self from my current perspective, that's a pretty strong example.

Hope that's clearer.

What you said, exactly, was:

Just hope you at least briefly consider that I was exactly at your stage one day

which is what I was responding to. I know you're not claiming that I'm 100% hackable, but yet you insist on drawing strong parallels between our states of mind, e.g., that being dismissive must stem from arrogance. That's the typical-minding I'm objecting to. Also, being smart has nothing to do with it, perhaps you might go back and carefully re-read my original comment.

The Turing test doesn't have a "reading comprehension" section, and I don't p... (read more)

I read your original post and I understood your point perfectly well. But I have to insist that you're typical-minding here. How do you know that you were exactly at my stage at some point? You don't.

You're trying to project your experiences to a 1-dimensional scale that every human falls on. Just because I dismiss a scenario, same as you did, does not imply that I have anywhere near the same reasons / mental state for asserting this. In essence, you're presenting me with a fully general counterargument, and I'm not convinced.

So, are all rationalists 70% susceptible, all humans? specifically people who scoff at the possibility of it happening to them? what's your prior here?

100 hours also seems to be a pretty large number. In the scenario in question, not only does a person need to be hacked at 100h, but they also need to decide to spend hour 2 after spending hour 1, and so on. If you put me in an isolated prison cell with nothing to do but to talk to this thing, I'm pretty sure I'd end up mindhacked. But that's a completely different claim.

Thanks for posting this, I recognize this is emotionally hard for you. Please don't interpret the rest of this post as being negative towards you specifically. I'm not trying to put you down, merely sharing the thoughts that came up as I read this.

I think you're being very naive with your ideas about how this "could easily happen to anyone". Several other commenters were focusing on how lonely people specifically are vulnerable to this. But I think it's actually emotionally immature people who are vulnerable, specifically people with a high-openness, "taki... (read more)

Please do tell what those superpowers are!

I definitely think so. The Turing test is a very hard target to hit, and we don't really have a good idea how to measure IQ, knowledge, human-likeness, etc. I notice a lot of confusion, anthropomorphizing, bad analogies, etc in public discourse right now. To me it feels like the conversation is at a level where we need more precise measures that are human and machine compatible. Benchmarks based on specific tasks (as found in AI papers) don't cut it.

(ep status: speculative) Potentially, AI safety folks are better positioned to work on these foundational issues than traditional academics, who are very focused on capabilities and applications right now.

I'm not buying the premise. Passing the Turing test requires to fool an alert, smart person who is deliberately probing the limits of the system. ChatGPT isn't at that level.

A specially tuned persona that is optimized for this task might do better than the "assistant" persona we have available now, but the model is currently incapable of holding a conversation without going on long, unwanted tangents, getting trapped in loops, etc.

The name I have in my head for this is "zones of control". In board- and videogames, sometimes a unit explicitly has an effect on tiles adjacent to its own. I expanded the term from there to include related phenomena, for example where the mere existence of strategy X blocks strategy Y from ever being played, even if X itself is almost never played either. X is in some sense providing "cover fire", not achieving anything directly, but pinning down another strategy in the process.

This case doesn't match that intuition exactly, but it's in the same neighborhood.

The difference between regulation and research is that the former has a large amount of friction, making it about as hard to push a 1% regulation through as a 10% one.

In contrast, the incremental 1% improvements in the development of capabilities is just what happens by default, as research organizations follow their charter.

Agreed. My main objection to the post is that it considers the involved agents to be optimizing for far future world-states. But I'd say that most people (including academics and AI lab researchers) mostly only think of the next 1% step in front of their nose. The entire game theoretic framing in the arms race etc section seems wrong to me.

Mmm, I would say the general shape of your view won't clash with reality, but the magnitude of the impact will.

It's plausible to me that a smart buyer will go and find the best deal for you when you tell it to buy laptop model X. It's not plausible to me that you'll be able to instruct it "buy an updated laptop for me whenever a new model comes out that is good value and sufficiently better than what I already have," and then let it do its thing completely unsupervised (with direct access to your bank account). That's what I mean by multiple complicated ob... (read more)

OK, well, you should retract your claim that the median LW timeline will soon start to clash with reality then! It sounds like you think reality will look basically as I predicted! (I can't speak for all of LW of course but I actually have shorter timelines than the median LWer, I think.)

I retract the claim in the sense that it was a vague statement that I didn't expect to be taken literally, which I should have made clearer! But it's you who operationalized "a few years" as 2026 and "the median less wrong view" as your view.

Anyway, I think I see the ou... (read more)

I do roughly agree with your predictions, except that I rate the economic impact in general to be lower. Many headlines, much handwringing, but large changes won't materialize in a way that matters.

To put my main objection succinctly, I simply don't see why AGI would follow soon from your 2026 world. Can you walk me through it?

Sure, let me do this as an exercise (ep stat: babble mode). Your predicions are pretty sane overall, but I'd say you handwave away problems (like integration over a variety of domains, long-term coherent behavior, and so on) that I see as (potentially) hard barriers to progress.

2022

• 2022 is basically over and I can't get a GPT instance to order me a USB stick online.

2023

• basically agree, this is where we're at right now (perhaps with the intensity turned down a notch)

2024

• you're postulating that "It’s easy to make a bureaucracy and fine-tune it and

This is reasonably close to my beliefs. An additional argument I'd like to add is:

• Even if superintelligence is possible, the economic path towards it might be impossible.

There needs to be an economically viable entity pushing AI development forward every step of the way. It doesn't matter if AI can "eventually" produce 30% worldwide GPD growth. Maybe diminishing returns kick in around GPT-4, or we run out of useful training data to feed to the models (We have very few examples of +6 SD human reasoning, as MikkW points out in a sibling comment).

Analogy:... (read more)

Nice! But we're still missing a capability, namely causing the model to respond to a specific prompt, not just output an arbitrary unsafe thing.

How likely is it that this becomes a legal problem rendering models unable to be published? Note that using models privately (even within a firm) will always be an option, as copyright only applies to distribution of the work.

This is an interesting thought, but it seems very hard to realize as you have to distill the unique contribution of the sample, as opposed to much more widespread information that happens to be present in the sample.

Weight updates depend heavily on training order of course, so you're really looking for something like the Shapley value of the sample, except that "impact" is liable to be an elusive, high-dimensional quantity in itself.

I also tend to find myself arguing against short timelines by default, even though I feel like I take AI safety way more seriously than most people.

At this point, how many people with long timelines are there still around here? I haven't explicitly modeled mine, but it seems clear that they're much, much longer (with significant weight on "never") than the average less wronger. The next few years will for sure be interesting as we see the "median less wrong timeline" clash with reality.

For meth it lists an ingredient (ether) that it doesn't actually use. And actual lab protocols are much more detailed about precise temperature, times, quantities, etc.

(ep stat: it's hard to model my past beliefs accurately, but this is how I remember it)

I mean, it's unsurprising now, but before that series of matches where AlphaStar won, it was impossible.

Maybe for you. But anyone who has actually played starcraft knows that it is a game that is (1) heavily dexterity capped, and (2) intense enough that you barely have time to think strategically. It's all snap decisions and executing pre-planned builds and responses.

I'm not saying it's easy to build a system that plays this game well. But neither is it paradigm-chan... (read more)

I agree, and I don't use this argument regarding arbitrary AI achievements.

But it's very relevant when capabilities completely orthogonal to the AI are being sold as AI. The Starcraft example is more egregious, because AlphaStar had a different kind of access to the game state than a human has, which was claimed to be "equivalent" by the deepmind team. This resulted in extremely fine-grained control of units that the game was not designed around. Starcraft is partially a sport, i.e., a game of dexterity, concentration, and endurance. It's unsurprising that... (read more)

I find this example interesting but very weird. The couple is determining fairness by using "probability mass of happiness" as the unit of account. But it seems very natural to me to go one step further and adjust for the actual outcomes, investing more resources into the sub-agent that has worse luck.

I don't know if this is technically doable (I foresee complications with asymmetric utility functions of the two sub-agents, where one is harder to satisfy than the other, or even just has more variance in outcomes), but I think such an adjustment should recover the VNM independence condition.

This confirms the suspicions I had upon hearing the news: the diplomacy AI falls in a similar category than the Starcraft II AI (AlphaStar) we had a while back.

"Robot beats humans in a race." Turns out the robot has 4 wheels and an internal combustion engine.

While I think this post overall gives good intuition for the subject, it also creates some needless confusion.

Your concept of "abstract entropy" is just Shannon entropy applied to uniform distributions. Introducing Shannon entropy directly, while slightly harder, gives you a bunch of the ideas in this post more or less "for free":

• Macrostates are just events and microstates are atomic outcomes (as defined in probability theory). Any rules how the two relate to each other follow directly from the foundations of probability.

• The fact that E[-log x] is th

Probabilistic Turing machines already exist. They are a standard extension of TMs that transition from state to state with arbitrary probabilities (not just 0 and 1) and can thus easily generate random bits.

Further, we know that deterministic TMs can do anything that probabilistic TMs can, albeit potentially less efficiently.

I suspect you're not really rejecting TMs per se, but rather the entire theory of computability and complexity that is standard in CS, more specifically the Church-Turing thesis which is the foundation of the whole thing.

It's more the latter. At the local level, salespeople are supposed to have a very good handle of their clients' future demand. A lot of the trade is also seasonal, and rises and falls in a correlated way (e.g. fruit exports will have good/bad years depending on climate)

When it comes to overall economic trends, I don't really know. That stuff was handled way above my paygrade by global HQ. But it definitely can happen that a certain route requests many empty boxes to be shipped to them, only to have them sitting there because projected demand did not materi... (read more)

Exactly. The worst transatlantic flight I ever had was one where I paid for "extra legroom". turns out it was a seat without a seat in front, i.e., the hallway got broader there.

However, other passengers and even the flight attendants certainly didn't act like this extra legroom belonged to me. Someone even stepped on my foot! On top of that I had to use an extremely flimsy table that folded out of the armrest.

Since most of us aren't weekly business flyers, this is a far cry from a free market.

Yes, it's mostly a diagnosis of exclusion. But Bayesian evidence starts piling up much sooner than a doctor is willing to write down a diagnosis on an Official Piece of Paper. However, there are some tell-tale signs like the myofascial trigger points mentioned by others, heightened pain when touching specific bones (e.g. the vertebrae), and other specific patterns how the body reacts to stimulus. This is the domain of rheumatologists.

Are your sleep issues stress-related? Like jumpiness, unable to settle into the relaxation of falling asleep? What I'm getti... (read more)

This story has been adapted into a (relatively faithful) movie: https://en.wikipedia.org/wiki/North_Face_(film)

For what it's worth, being afraid of others' judgements is a very normal thing. It's also pretty normal that it gets exaggerated when one is isolated.

Now, you are a clear outlier along that dimension, but I think I can empathize with your situation at least a little bit, based on my own experiences with isolation, of which there are two: (1) for the last few years, due to complicated health issues I won't go into right now, I am much less socially active than I'd like to be. Constantly cancelling on my friends, and being "more work" to be around has consis... (read more)

Our base expectation for asymptomatic spread should be quite low, because previous variants of monkeypox and smallpox (mostly) didn't spread like that. So I disagree with your "MSM with AIDS" scenario. It wouldn't be that surprising for the spread to be contained to the particularly vulnerable AIDS population.

This phrasing bothers me a bit. It presupposes that it is only a matter of time; that there's no error about the nature of the threat AGI poses, and no order-of-magnitude error in the timeline. The pessimism is basically baked in.

Right, but we wouldn't then use this as proof that our children are precocious politicians!

In this discussion, we need to keep separate the goals of making GPT-3 as useful a tool as possible, and of investigating what GPT-3 tells us about AI timelines.

It is definitely misleading, in the same sense that the performance of a model on the training data is misleading. The interesting question w.r.t. GPT-3 is "how well does it perform in novel settings?". And we can't really know that, because apparently even publicly available interfaces are inside the training loop.

Now, there's nothing wrong with training an AI like that! But the results then need to be interpreted with more care.

P.S.: sometimes children do parrot their parents to an alarming degree, e.g., about political positions they couldn't possibly have the context to truly understand.