All of Pablo Villalobos's Comments + Replies

The arguments you make seem backwards to me.

All this to say, land prices represent aggregation effects / density / access / proximity of buildings. They are the cumulative result of being surrounded by positive externalities which necessarily result from other buildings not land. It is the case that as more and more buildings are built, the impact of a single building to its land value diminishes although the value of its land is still due to the aggregation of and proximity to the buildings that surround it.

Yes, this is the standard Georgist position, and... (read more)

1Blog Alt
I mostly disagree with your example (the use of RV's is nice, although I think it changes the equations too much) but I can grant you (and Malcolm Ocean) the point: >Yes, this is the standard Georgist position, and it's the reason why land owners mainly capture (positive and negative) externalities from land use _around_ them, not in their own land. Might we agree that positive externalities outweigh negative ones as evidenced by high land prices? If we take this position at it's best, at the end of the day, how is taxing the capture of a positive externality (LVT) any better than the taxing the creation of value (SQ)? (Given that the positive externality already exists and would be captured for free under the SQ.) >The question is, which of these effects is bigger? I would say that landowners have more influence over their own land than over surrounding land, so a priori I would expect more density to result from an LVT The effects you've identified I think are fairly comprehensive. However I think that "Moving to a less dense place" is actually a method to reduce density in the surrounding land i.e. picking up all the skyscrapers and spreading them apart (the regularisation effect over density) and is a decision which landowners frequently make.   
5Jiro
That brings up the splitting question. If two people owning adjacent parcels of land each build a garbage dump, they will in fact reduce both of their taxes since they each affect each other's. And if we're going with "the government tracks such things and does calculations to prevent splitting from mattering" then it should be possible to build it on your own land and still get the tax reduction.

We'll be at the ground floor!

Not quite. What you said is a reasonable argument, but the graph is noisy enough, and the theoretical arguments convincing enough, that I still assign >50% credence that data (number of feedback loops) should be proportional to parameters (exponent=1).

My argument is that even if the exponent is 1, the coefficient corresponding to horizon length ('1e5 from multiple-subjective-seconds-per-feedback-loop', as you said) is hard to estimate.

There are two ways of estimating this factor

  1. Empirically fitting scaling laws for whatever task we care about
  2. Reasoning ab
... (read more)
2Daniel Kokotajlo
OK, I think we are on the same page then. Thanks.

Note that you can still get EUM-like properties without completeness: you just can't use a single fully-fleshed-out utility function. You need either several utility functions (that is, your system is made of subagents) or, equivalently, a utility function that is not completely defined (that is, your system has Knightian uncertainty over its utility function).

See Knightian Decision Theory. Part I

Arguably humans ourselves are better modeled as agents with incomplete preferences. See also Why Subagents?

2romeostevensit
from Knightian Decision Theory: This has been discussed here before, but it's a nice succinct description. Immediately following and relevant to this discussion:

Yes, it's in Spanish though. I can share it via DM.

1tamgent
Yes please

I have an intuition that any system that can be modeled as a committee of subagents can also be modeled as an agent with Knightian uncertainty over its utility function. This goal uncertainty might even arise from uncertainty about the world.

This is similar to how in Infrabayesianism an agent with Knightian uncertainty over parts of the world is modeled as having a set of probability distributions with an infimum aggregation rule.

2cfoster0
These might be of interest, if you haven't seen them already: Bewley, T. F. (2002). Knightian decision theory. Part I. Decisions in economics and finance, 25, 79-110. Aumann, R. J. (1962). Utility theory without the completeness axiom. Econometrica: Journal of the Econometric Society, 445-462.

This not the same thing, but back in 2020 I was playing with GPT-3, having it simulate a person being interviewed. I kept asking ever more ridiculous questions, with the hope of getting humorous answers. It was going pretty well until the simulated interviewee had a mental breakdown and started screaming.

I immediately felt the initial symptoms of an anxiety attack as I started thinking that maybe I had been torturing a sentient being. I calmed down the simulated person, and found the excuse that it was a victim of a TV prank show. I then showered them with... (read more)

1tamgent
Do you have the transcript from this?

I think the median human performance on all the areas you mention is basically determined by the amount of training received rather than the raw intelligence of the median human.

1000 years ago the median human couldn't write or do arithmetic at all, but now they can because of widespread schooling and other cultural changes.

A better way of testing this hypothesis could be comparing the learning curves of humans and monkeys for a variety of tasks, to control for differences in training.

Here's one study I could find (after ~10m googling) comparing the learni... (read more)

0DragonGod
I agree that training/learning/specialisation seem more likely to explain the massive gaps between the best and mediocre than innate differences in general cognitive prowess. But I am also genuinely curious what the difference in raw g factor is between median human and +6 SD human and median human and -2 SD human.

I second the other answers that even if we completely solve cybersecurity, there would be substantial AI risk just by having the AI interact with humans, via manipulation, etc.

That said, I think it would close a huge part of the attack surface for the AI. If, in addition to that, suddenly in 2032 we discover how to make humans invulnerable to manipulation, I would feel much better about running experiments with unaligned AI, boxing, etc.

So I'd say it's something like "vastly better cybersecurity is not enough to contain unaligned AGI, but any hope of containing unaligned AGI requires vastly better cybersecurity"

Literally the only thing in the story that lets the AGI win is the nanobots. That's it. All the rest is surperfluous.

Well, if nanobots are possible then they are such a powerful technology that any AGI will eventually want to build them, unless it has something even better. But let's assume that nanobots are impossible and try to build a story.

I'm going to be lazy and use Gwern's premise of an AGI that escapes during training and hacks its way into some money and some hardware to run in. Instead of going fast and doing overt actions, the AGI stays hidden.

I... (read more)

0LGS
Something like that is what I had in mind, but note that: 1. It requires humans to fail to see the AGI "spy" that's embedded into every single powerful computing system, and fail to see this for years. Gwern was assuming humans would catch on in days, so he had his AGI scramble to avoid dying before the nanobots strike. 2. "Surviving a short amount of time without human input" is not enough; the robots need to be good enough to build more robots (and better robots). This involves the robots being good enough to do essentially every part of the manufacturing economy; we are very far away from this, and a company that does it in a year is not so plausible (and would raise alarm bells fast for anyone who thinks about AI risk). You're gonna need robot plumbers, robot electricians, etc. You'll need robots building cooling systems for the construction plants that manufacture robots. You'll need robots to do the smelting of metals, to drive things from factory A to factory B, to fill the gas in the trucks they are driving, to repair the gasoline lines that supply the gas. Robots will operate fork lifts and cranes. It really sounds roughly "human-body complete".

For example, we could simulate a bunch of human-level scientists trying to build nanobots and also checking each-other's work.

That is not passively safe, and therefore not weak. For now forget the inner workings of the idea: at the end of the process you get a design for nanobots that you have to build and deploy in order to do the pivotal act. So you are giving a system built by your AI the ability to act in the real world. So if you have not fully solved the alignment problem for this AI, you can't be sure that the nanobot design is safe unless you are c... (read more)

Q has done nothing to prevent another AGI from being built

Well, yeah, because Q is not actually an AGI and doesn't care about that. The point was that you can create an online persona which no one has ever seen even in video and spark a movement that has visible effects on society.

The most important concern an AGI must deal with is that humans can build another AGI, and pulling a Satoshi or a QAnon does nothing to address this.

Even if two or more AGIs end up competing among themselves, this does not imply that we survive. It probably looks more like Europe... (read more)

0LGS
  Well, you did specifically ask if I would be able to tell if Q were an AGI, and my answer is "yup". I would be able to tell because the movement would start achieving some AGI goals. Or at least I would see some AGI goals starting to get achieved, even if I couldn't trace it down to Q specifically. Wait, you are claiming that an AGI would be able to convince the world AGI is impossible after AGI has already, in fact, been achieved? Nonsense. I don't see a world in which one team builds an AGI and it is not quickly followed by another team building one within a year or two. The AGI would have to do some manipulation on a scale never before observed in history to convince people to abandon the main paradigm -- one that's been extraordinarily successful until the end, and one which does, in fact, work -- without even one last try. Of course. We would eventually reach fully automated luxury space communism by ourselves, even without AGI. But it would take us a long time, and the AGI cannot afford to wait (someone will build another AGI, possibly within months of the first).   That's exactly what motivated my question! I read it, and I suddenly realized that if this is how AGI is supposed to win, perhaps I shouldn't be scared after all. It's totally implausible. Prior to this, I always assumed AGI would win easily; after reading it, I suddenly realized I don't know how AGI might win at all. The whole thing sounds like nonsense. Like, suppose the AGI coordinates social media attacks. Great. This lasts around 5 seconds before AI researchers realize they are being smeared. OK, so they try to communicate with the outside world, realize they are being blocked on all fronts. Now they know they are likely dealing with AGI; no secrecy for the AGI at this point. How long can this stay secret? A couple days? Maybe a couple weeks? I can imagine a month at most, and even that is REALLY stretching it. Keep in mind that more and more people will be told in person about this, s

It's somewhat easier to think of scenarios where the takeover happens slowly.

There's the whole "ascended economy" scenarios where AGI deceptively convinces everyone that it is aligned or narrow, is deployed gradually in more and more domains, automates more and more parts of the economy using regular robots until humans are not needed anymore, and then does the lethal virus thing or defects in other way.

There's the scenario where the AGI uploads itself into the cloud, uses hacking/manipulation/financial prowess to sustain itself, then uses manipulation to ... (read more)

0LGS
  That would be incredibly risky for the AGI, since Q has done nothing to prevent another AGI from being built. The most important concern an AGI must deal with is that humans can build another AGI, and pulling a Satoshi or a QAnon does nothing to address this. I personally would likely notice: anyone who successfully prevents people from building AIs is a high suspect of being an AGI themselves. Anyone who causes the creation of robots who can mine coal or something (to generate electricity without humans) is likely an AGI themselves. That doesn't mean I'd be able to stop them, necessarily. I'm just saying, "nobody would notice" is a stretch.   I agree that the AGI could build a cultish following like Yudkowsky did.

Some things that come to mind, not sure if this is what you mean and they are very general but it's hard to get more concrete without narrowing down the question:

  • Goodharting: you might make progress towards goals that aren't exactly what you want. Perhaps you optimize for getting more readers for your blog but the people you want to influence end up not reading you.
  • Value drift: you temporarily get into a lifestyle that later you don't want to leave. Like starting a company to earn lots of money but then not wanting to let go of it. I don't know if this
... (read more)
1[comment deleted]

I'm not sure if using the Lindy effect for forecasting x-risks makes sense. The Lindy effect states that with 50% probability, things will last as long as they already have. Here is an example for AI timelines.

The Lindy rule works great on average, when you are making one-time forecasts of many different processes. The intuition for this is that if you encounter a process with lifetime T at time t<T, and t is uniformly random in [0,T], then on average T = 2*t.

However, if you then keep forecasting the same process over time, then once you surpass T/2 you... (read more)

2Measure
Don't both strategies start out with forecasts < 10 years and act immediately? Actually, calculating the utility-maximizing decision depends on the distribution forecast, not just the best-guess point estimate.

Wait, how is Twilight Princess a retro game? It's only been 16 years! I'm sorry but anything that was released during my childhood is not allowed to be retro until I'm like 40 or so.

Let me put on my sciency-sounding mystical speculation hat:

Under the predictive processing framework, the cortex's only goal is to minimize prediction error (surprise). This happens in a hierarchical way, with predictions going down and evidence going up, and upper levels of the hierarchy are more abstract, with less spatial and temporal detail.

A visual example: when you stare at a white wall, nothing seems to change, even though the raw visual perceptions change all the time due to light conditions and whatnot. This is because all the observations are con... (read more)

Why do you think this sort of training environment would produce friendly AGI?
Can you predict what kind of goals an AGI trained in such an environment would end up with?
How does it solve the standard issues of alignment like seeking convergent instrumental goals?

1eg
We have unpredictable changing goals and so will they. Instrumental convergence is the point. It's positive-sum and winning to respectfully share our growth with them and vice-versa, so it is instrumentally convergent to do so.

Re: April 5: TV host calls for killing as many Ukrainians as possible.

I know no Russian, but some people in the responses are saying that the host did not literally say that. Instead he said some vague "you should finish the task" or something like that. Still warmongering, but presumably you wouldn't have linked it if the tweet had not included the "killing as many Ukrainians as possible" part.

Could someone verify what he says?

3sciuru
Also important. Neither of these claims could be inferred from the article alone. Below is an excerpt, characteristic of official anti-western narrative (with my translation): Коллективный Запад сам является проектировщиком, источником и спонсором украинского нацизма, в то время как западенские бандеровские кадры и их "историческая память" — лишь один из инструментов нацификации Украины.  Collective West is a designer, source and sponsor of Ukrainian Nazism, whereas the western Ukrainian Banderites with their "historical memory" - is merely one of the tools [among others] of nazification of Ukraine.    West/Us mentions are scattered throughout the article, here's tldr:  * West and US have total control over Ukraine, they've inflated its artificial sense of separate national identity to pit it against Russia. Ukraine is totally instrumental  * West and US as civilizations follow a path of degradation and lure others onto it  * West and US are a superpower-colonizer of the world 
5sciuru
As delirious as the article "What Russia must do to Ukraine" clearly is, I want to point out incorrect translations and filtered-evidence style omissions from it. This might seem like a petty nitpicking, moral-wise, but since Zvi did choose to cite this "translation" in the first place (instead of just mentioning the gist of it), I think it warrants correction.   I chose a couple of fragments from original and wrote them down in Russian. * M - my literal translation (which probably sounds awkward, but is optimized for accuracy) * S - "Short" translation from this twitter thread  * L - "split the difference" (aka "Long") translation from this twitter thread    1. Нацисты, взявшие в руки оружие, должны быть по максимуму уничтожены на поле боя.  M) Nazis who took up arms, must be maximally eliminated on the battlefield. S) Nazis who took weapons, must be killed in numbers as much as possible... L) First, he goes, everyone who took arms against Russia must be eliminated. Totally. No trials...   2. Однако, помимо верхушки, виновна и значительная часть народной массы, которая является пассивными нацистами, пособниками нацизма. Они поддерживали нацистскую власть и потакали ей. Справедливое наказание этой части населения возможно только как несение неизбежных тягот справедливой войны против нацистской системы, ведущейся по возможности бережно и осмотрительно в отношении гражданских лиц.   M) However, besides the top ranks, a significant fraction of the masses is guilty too, they [the masses] are passive Nazis, abettors of Nazis. They supported Nazis' rule and pandered to it. Just punishment of this part of population is possible only through bearing [by this part] of inexorable burdens of just war against Nazi system, which is waged, whenever possible, with care and caution toward civilians.  S) Not just the elites, the most of the people are guilty, they are passive Nazis, Nazi enablers. They supported these elites and must be punished L)"A substantial part

I am Russian and I can confirm that he most certainly did not call for "killing of as many Ukrainians as possible".

He said that there can be no negotiations with Nazis outside of putting a boot on their neck because it will be seen as weakness, and that you shouldn't try to have "small talk" with them, or shake hands with them. He did say "a task is set and it should be finished". He did not explicitly say anything about killing, let alone "as many as possible", at least not in that clip.

It seems like one literally cannot trust anyone to report information about this war accurately.

I'm sorry, but I find the tone of this post a bit off-putting. Too mysterious for my taste. I opened the substack but it only has one unrelated post.

I don’t think there is a secular way forward.

Do you think that there is a non-secular way forward? Did you previously (before your belief update) think there is a non-secular way forward?

We just shamble forward endlessly, like a zombie horde devouring resources, no goal other than the increase of some indicator or other.

I can agree with this, but... those indicators seem pretty meaningful for me. Life expectan... (read more)

2Carlos Ramirez
The Presence of Everything has two posts so far, and both are examples of the sort of panoramic analogical thinking we need to undo the various Gordian Knots bedeviling us.
5lsusr
Disease is down. War is down. Poverty is down. Democracy is up (on the timescale of centuries). Photovoltaics are cheaper than coal. This all seems worthwhile to me. If world peace, health and prosperity aren't worthwhile then what is? Masters of the contemplative arts are obsessed with compassion. They're generally supportive of making the material world a better place. The Dalai Lama and Daniel Ingram support scientific advancement. The QRI is even inventing indicators to quantify weird contemplative stuff. I don't think there's a conflict between weird contemplative stuff and making the world better in a measurable way. If the two conflict then you're doing the contemplative stuff wrong. Pursuing one kind of good doesn't invalidate other kinds of good. To the contrary, I think the perceived invalidation is a useful way to distinguish good people from evil. When evil people see good, they try to undermine it. When good people see good, they celebrate it.
2Carlos Ramirez
  Yes, I did always think there was a non-secular way forward for all sorts of problems. It's just that I realized AI X-risk is just one head of an immense hydra: technological X-risks. I'm more interested in slaying that hydra, than in coming up with ways to deal with just one of its myriad heads. Yeah, the indicators are worth something, but they are certainly not everything! Slavish devotion to the indicators renders one blind to critical stuff, such as X-risk, but also to things like humanity becoming something hideous or pathetic in the future. The hard problem of consciousness combined with learning the actual tenets of Hinduism (read the Ashtavakra Gita), was the big one for me. Dostoyevsky also did a bang up job depicting the spiritual poverty of the materialist worldview.

Let me paraphrase your argument, to see if I've understood it correctly:

  • Physical constraints on things such as energy consumption and dissipation imply that current rates of economic growth on Earth are unsustainable in the relatively short term (<1000 years), even taking into account decoupling, etc.

  • There is a strong probability that expanding through space will not be feasible

  • Therefore, we can reasonably expect growth to end some time in the next centuries

First of all, if economic progress keeps being exponential then I think it's quite po... (read more)

2invertedpassion
Yes, you've paraphrased the argument well. While progress for any certain technology can certainly be exponential, the benefit or impact of it may not be exponential if we assume that the lowest hanging fruits (in terms of cost/benefit tradeoff) gets plucked first, and then it becomes harder and harder to derive economically relevant benefit even with tech progress. What unbounded benefit relies on whether we can keep on discovering new types of general-purpose technologies, and not just an improvement in current technologies. E.g. faster air travel depends on some new uncertain technology to be invented. We can only squeeze incremental benefits from further improving the current air travel technologies.  Also see Eroom's law which suggests declining R&D productivity for drug discovery. Whether we can keep on discovering such general-purpose technologies is pure speculation, so there's some probability that we may actually not be in a weird cross-roads within the next 200 years.

So, assuming the neocortex-like subsystem can learn without having a Judge directing it, wouldn't that be the perfect Tool AI? An intelligent system with no intrinsic motivations or goals?

Well, I guess it's possible that such a system would end up creating a mesa optimizer at some point.

4Steven Byrnes
A couple years ago I spent a month or two being enamored with the idea of tool AI via self-supervised learning (which is basically what you're talking about, i.e. the neocortex without a reward channel), and I wrote a few posts like In Defense of Oracle ("Tool") AI Research and Self-Supervised Learning and AGI Safety. I dunno, maybe it's still the right idea. But at least I can explain why I personally grew less enthusiastic about it. One thing was, I came to believe (for reasons here, and of course I also have to cite this) that it doesn't buy the safety guarantees that I had initially thought, even with assumptions about the computational architecture. Another thing was, I'm increasingly feeling like people are not going to be satisfied with tool AI. We want our robots! We want our post-work utopia! Even if tool AI is ultimately the right choice for civilization, I don't think it would be possible to coordinate around that unless we had rock-solid proof that agent-y AGI is definitely going to cause a catastrophe. So either way, the right approach is to push towards safe agent-y AGI, and either develop it or prove that it's impossible. More importantly, I stopped thinking that self-supervised tool AI could be all that competent, like competent enough to help solve AI alignment or competent enough that people could plausibly coordinate around never building a more competent AGI. Why not? Because rewards play such a central role in human cognition. I think that every thought we think, we think it in part because it's expected to be higher reward than whatever thought we could have thunk instead. I think of the neocortex as having a bunch of little pieces of generative models that output predictions (see here), and a bit like the scientific method, these things grow in prominence by making correct predictions and get tossed out when they make incorrect predictions. And like the free market, they also grow in prominence by leading to reward, and shrink in prominenc

"A PP-based AGI would be devilishly difficult to align"

Is this an actual belief or a handwavy plot device? If it's the first, I'm curious about the arguments

My perspective as a native speaker who doesn't remember his grammar lessons very well:

The subjunctive mood has a lot of uses, at least in Spain (I'm not really familiar with other varieties of Spanish). Some examples off the top of my head:

1. Counterfactual conditionals: "Si Lee Harvey Oswald no hubiera disparado a JFK, alguien más lo habría hecho" (If Lee Harvey Oswald hadn't shot JFK, someone else would have), here "no hubiera disparado" is subjunctive and means "hadn't shot".

2. To speak about people'... (read more)

Your example is right, but it's not true that it's used in all subordinate clauses. For example, "Estoy buscando a la persona que escribió ese libro" (I'm looking for the person who wrote that book) does not have any verb in subjunctive mood.

I don't reflect on it. This happens in two ways:

  1. I find reflecting much more cognitively demanding than reading, so if there is a 'next post' button or similar, I tend to keep reading.

  2. Also, sometimes when I try to actually think about the subject, it's difficult to come up with original ideas. I often find myself explaining or convincing an imaginary person, instead of trying to see it with fresh eyes. This is something I noticed after reading the corresponding Sequence.

I guess establishing an habit of commenting would help me solve these problems.

Hello, I'm a math-cs undergrad and aspiring effective altruist, but I haven't chosen a cause yet. Since that decision is probably one of the most important ones, I should probably wait until I've become stronger.

To that end, I've read the Sequences (as well as HPMOR), and I would like to attend a CFAR workshop or similar at some point in the future. I think one of my problems is that I don't actually think that much about what I read. Do you have any advice on that?

Also, there are a couple of LWers in my college with whom I have met twice, and we would lik... (read more)

3Elo
try rewriting what you have read or teaching it to other people. This will help you feel like you understand it better and go back and re-learn what you might have missed. See also: Feynman notebook method.
0Elo
karma awarded.
2Richard Korzekwa
Do you mean that you don't put much thought into deciding what to read, or that when you read something you don't reflect on it?