All of Bernhard's Comments + Replies

Well please do derive it then, because to me it seems you just focused on one aspect and then concluded that that aspect definitely is the correct answer. 

If the goal was to reward the best and the brightest, then why does china make some of them disappear from time to time? Why reeducate the odd billionaire who misbehaves? The idea was to get him in power because he knows better and generates riches, no?

On giving away stuff for 'free': what would be good examples in your opinion? Steel? Silicon or finished solar cells? Electric cars and batteries? Ma... (read more)

They cash in politically. 

Imagine some middle eastern oil and money rich country going abroad to help develop a subsaharan economy to help them extract their resources. Imagine china doing the same. Who will have more success?

2Yair Halberstadt
I agree they do, I just doubt that's more than a tiny percentage of the reason Chinese individuals are far better off than they were 50 years ago, and I think they fact they manufacture things cheaply is large percentage of why they're better off.

I disagree with Eliezer when he says that countries with surpluses never cash them in, which is kind of part of the core of his argument.

(Equity) Billionaires live lavish lifestyles, even though they never sell their equity. Doing so would give up their power. Instead they borrow for cheap (c.f. " buy, borrow, die"). I wonder which countries typically can borrow money very cheap. Maybe Japan, or Germany...? 

 

He just looks at a single number, and then utters "these numbers don't make sense". Well of course.

Look at the whole picture instead. 

T... (read more)

7Daniel V
Literally macroeconomics 101. Trade surpluses aren't shipping goods for free. There is a whole balance of payments to consider. I'm shocked EY could get that so wrong, surprised that lsusr is so ready to agree, and confused because surely I missed something huge here, right?
7Yair Halberstadt
The reason Chinese individuals have increasing quality of life, seems to be far more because they manufacture things cheaply than any sort of cashing in they do.

Google is skilled at marketing. The only other new information is that they were able to prove that error correction capability scales faster than errors when adding 'physical' qubits. They use about hundred 'physical'qubits to emulate a single (error corrected) 'logical' qubit.

So once they are able to scale up from N=1 to something like 256, they will be able to do what you say.

Importantly, before that happens, let's say for N=100, they will be able to simulate the behavior of chemical systems composed of 100 atoms. This is a huge thing, and your will hea... (read more)

but I see no reason to think they'll just utterly fail at scaling production at those factories

Oh they'll scale just fine.

It's just that nobody will buy all those cars. They are already not selling them all, and we are about to enter the biggest recession of many of our lifetimes

8ESRogs
Why would this be true? Teslas are generally the most popular car in whatever segment they're in. And their automotive gross margins are at 25+%, so they've got room to cut prices if demand lightens a bit. Add to this that a big tax credit is about to hit for EVs in the US and it's hard for me to see why demand would all-of-a-sudden fall off a cliff.
1Noosphere89
I do think we will be in a mild recession unless the Fed does a soft landing, but the economy is actually okay. So this recession will be much milder than previous recessions.

Well, for one did you ever notice how people act differently in different situations? (for example among family, friends, work, acquaintances at the gym, or online) If you limit yourself to a single situation, there is not any person on earth that you could 'reconstruct' sufficiently well.

2mako yass
A minor example... I'm fairly sure you can make guesses about what kinds of expressions a person makes a lot from a few photos of their face. I'm not sure what else to point at to convey this intuition, but I seem to believe that behaviors in very different contexts leak information that'll all become apparent with enough data. I guess I can believe that there are probably a lot of people who don't output enough content for this to work, maybe even among the users of this forum, but I don't think it's a large proportion of them.

The 50% annual revenue growth that they've averaged over the last 9 years shows no signs of stopping

What makes you think that?

I am of the completely opposite opinion, and would be amazed if they are able to repeat that even for a single year longer.

All the "creative" bookkeeping only work for so long, and right now seems to be the moment to pop bubbles, no?

2ESRogs
If we just look at the next year, they have two new factories (in Berlin and Austin) that have barely started producing cars. All they have to do to have another 50-ish% growth year is to scale up production at those two factories. There may be some bumps along the way, but I see no reason to think they'll just utterly fail at scaling production at those factories. Scaling in future years will eventually require new factories, but my understanding is that they're actively looking for new locations. Their stated goal is to produce 20 million cars in 2030. I think that's ambitious, but plausible. And I wouldn't be too worried about my investment if they're only at 10 million in 2030, or if it takes them until 2033 to reach 20M.

A sufficiently detailed record of a person's behavior

What you have in mind is "A sufficiently detailed record of a person's behavior when interacting with the computer/phone"

How is that sufficient to any reasonable degree?

2mako yass
What sorts of things, that you would want preserved, or that the future would find interesting, would not be captured by that?

Why?

Because of perverse, counterproductive and wrong monetary incentives.

There are a few complexities:

There is only really one, and that is not accounted for.

You want to generate electricity that you actually use.

I'm no expert on your part of the world, but in Central Europe electricity prices sometimes turn negative, because eletricity is generated that nobody needs. So large producers have to pay money, to get rid of it. Taking a pickaxe to your solar panel would be net positive in that situation.

Why? because everybody maximizes electricity produced and ... (read more)

3Yair Halberstadt
In Israel it's the opposite. A lot of electricity is needed at noon, for air conditioning, and much less in the mornings/evenings when it's cooler.

Are you familiar with ergodicity economics?

https://twitter.com/ole_b_peters/status/1591447953381756935?cxt=HHwWjsC8vere-pUsAAAA

I recommend Ole Peters' papers on the topic. That way you won't have to construct your epicycles upon the epicicles commonly know as utility calculus.

 

We are taught to always maximize the arithmetic mean

By whom?

The probable answer is: By economists.

Quite simply: they are wrong. Why?

 

That's what ergodicity economics tries to explain.

In brief, economics typically wrongly assumes that the average over time can be substituted... (read more)

To me this is a good example of a too theoretic discussion, and as the saying goes: In theory, there is no difference between theory and practice. (But in practice there is).

My counterargument is a different one, and I kind of already have to interrupt you right at the start:

If there is no death, [,,,]

Putting "immortal animals" into any search engine gives lots of examples of things that get pretty close. So we can talk about reality, no need to talk only about Gedankenexperimente. So the first question cannot be: "Why is the counterargument wrong"?

Instead... (read more)

The most powerful one is probably The Financial System. Composed of stock exchanges, market makers, large and small investors, (federal reserve) banks, etc...

I mean that in the sense that an anthill might be considered intelligent, while a single ant will not.

Most of the trading is done algorithmically, and the parts that are not might as well be random noise for the most part. The effects of the financial system on the world at large are mostly unpredictable and often very bad.

The financial system is like "hope" according to one interpretation of the myth... (read more)

Very good idea
I did not do it. My argument would be that the impetus is not my own, it is external, your written word.

What stops you from making increasingly outlandish claims ("Your passphrase is actually this (e.g illegal/dangerous/lethal) action, not a simple thought" Where to draw the line?

Just as a point of reference, as a kid I regularly thought thoughts of the kind: "I know you're secretly spying on my thoughts but I don't care lalalala....." I never really specified who "you" was, I just did it so I could catch "them" unawares, and thereby "win". J... (read more)

Just for completeness, I found [this paper](http://dx.doi.org/10.1016/j.neuron.2021.07.002), where they try to simulate the output of a specific type of neuron, and for best results require a DNN of 5-8 layers (with widths of ~128)

We Live in a Post-Scarcity Society

Do you mean "we Americans"? or "we, the people living on the East West Coast"? Because it certainly is not true on a national/worldwide level.

For example, in a "magical post-scarcity society" , you would probably be okay to be (born as) really anybody. You shouldn't really care as much as you might during medieval times for example.

How about right now? Do you care? Are you willing to trade places? I certainly am not.

Furthermore, you picked the worst possible timing for this post. One should not characterize a society based... (read more)

I guess that would be one way to frame it. I think a simpler way to think of it (Or a way that my simpler mind thinks of it) is that for a given number of parameters (neurons), more complex wiring allows for more complex results. The "state-space" is larger if you will.

3+2, 3x2 and 3² are simply not the same.

From my limited knowledge (undergraduate-level CS knowledge), I seem to remember, that typical deep neural networks use a rather small number of hidden layers (maybe 10? certainly less than 100??  (please correct me if I am wrong)). I think this c... (read more)

First of all, kudos to you for making this public prediction.

To keep this brief: 1 (95%), 2 (60%), 3 (75%), 4(<<5%), 5 (<<1%)

I don't think we are in a hardware overhang, and my argument is the following:

Our brains are composed of ~10^11 neurons, and our computers of just as many transistors, so in a first approximation, we should already be there.

However, our brains have approximately 10^3 to 10^5 synapses per cell, while transistors are much more limited (I would guess maybe 10 on average?).

Even assuming that 1 transistor is "worth" one neuron... (read more)

3Bernhard
Just for completeness, I found [this paper](http://dx.doi.org/10.1016/j.neuron.2021.07.002), where they try to simulate the output of a specific type of neuron, and for best results require a DNN of 5-8 layers (with widths of ~128)
3lsusr
Do you mean that each additional layer contributed too much to hypothesis space entropy?

I have a similar objection. Not particularly because they are Refugees, but instead because they are Foreigners.

I actually quite liked the presented idea, but I think it is very heavily slanted towards ideas that are oriented on the political left (which I generally favor if I were forced to choose). Still, some concepts from the political right are important here, particularly those of culture, and personal responsibility.

Summarized very briefly (and therefore certainly wrong)

Culture, according to the left is something that is imposed from above, and can ... (read more)

"Probably most ambitious people are starved for the sort of encouragement they'd get from ambitious peers"

If you were to substitute "intelligent" for "ambitious", I would agree. Some kind of dialog is needed to flourish, and a dialog between equals is strongly preferred. Or said another way, when training, it makes no sense to train with to little weight.

 

The smartest people tend to be ambitious.

I strongly disagree. Assuming a certain bias regarding the selection of examples, this is just a tautology: Highly visible people are highly visible. Successf... (read more)

we would task AGI with optimizing

 

I see, that kind of makes sense. I still don't like it though, if that is the only process to optimize.

For me, in your fictional world, humans are to AI what in our world pets are to humans. I understand that it could come about, but I would not call it a "Utopia".

this was assuming a point in the future when we don't have to worry about existential risk

This is kind of what I meant before. Of course you can assume that, but it is such a powerful assumption, that you can use it to derive nearly anything at all. (Just li... (read more)

1Jozdien
If I understand your meaning of this correctly, I think you're anthropomorphizing AI too much.  In the scenario where AI is well aligned to our values (other scenarios probably not having much of a future to speak of), their role might be something without a good parallel to society today; maybe an active benevolent deity without innate desires. I think it's possible we would still die at the end of the universe.  But even without AI, I think there would be a future point where we can be reasonably certain of our control over our environment to the extent that barring probably unprovable problems like simulation theory, we can rest easy until then.

I read the original post, and kind of liked it, but I also very much disagreed with it.

I am somewhat befuddled by the chain of reasoning in that post, as well as that of the community in general.

In mathematics, you may start from some assumptions, and derive lots of things, and if ever you come upon some inconsistencies, you normally conclude that one of your assumptions is wrong (if your derivation is okay).

Anyway, here it seems to me, that you make assumptions, derive something ludicrous, and then tap yourself on the shoulder and conclude, that obviously... (read more)

1Jozdien
Thinking it back after a couple days, I think my reply with finding maximums was still caught up in indirect measures of achieving hedons.  We have complete control over our sensory inputs, we can give ourselves exactly whatever upper bound there exists.  Less "semi-random walks in n-space to find extrema" and more "redefine the space so where you're standing goes as high as your program allows". For what it's worth, that was just to keep in with the fictional scenario I was describing.  In a more realistic scenario of that playing out, we would task AGI with optimizing; we're just relatively standing around anyway. In that scenario, though: why do we consider growth important?  You talked about surviving, I'm not clear on that - this was assuming a point in the future when we don't have to worry about existential risk (or they're the kind we provably can't solve, like the universe ending) or death of sentient lives.  Yes, growth allows for more sophisticated methods of value attainment, but I also said that it's plausible that we reach so high that there we start getting diminishing returns.  Then, are the benefits of that future potential worth not reaping them to their maximum for a longer stretch of time?

I pretty much agree with you. Human intelligence may be high because it is used to predict/interpret the behaviour of others. Consciousness may be that same intelligence turned inward. But:

3. Given enough computational power and a compatible architecture, the agent will develop consciousness if and only if it needs to interact with other agents of the same kind, or at least of similar level of intelligence.

This does not automatically follow I think. There may be other ways that can lead to the same result.

An existing example would be cephalopods (octopus, ... (read more)

each UV photon that hits exactly in the right spot will cause permanent DNA changes that eventually lead to cancer

Pretty sure this is incorrect. It's not the damage that causes cancer, but the failure of the body to heal/repair it. Such failures can be caused for example by you being very old, and therefore healing slower, or by getting a sunburn (= too much exposure in a short time, overwhelming repair capability).

I think the most important thing here is that things scale very much not linearly.

See also this, which argues/claims that more sun exposure (wi... (read more)

Lots of interesting answers, and all of them correct (most of the time anyway). One I haven't seen mentioned, is the one described in this preprint titled "How to Increase Global Wealth Inequality for Fun and Profit".

In short:

  • If you buy x shares of something, the price goes up slightly (This is strictly true for X-> infinity)
  • If you sell, the price drops
  • If you do both in quick succession (a full circle), the price should not change (Otherwise you invented the economic version of a perpetuum mobile)
  • Bid-Ask spread exists. Sellers want to sell for as high a
... (read more)

Why Mars for your thought experiment?

I tried to think of a case where a "new" state is founded. On earth, every (dry) place is already owned, so that wouldn't have worked (without seceding). Seasteads might work nearly as well as space colonies. Assuming infinite monetary resources, in both cases it is conceivable to set up something completely disconnected from existing power structures.

In a way it's just a toy example, that disregards wars and other military actions. 

I really like this, and agree that incentives are very important.

A related idea for example is to limit the pay of CEOs to X times that of the average/lowest salary of employees.

A trivial (as in too simple to actually work) toy example in the same vein is to continuously make the least powerful person the most powerful person.

That person than either helps themselves (consequently loosing office), or helps everybody at the bottom. Either way, inequality decreases (Yes, I know this won't work).

Hard cases make bad law is a general legal maxim.

I agree.

I don't want to constantly create new laws, but instead constantly shine light on things that go wrong (This may or may not happen already, depending on which news you consume. Sadly however, mostly nothing comes of it, since change is hard). From there, if patterns emerge, then new laws should be proposed (For example, one pattern could be "unemployment in sector XY increases due to automation". For how many sectors should this have happened, before general action is warranted?)

You can't reliably co

... (read more)
2ChristianKl
The system you propose above is intended to fix things even when nothing went wrong (Trump didn't pardon himself). Representative democracy allows accountability for the effects of laws that politicians pass. Evaluating politicians by their past actions instead of their promises for the future works much better.  The idea that voters who don't get a good understanding of how the taxes they pay change from looking at their tax returns will get a good idea of that by them looking at unit tests on some government website sounds illusory to me. The whole point of unit tests is that it's a way not to try certain changes.  You are essentially preventing that by getting people to focus on unit tests that can be made based on the text of the law. If you evaluate laws based on an analysis of the law instead of an analysis of the effects on the law in empirical reality you reduce the amount that people learn from empirical reality.  Do you seriously think that politicians passing new laws for drug legalization don't look at the existing laws to guide them? The Brexit negotiations were dominated by power conflicts and as such the deal isn't good in a lot of respects. Both sides defects in the prisoner dilemma.  I can completely understand bureaucrat from the UK to say to an EU bureaucrat: "I think the way the EU regulated IT security in 2008 was fine, let's just copy it over to our deal." Then the EU bureaucrat says "Fine, lets move on to one of the issue where we disagree" Trade deals are not normal laws and the process that leads to them shouldn't be seen as normal laws. The process by which they are made is very different.  Gell-Mann Amnesia is about forming your opinions by reading the news. For me that's not where the foundation of my thinking about how law gets made comes from. My foundation rather comes from informal conversations which people who are involved in the process. What kind of case do you have for not suffering from Gell-Mann Amnesia?

automating it with git would not in any way change the fundamental power structures at play

Well, currently, a lobbyist provides the desired changes, and your politician implements them. I am proposing to make it possible for everybody to propose changes easily.

proposing that anyone can change the law would clearly be insane

I agree. I was thinking more of something like wikipedia vs a classic encyclopedia. Many people determine what actually makes up wikipedia's content, but far fewer are in charge and oversee the final approval. As an example, in the past ... (read more)

2ChristianKl
It's easy to propose changes in a representative democracy. That's what letters to politicians are for. They are not a tool that's limited to lobbyists.  There are also other tools such as petitions or responding to requests for comments to engage with the process. Wikipedia has no process that's well described by the words "final approval". That's misrepresents Wikipedia policy for most articles. "Factually correct" is not a category that Wikipedia cares about. It cares about whether the article is referenced with reliable sources and whether it's notable.  Even if this would be how Wikipedia works, it's benefitial towards producing a quanity of articles. When it comes to laws you don't want quantity. You want as little laws as possible because with increased quantity of laws it gets more complex.
2Ericf
A lobbyist is just a programmer for laws. "Everyone" already has the power to suggest changes, but only some people have taken the time to learn how to "code." The main issue is still "who gets the power?" And Wikipedia suffers from the same coordination problem that plagues governments: if a small group values X highly, and 90% of everyone values "not X" just a little bit (or, more often,, would value "not X" if they were made aware of X), X gets implemented (because the people in charge don't know about the second group).