All of Dirichlet-to-Neumann's Comments + Replies

The transformer architecture was basically developed as soon as we got the computational power to make it useful. If a thought assessor is required and we are aware of the problem, and we have literally billions in funding to make it happen, I don't expect this to be that hard. 

There are maps now that can completely lock up the use of some specific applications or websites. Digital Detox for example has all the basic functionalities you expect and is free.

You can use the Dirichlet-to-Neumann operator associated with an unknown elliptic operator to reconstruct the coefficient of the operator and consequently the structure of the inside of the domain. It's a problem proposed by Calderon and is well understood for the Laplacian. Good luck with the Stokes operator tomorrow though.

I've often heard say, among charities people who work with homeless people, that you need as long to get out of the street than you spent living in the street.

When world chess champion Anand won arguably his best and most creative game, with black, against Aronian, he said in an interview afterward "yeah it's no big deal the position was the same as in [slightly famous game from 100 years ago]".

Of course the similarity is only visible for genius chess players.

So maybe pattern matching and novel thinking are, in fact, the same thing.

On the politics part : one thing I like very much with the Roman republic system was the concept of the "cursus honorum". Basically if you wanted to go for a politician career you had to start at the bottom, get elected to a first position, do well, get elected to something more prestigious, etc. And it worked very well - a significant part of Roman success was that their government (and generals) were way better than competing powers, in this was mainly due to having a lot of experienced, competent politicians and generals with somewhat well aligned incentives.

That really depends of which part of France you are talking about. Provence uses mostly olive oil. In the South West they often use duck fat.

French are also apparently slightly less obese than their neighbours, the difference is not only with the US.

4ChristianKl
They seem to have similar average BMI and the Swiss seem to have an even lower obesity rate.  Belgium seems lower obesity rates than France but slightly higher average BMI. Andorra has lower obesity rates but a significantly higher average BMI. The UK, Spain and Germany are doing worse than France.  A bit of chatting with Gemini suggests what Belgium, France and the Swiss share is a strong market culture so food is more fresh.

That used to be the French model, which imo kick way above its (abysmal) low funding.

3TeaTieAndHat
Could you expand on that? I’m French, and even though I’m not involved in research I notice that: 1. French researchers are very underfunded 2. Somehow they’re still pretty good at building a lab with duct tape and string. But all the good metascience articles are written in English and focus on the US, so I have no clue how research funding in France has evolved over time, what’s been getting worse or better, etc., and I’d like to know more.
2Roman Leventov
That also was, naturally, the model in the Soviet Union, with orgs called "scientific research institutes". https://www.jstor.org/stable/284836

Steelmanning is about finding the truth, ITT is about convincing someone in a debate. Different aims.

There's a big gap between "you have to complete the task in exactly this way" and "mistake is a mistake, only the end result count".

I routinely gives full marks if the student made a small computation mistake but the reasoning is correct. My colleagues tend to be less lenient but follow the same principle. I always give full grade to correct reasoning even if it is not the method seen in class (but I quite insistently warn my students that they should not complain if they make mistakes using a different method).

I do exactly what you describe with my students, but sadly with extremely limited results.

"conveniently ignores the fact that the kids who didn't have a problem with the lecture were the ones who already knew all of that from some other source."

This is definitely not true in general and probably a rare case. N=1 of course, but I never had problems with maths lectures (or any other lectures) and I never was in the situation of knowing all of the maths before the lecture (I usually knew history and physics lessons in advance though). And it's the same thing with my current students : even the best ones are clearly unfamiliar with the material I c... (read more)

Upvoted because it's a good intro discussion to a problem that I am personally involved with (as a maths teacher). But my personal experience is that what makes a good maths curriculum is much more complicated than that. In particular I'm pretty certain now that different students have such wildly different needs that any attempt at (universal) standardisation is doomed to fail (of course some curriculum are still better than others...).

1DavidHolmes
I think this is a key point. Even the best possible curriculum, if it has to work for all students at the same rate, is not going to work well. What I really want (both for my past-self as a student, and my present self as a teacher of university mathematics) is to be able to tailor the learning rate to individual students and individual topics (for student me, this would have meant 'go very fast for geometry and rather slowly for combinatorics'). And while we're at it, can we also customise the learning styles (some students like to read, some like to sit in class, some to work in groups, etc)? This is technologically more feasible than it was a decade ago, but seems far from common.

Speaking as a Catholic, this won't have much impact but mostly because the Catholic Church as a whole is already extremely wary about AI. It's good that it is explicitly written at the highest level though (note that what you feel is vague is just Vatican-speak).

However there is still no understanding about how powerful new AI models will be. In particular Catholics in general are skeptical about the possibility of AGI (mainly for philosophical/theological reasons). Their concerns will side more with AI-ethic rather than AI-alignment, but they will be natural allies for any "pause" or "slow-down" movement.

The pope has advisors. Some may even be young !

The Catholic Church has a long intellectual tradition even if it's very different from the one on lesswrong, and it has always been wary of potential misuses of new technologies. So nothing really surprising here for those who are used to Vatican-speak.

I've remarked that I've recently begin to strong upvote more and I think it's a bad habit. How often would you say you upvote vs strong upvote?

2MondSemmel
Speaking just for myself, and not suggesting this as a norm: Checking my vote history, stats based on my last 6 weeks of voting: I ~never strong-upvote my own comments (whereas one's own posts are strong-upvoted by default). I very rarely strong-vote comments (on the order of <1-2%) one way or another. I rarely agree-vote one way or another. I occasionally (5-25%) strong-upvote posts, with meanings like "more people should see this" or "I want to see more stuff like this". Especially posts like LW feature announcements, practical high-effort posts, etc. I strong-downvoted one single post in this timeframe, on a hot-button politics issue. One confounder here is that when I browse the LW feed, I tend to bookmark posts and then only read them months later. So by the time I read such a post, all the comments already have plents of votes, and the karma order usually looks more or less fine to me, so I see little point in bothering with using strong-upvotes. Similarly, by that time truly unpalatable low-value comments are usually already at negative karma, so again, no reason for me to pile on. One situation where I do "make use of my voting power" is when voting on norms in the LW/EA communities. E.g. I often strong-vote and (dis)agree-vote during community controversies. Fortunately there either weren't any in the past six weeks (so this behavior didn't appear in the stats above), or there were and I didn't see them.

In retrospect Alpha0 was really the wake up call for me, not because it was so strong at chess but because it looked so human playing chess.

What you are looking for are "studies", which are generally in the form of "white to play and win" or "white to play and draw", and requires to find a sequence of moves forcing a winning (or drawn) position, but without going to the mate.

Most are devilishly hard though. I expect 2000 elo players to have a hard time with your average study (but in your experimental set-up the advisors could just look up the solution).

Works with most crimes tbh.

That, but getting your army from mostly melee to mostly range and solving your operational problems helps a lot too.

I mean that AGI and autonomous cars are orthogonal problems, especially because autonomous car requires solving engineering issues (which have been discussed by other commentators) which are different from the software issues. It's quite usual here on less wrong to handwave the engineering away once the theoretical problem is solved.

Some animals species are able to adopt contraception-like practices too. For example birds of preys typically let some of their offsprings die of hunger when preys are space.

Note, though, that agrarian Eurasians empires ended up winning their 1000 thousand years struggle against the steppe peoples.

2Roko
Yes, and I believe that the invention and spread of firearms was key to this as they reduce the skill dependence of warfare, reducing the advantage that a dedicated warband has over a sedentary population.

No, because the events which led to the unification of the Mongol tribes by Gengis Khan were highly contingent.

However, the military power overhang of the steppe peoples vs agrarian states should have been obvious for anyone since both the Huns and the Turks did the same thing centuries before.

As usual you fall onto the trap of neglecting the engineering and social organisation problems and the time required to solve them. We don't need AGI for autonomous car, it will just take time.

1[deactivated]
What do you mean?

High levels GPUs are needed for basically anything mundane today. No need to bring in AGI worries to make it a strategic ressource.

1O O
I think the timing and perf focus of it all makes it clear it’s related to foundation models.

Yet human routinely sacrifice their own lives for the good of other (see : firefighters, soldiers, high mountain emergency rescuers, etc.). The X-risk argument is more abstract but basically the same.

5O O
A lot of our moral frameworks breakdown once immortality is a real choice. Sacrificing your own life for the own good can be reframed as dying a little earlier. Many of these people go in knowing there’s a small chance of death. A lot of them would probably change their minds if it was a suicide mission (except a minority). If the 5 year mortality rate of firefighting was 100%, how many would still do that job?

Less wrong has imo a consistent bias toward thinking only ideas/theory are important and that the dirty (and lengthy) work of actual engineering will just sort itself out.

For a community that prides itself on empirical evidence it's rather ironic.

Answer by Dirichlet-to-Neumann20

I'd be happy to play any of the A, B and C roles.

I'm a around 1850 elo FIDE, about 2000-2100 on lichess. I play a couple of blitz games daily.

I'd be willing to play at almost any cadence and have a lot of free time. I actually live in France, so a one-move-per-day game with someone living in the US would probably be ideal. Live sessions can be programmed from 16 GMT to 23 GMT on weekdays, and from 7 GMT to 23 GMT on weekends.

As I said I would be happy to play any role. I think it would be more interesting if the lower player is actually not a total beginne... (read more)

Well the Good Samaritan parable is the most well known, most important and most striking parable in the Gospels on the very specific topic of who you should help and how you should help. It's not a wonder it's a recurring name for Christian inspired charities.

DeepL is generally better than Google translate anyway.

Most western polytheistic religion (Roman, Greek...). Judaism*. Islam*. Buddhism. In fact Christianism with its overemphasized focus on dogmas is somewhat an exception.

  • I'm not saying those religions don't include beliefs but that they are not defined by those beliefs.
3Jim Pivarski
I think it would be more correct to say that a focus on believing particular assertions is a fairly recent trend in religion, encompassing the past millennium or two, but really picking up in the last few centuries. It happened in or between Christianity and Islam (as isusr points out), and they probably both influenced each other. For example, Protestant Christianity focuses a lot more on a holy book than Catholicism and Orthodox Christianity, but in a way that resembles Islam's veneration of the Quran: citing verses to prove points. Since then, Catholics and Orthodox have also stepped up their focus on the Bible. There's a lot of cross-pollination. In the last century or so, religious statements have been presented as a kind of alternate-science (e.g. Young Earth science), presumably to respond to an apparent threat, but this is a very new way of taking about religion. There were biblical literalists (and non-literalists) throughout Christian history, but ancient theologians would probably accuse these people of missing the point. Meanwhile, religions with only recent sustained contract with Christianity and Islam (past half-millennium) and religions that preceded them focus a lot more on practice, i.e. ritual and social behaviors. Some belief is implicit (e.g. why leave offerings for gods or ancestors if you don't think they exist in a form that would benefit from the offerings?), but they are much less the focus.
lsusr104

The first pillar of Islam is an assertion of faith. Every Islamic teacher and academic I've listened to talks as if belief is just as important to Islam as it is to Christianity. Technically subordination is more important, but it's pretty hard to have subordination without belief. Where did you get the idea that Islam doesn't care what you believe? Are you referring to stuff like formally identifying as a Muslim in an Islamic theocracy to get a reduced tax burden?

But otherwise, yeah, you're correct. Roman and Greek religion definitely count. So does Norse... (read more)

I would find the shiny look rather annoying, wouldn't a matte finish be better ? 

5jefftk
They will stop being shiny with a bit of use; the one I made a year ago isn't very shiny anymore. The shinier finishes are generally more durable, so I didn't want to start with a matte one.

I kind of feel like the moment is not ideal for a huge development project in collaboration between Finland, Russia, the US and Canada.

1AnthonyRepetto
Collaboration might yet work, if Russia lets Sweden back in charge. oof! :0

Genuine question : how much your opinion on college and higher education are due to the American system being insane ?

Because for example in France university/college is mostly free. Nobody get into debt to pay for tuition. How much would it change your opinion?

2Brendan Long
Presumably French universities still cost 4+ years of your life, which is the majority of the cost of US universities as well. The financial cost of sane schools in the US is fairly low.
3Zvi
Being free to the student (although of course the French overall still pay taxes to fund the real costs) makes it less toxic, but it also means you have that much less excuse if  you don't go. So my guess is this makes it maybe 25% less bad?

Concerning MAID, if past trend are to be believed, the most terrifying thing is that these numbers will only get worse.

8Lukas_Gloor
Is it terrifying because you think it's bad if some people kill themselves with the program who otherwise wouldn't have? Or is it terrifying that things are so bad in general that, once you give them more options, more will choose death? If you mean the former – I really don't share this judgment. Maybe you're worried that sociopathic or act-utilitarian doctors will pressure patients – but these risks could be mitigated with safeguards, and they have to be weighed against the benefits (people being able to make informed choices about what they want and have their options increased).  If it's the latter, then I agree. But I'm not surprised by this information, sadly. Edit:  And the same question to Zvi:  What does this mean? What's the implication for policies you would push for? Should other countries try to install something similar or would that be terrible?  I guess it's reasonable to be like "I don't know, seems like a tough call and terrifying either way." But there's a risk that, if one feels an impulse to shy away from contemplating some topic in depth because it seems "terrifying," it leads to biased opinions. 
Answer by Dirichlet-to-Neumann2-19

To be honest I'm just as afraid of aligned AGI as of unaligned AGI. An AGI aligned with the values of the PRC seems like a nightmare. If it's aligned with the US army it's only really bad, and Yudkowsky dath illan is not exactly the world I want to live in either...

4Grant Demaree
I don't agree, because a world of misaligned AI is known to be really bad. Whereas a world of AI successfully aligned by some opposing faction probably has a lot in common with your own values Extreme case: ISIS successfully builds the first aligned AI and locks in its values. This is bad, but it's way better than misaligned AI. ISIS want to turn the world into an idealized 7th Century Middle East, which is a pretty nice place compared to much of human history. There's still a lot in common with your own values

1 000 mg is the standard dose in France, with 500mg being used almost only for children.

Sadly both my time and capacity are limited to "try some prompts around to get a feeling of what the results look like." I may do more if the results are actually interesting.

One of the first tasks I tested was actually to write essays in English with a prompt in French, which it did very well, I would say better than when asked for an essay in French. I've not looked at the inverse task though (prompt in English for essay in French).

I'll probably translate the prompts through DeepL with a bit of supervision and analyse the results using a thoroughly scientific "my gut feeling" with maybe some added "my mother's expertise".

I'd like to make a quite systematic comparison of openAi's chatbot performances in French and English. After a couple days trying things I feel like it is much weaker in French - which seems logical as it has much less data in French. I would like to explore that theory, so if you have interesting prompts you would like me to test let me know !

1Jay Bailey
How systematic are we talking here? At research-paper level, BIG-Bench (https://arxiv.org/pdf/2206.04615.pdf) (https://github.com/google/BIG-bench) is a good metric, but even testing one of those benchmarks, let alone a good subset of them (Like BIG-Bench Hard) would require a lot of dataset translation, and would also require chain-of-thought prompting to do well. (Admittedly, I would also be curious to see how well the model does when self-translating instructions from English to French or vice-versa, then following instructions. Could GPT actually do better if it translates French to English and then does the prompt, vs. just doing it in French?) Even if you're just playing around though, BIG-Bench should give you a lot of ideas.
Answer by Dirichlet-to-Neumann12

Training teachers is probably the main physical cost (it was a big problem for computer science in France), but the main social obstacle is the opposition to change from basically everyone : parents don't want their children to learn different things than they did, teachers don't want to lose curriculum hours to make room for new subjects, and administrators don't want to risk making anything new.

Answer by Dirichlet-to-Neumann85

Epistemic status : n=1.
I very much enjoyed my school years. I learned a lot on subject that turned out to be actually useful for me like maths and English, and on subject that were enjoyable to me (basically everything else). I would definitely have learned much less without the light coercion of the school system, and would have been overall less happy (In later years at college level where I was very much my own master I learned less and was less happy ; in my three years of "classe prépa", the most intensive years of my studies I learned the most and wa... (read more)

I tried to make it play chess by asking for specific moves in opening theory. I chose a fairly rare line I'm particularly fond off (which in hindsight was a bad choice, I should have sticked with the Najdorf). It could identify the line but not give any theoretical move and reverted to non-sense almost right away.

Interestingly it could not give heuristic commentary either ("what are the typical plans for black in the Bronstein-Larsen variation of the Caro-Kann defense").

But I got it easily to play a game by... just asking"let's play a chess game". It could... (read more)

My reaction has nothing to do with "allowing AI to deceive" and everything with "this is a striking example of AI reaching better than average human level at a game that integrates many different core capacities of general intelligences such has natural language, cooperation, bargaining, planning, etc.

Or too put it an other way : for the profane it is easy to think of GPT-3 or deepL or Dall-e as tools, but Cicero will feels more agentic to them.

While I don't think anyone aware of AI alignment issues should really update a lot because of Cicero, I've found this particular piece of news to be quite effective at making unaware people update toward "AI is scary".

Quinn3411

For me the scary part was Meta's willingness to do things that are minimally/arguably torment-nexusy and then put it in PR language like "cooperation" and actually with a straight face sweep the deceptive capability under the rug.

This is different from believing that the deceptive capability in question is on it's own dangerous or surprising.

My update from cicero is almost entirely on the social reality level: I now more strongly than before believe that in the social reality, rationalization for torment-nexus-ing will be extremely viable and accessible to... (read more)

Musk proposition gives Ukraine no significant security guarantees AND forces it to lose territory. It's basically a total win for Russia, and an excellent incentive to try again in 10 years (or maybe vs. the Baltic states or Georgia).

-2ChristianKl
The Baltic states don't have areas where Russia would gain anything from them having a referendum to join Russia because nobody would vote "Yes". The Baltic states are protected by NATO. Even for those who considered the argument about the Baltic states reasonable before the latest invasion the performance of the Russian army should clearly suggest that any violation of NATO borders is a bad idea for Russia.  It's difficult to hold territory against the wishes of the local population and produces all sorts of internal problems. The Wikileaks cable is really interesting in that it regards the Russian foreign policy crowd as wanting to avoid having to act within Ukraine.  The present Georgian situation seems to be fine for Russia without a need to do something about it. From one article: As far as Ukraine goes, Ukraine can improve its military as well over the next ten years. Clearing the reasons why the Russian public wanted this war is a way to reduce incentives for another invasion as well. 

I downvoted because it's not a particularly interesting critique of religion (contrary to, say, Eliezer's https://www.lesswrong.com/posts/fAuWLS7RKWD2npBFR/religion-s-claim-to-be-non-disprovable which is really solid. The paragraph on free will is weak because it fails to engage with what the other side is saying (yet alone refute a steel man version)

Besides, making a big post on lesswrong about how religion is silly is just preaching to the choir - in the same way that if you really want to make a post on why AI is dangerous, don't do it if you don't have something new to bring to the debate.

It's certainly interesting although to be honest I'm pretty confident the top human stratego players are nowhere near the top achievable level for a human player (contrasting with games like chess or StarCraft).

Load More