All of Raphael Roche's Comments + Replies

I agree that it is disappointing but not completely surprising to see no economic effect from access to knowledge via the democratization of LLMs.

Previously, there was the encyclopedia of Diderot and d'Alembert, there was the Encyclopedia Britannica, there was Wikipedia, but those who consulted these sources were often people who already had a novice or established intellectual profile. I doubt that the typical entrepreneur, salesperson, craftsman, or farmer spent much time on average reading these works. It could be the same for the educational use of LLM... (read more)

If we consider that some people have already fallen in love with their AI chatbot or have made it their best friend, this type of phenomenon is likely to amplify if the agents become even more human-like. It is reasonable to wonder if, instead of raising awareness among the general public about the risks of AI, this could have the opposite effect. Love is blind, as they say. 

However I think that, good or bad, LLM-based AI will become more and more human-like in surface. The data training set is human (or human-like if synthetic), thus, because of RL p... (read more)

The user can edit it or clear it (even disable it), but it is primarily edited by the AI itself.

As mentionned by the author, OpenAI chose to equip its chat models, at least since 4o, with an editable contextual memory file that serves as a small episodic memory. I was impressed to observe the model's evolution as this file filled up. The model seemed increasingly intelligent and more 'human,' likely because this memory added a layer of user-specific fine-tuning tailored to me. It understood me better, responded more accurately to my expectations, appeared to share my values and interests, and could make implicit or explicit references to previous con... (read more)

You're right. I said "pronunciation," but the problem is more exactly about the translation between graphemes and phonemes.

You're right. The idea behind Académie française style guidelines is that language is not only about factual communication, but also an art, literature. Efficiency is one thing, aesthetics another. For instance, poetry conveys meaning or at least feeling, but in a strange way compared to prose. Poetry would not be very effective to describe an experimental protocol in physics, but it is usually more beautiful to read than the methodology section of a scientific publication. I also enjoy the 'hypotaxic' excerpt above much more than the 'parataxic' one. Rich... (read more)

This is interesting. I think English concentrates its weirdness in pronunciation, which is very irregular. Although adult native speakers don't realize it, this presents a serious learning difficulty for non-native speakers and young English-speaking children. Studies show that English-speaking students need more years of learning to master their language (at least for reading) than French students do, who themselves need more years than young Italian, Spanish or Finnish students (Stanislas Dehaene, Reading in the brain).

Redundancy makes sure the information passes through. In French, the word 'aujourd'hui' ('today') etymologically means 'au jour de ce jour' ('on the day of this day'), but it is not uncommon to say 'au jour d'aujourd'hui' which would literally mean 'on the day of on the day of this day'. It is also common to say 'moi, je' ('me, I') and increasingly people even say 'moi, personnellement, je' ('me, personally, I'). This represents a kind of emphasis but also a kind of fashion, simular to what happens in the fashion industry, or a kind of drift, similar to what happens in the evolution of species.

AI is very useful in legal matters and is clearly a promising sector for business. It is possible that some legal jobs (especially documentation and basic, non-personalized legal information jobs) are already being challenged by AI and are on the verge of being eliminated, with others to follow sooner or later. My comment was simply reacting to the idea that many white-collar jobs will be on the front line of this destruction. The job of a lawyer is often cited, and I think it's a rather poor example for the reasons I mentioned. Many white-collar jobs combine technical and social skills that can be quite challenging for AI.

Because of this, I think that there will be an interim period where a significant portion of white collar work is automated by AI, with many physical world jobs being largely unaffected.

I have read numerous papers suggesting that white-collar jobs, such as those of lawyers, will be easily replaced by AI, before more concrete or physical jobs as discussed by the author's. However, I observe that even the most advanced models struggle with reliability in legal contexts, particularly outside of standardized multiple-choice questions and U.S. law, for which th... (read more)

Assuming no technology is absolutely perfect, absolutely risk free, what if the the nuclear warhead detonate accidentally ? Wouldn't be less risky that, for instance, a russian nuclear warhead accidentally detonate in a russian military base in Siberia rather than in the russian consulate in the center of NYC ? 

Impressive prospective work. It's frightening, both scenarios, even though one is worse than the other. The evolution seems unstoppable, and even if superintelligent AGI doesn't happen in 2027-2030 but in 2040 or 2050, the feeling isn't very different. I have young children, and while I don't really care for myself, I really care for them. It was cool when it was just sci-fi. It was still fun when we first played with ChatGPT. It doesn't look fun anymore, at all. My own thinking about it is that we're indeed locked in a two-option scenario, probably not th... (read more)

I think what gives you this idea in the double slit experiment is that depending on how you observe or measure the object, it seems to exhibit different behavior. How is this possible? Isn't it mysterious? To resolve this mystery, you appeal to an explanation like awareness. But although it feels like an explanation, it actually explains nothing. Putting a word on something is reassuring, but behind the word we're not sure what we're talking about - we don't know how it's supposed to function; there is no actual explanation. It purports to explain everythi... (read more)

Brilliant essay. It reminds me of the work of James C. Scott. However, I am quite surprised by the conclusion: "I do not understand the Poverty Equilibrium. So I expect that a Universal Basic Income would fail to eliminate poverty, for reasons I don't fully understand." To me, the explanation of the Poverty Equilibrium is quite simple. Yes, there are diminishing returns in the marginal value of all resources, but there is also an increase in the subjective value of all resources in consideration of what you know others possess. Alice is happy with one bana... (read more)

Thanks for this precision, That's interesting.

Thanks for the study. In my opinion, there is a more direct evidence of how gay is the Vatican, or the Catholic church in general. In the general population, victims of sexual assault are overwhelmingly female, and perpetrators are overwhelmingly male. Even in the rare cases where the perpetrators are female, contrary to what one might imagine, the victims are still predominantly female. However, when the perpetrator is a priest or another representative of the Catholic Church, the victims are predominantly male (for a recent and global scale study in France : https://www.ciase.fr/rapport-final/ ).

The comparison with elite athletes also jumped to my mind. Mature champions could be good advisors to young champions, but probably not to people with very different profiles and capacities, facing difficulties or problems they never considered, etc. We imagine that because people like Bill Gates or Jeff Bezos succeeded with their companies, they are some kind of universal geniuses and prophets. However, it is also quite possible that if these same people were appointed (anonymously or under a pseudonym, without the benefit of their image, contacts, or for... (read more)

The it from bit (or qbit) hypothesis is fascinating, so is the information paradox, so is quantum mechanics, but I don't think there is any empirical nor theoretical evidence supporting "awareness" - what may it be - of the universe in any of this. No more than evidence supporting god(s) or a flying spaghetti monster. Creating a narrative does not constitute evidence (edit : even if gedankenexperiments are valuable). We are free to speculate, and it is very respectable, however an extraordinary affirmation needs an equally extraordinary amount of proof and... (read more)

Exactly. Future is hard to predict and the author's strong confidence seems suspicious to me. Improvements came fast last years. 

2013-2014 : word2vec and seq2seq 

2017 : transformer and gpt-1 

2022 : CoT prompting 

2023 multimodal LLMs

2024 reasonning models.

Are they linear improvements or revolutionnary breakthroughs ? Time will tell, but to me there is no sharp frontier between increment and breakthrough. It might happen that AGI results from such improvements, or not. We just don't know. But it's a fact that human general intelligence re... (read more)

The authors of the paper remain very cautious about interpreting their results. My intuition regarding this behavior is as follows. 

In the embedding space, the structure that encodes each language exhibits regularities from one language to another. For example, the relationship between the tokens associated with the words 'father' and 'mother' in English is similar to that linking the words 'père' and 'mère' in French. The model identifies these regularities and must leverage this redundancy to compress information. Each language does not need to be r... (read more)

From my perspective, the major issue remains Phase 1. It seems to me that most of the concerns mentioned in the article stem from the idea that an ASI could ultimately find itself more aligned with the interests of socio-political-economic systems or leaders that are themselves poorly aligned with the general interest. Essentially, this brings us back to a discussion about alignment. What exactly do we mean by "aligned"? Aligned with what? With whom? Back to phase 1.

But assuming an ASI truly aligned with humanity in a very inclusive definition and with hig... (read more)

I agree, finding the right balance is definitely difficult.

However, the different versions of this parable of the grasshopper and the ant may not yet go far enough in subtlety.

Indeed, the ants are presented as champions of productivity, but what exactly are they producing? An extreme overabundance of food that they store endlessly. This completely disproportionate and non-circulating hoarding constitutes an obvious economic aberration. Due to the lack of significant consumption and circulation of wealth, the ants' economy—primarily based on the primary sec... (read more)

Don't you think that articles like "Alignment Faking in Large Language Models" by Anthropic show that models can internalize the values present in their training data very deeply, to the point of deploying various strategies to defend them, in a way that is truly similar to that of a highly moral human? After all, many humans would be capable of working for a pro-animal welfare company and then switching to the opposite without questioning it too much, as long as they are paid.

Granted, this does not solve the problem of an AI trained on data embedding unde... (read more)

 "I think the Fall is not true historically". 

While all men must die and all civilizations must collapse, the end of all things is merely the counterpart of the beginning of all things. Creation, the birth of men, and the rise of civilizations are also great patterns and memorable events, both in myths and in history. However, the feeling does not respect symmetry, perhaps due to loss aversion and the peak-end rule, the Fall - and tragedy in general -carries a uniquely strong poetic resonance. Fatum represents the story's inevitable conclusion. T... (read more)

Indeed, nature, and particularly biology, disregards our human considerations of fairness. The lottery of birth can appear as the greatest conceivable inequality. But in this matter, one must apply the Stoic doctrine that distinguishes between what depends on us and what does not. Morality concerns what depends on us, the choices that belong to the moral agents we are.

If I present the lottery of birth in an egalitarian light, it is specifically in the sense that we, as humans, have little control over this lottery. Particularly regarding IQ at birth, regar... (read more)

Yes, of course. Despite its stochastic nature, it is extraordinarily unlikely for an advanced LLM to respond with anything other than 2 + 2 = 4 or Paris for the capital of France. A stochastic phenomenon can, in practice, tend toward deterministic behavior. However, deception in a context such as the one discussed in Apollo Research's article is not really comparable to answering 2 + 2 = ?. What the article demonstrates is that we are dealing with tendencies, accompanied by considerable randomness, including in the intensity of the deception.

Assuming a mor... (read more)

Thank you for this publication. Just an idea (maybe stupid): GPT-type models are stochastic by design. This characteristic might be used as a way to control them. Your study with Apollo Research illustrates this well—there is a certain probability that a given model will deceive users under specific circumstances and to varying degrees. However, this remains fundamentally random, non-deterministic, and that property is quite interesting. There is always a chance that deception does not occur, is clumsy, or stops at an early stage.

For a single response, it... (read more)

You are right. When I wrote my initial comment, I believed the argument was self-evident and did not require elaboration. However, "self-evidence" is not an objective concept, and I likely do not share the same socio-cultural environment as most users of this platform. Upon reading your comment and Ben Pace's, I realize that this apparent self-evidence is far from universally shared and requires further explanation. I have already expanded on my argument in my previous response, but here are the specific reasons why I think the author's project (and indeed... (read more)

Thank you for your kind advice (I made some edits to my previous comment in consequence). I must have expressed myself poorly because I am in no way questioning the idea that science and technology have greatly contributed to improving the condition of humanity. My remark was about inequality. Scientific development is not inherently linked to the increase in inequalities. On the contrary, many scientific and technological advances are likely to benefit everyone. For instance, in many countries, both the rich and the poor use the same tap water, of good qu... (read more)

It is the great paradox of this forum to serve both as a platform where individuals aligned with the effective altruism movement raise alarms and invite deep reflection on existential risks—usually with a nuanced, subtle, and cautious approach to these topics—and as a space where individuals adhering to libertarian or transhumanist ideologies also promote rather radical ideas, which themselves might constitute new existential challenges for humanity. I say this not with the intent of disparagement but rather as an observation.

This topic is a fascinating ex... (read more)

A new existential risk that I was unaware of. Reading this forum is not good for peaceful sleeping. Anyway, a reflexion jumped to me. LUCA lived around 4 billion years ago with some chirality chosen at random. But, no doubt that many things happened before LUCA and it is reasonable to assume that there was initially a competition between right-handed protobiotic structures and left-handed ones, until a mutation caused symmetry breaking by natural selection. The mirrored lineage lost the competition and went to extinction, end of the story. But wait, we spe... (read more)

I am sorry to say that on a forum where many people are likely to have been raised in a socio-cultural environnement where libertarian ideas are deeply rooted. My voice will sound dissonant here and I call to your open-mindedness.

I think that there are strong limitations to such ideas as developed in the OP proposal. Insurance is mutualization of risk, it's a statistic approach relying on the possibility to assess a risk. It works for risks happening frequently, with a clear typology, like car accidents, tempest, etc. Even in these cases there is always an... (read more)