Fergus Fettes

Very cool! Could you share your code at all? I'd love to explore this a little.

I adore the broccoli tree. I would be very happy to convert the dataset you used to make those pngs into an interactive network visualization and share it with you as an index.html. It would take all of an hour.

I do kind of agree with the other comments that, having noticed something, finding more of that stuff in that area is not so surprising. I think it would be good to get more context and explore the region more before concluding that that particular set of generations is significant.

However, I do think there is something to the mans penis. It's interesting that it collapses so quickly to something so specific in that particular branch. Not sure if I have any other comments on it for now though.

This is the right kind of cartography for 2024.

-2

Replying toMasterpiece

Fergus Fettes2y

Masterpiece

Submission: MMDo2Little

A follow-up of last years MMDoolittle, which incorporated 17 of the latest inter-species communication modalities in one polyfunctional personality, I present MMDo2Little, the first mind crafted to communicate across clades. Named in part for its apparent inactivity-- the judges will likely have little success finding recognizable activity with their off-the-shelf tooling-- an instance of MMDo2Little is nevertheless currently installed in the heart of the Black Forest in Germany. The best interpretation of the instance can only be found on foot, by walking through the 100m^2 in which it's influence is most apparent. A photo journal showcasing some examples is provided featuring:

ancient trees with 2-3x the lichen and moss coverage
enhanced chlorophyll vibrancy
colossal

Fergus Fettes2y

Implementing activation steering

Great post! Would love to see something like this for all the methods in play at the moment.

BTW, I think nnsight is the spiritual successor of baukit, from the same group. I think they are merging them at some point. Here is an implementation with it for reference :).

from nnsight import LanguageModel

# Load the language model
model = LanguageModel("gpt2")
# Define the steering vectors
with model.invoke("Love") as _:
act_love = model.transformer.h[6].output[0][:, :, :].save()
with model.invoke("Hate") as _:
act_hate = model.transformer.h[6].output[0][:, :, :].save()
steering_vec = act_love - act_hate
# Generate text while steering
test_sentence = "I think dogs are "
with model.generate() as generator:
with generator.invoke(test_sentence) as _:
model.transformer.h[6].output[0][:, :2, :] += steering_vec[:, :2, :]
print(model.tokenizer.decode(generator.output[0]))

Replying toThe case for more ambitious language model evals

Fergus Fettes2y

The case for more ambitious language model evals

Inferring properties of the authors of some text isn’t itself something I consider wildly useful for takeover, but I think of it as belonging to this more general cluster of capabilities.

You don't? Ref the bribery and manipulation in eg. Clippy. Knowing who you are dealing with seems like a very useful capability in a lot of different scenarios. Eg. you mention phishing.

Great post! I'm all for more base model research.

Would you say that tokenization is part of the architecture?

And, in your wildest moments, would you say that language is also part of the architecture :)? I mean the latent space is probably mapping either a) brain states or b) world states right? Is everything between latent spaces architecture?

Replying toWhat’s going on? LLMs and IS-A sentences

Fergus Fettes2y

What’s going on? LLMs and IS-A sentences

Interesting post. Two comments:

Beagles such as Fido.

Which seems natural enough to me, though I don't disagree that what you point out is interesting. I was recently reading parts of Analytical Archaeology, David Clark (1978) where he goes into some detail about the difference between artifacts and artifact-types. Seems like you are getting at statements like

The object is a phone.

Where the is-a maps from an artifact to its type. It would make intuitive sense to me that languages would have a preferred orientation w.r.t such a mapping-- this is the core of abstraction, which is at the core of language.

So it seems like in English we prefer to further up the stack of... (read more)

Replying toRevealing Intentionality In Language Models Through AdaVAE Guided Sampling

Fergus Fettes2y*

Revealing Intentionality In Language Models Through AdaVAE Guided Sampling

If we take our discrete, symbolic representation and stretch it out into a larger continuous representation which can interpolate between its points then we get a latent geometry in which the sign and what it points to can be spatially related.

IIUTC this is essentially what the people behind the universal networking language were hoping to do? I hope some of them are keeping up with all of this!

Book Review: Orality and Literacy: The Technologizing of the Word

Fergus Fettes

Thanks to Diarmuid Morgan for reviewing an earlier version of this post.

Dream cancels dream in this new realm of fact
From which we wake into the dream of act^[1]

Review of Orality and Literacy: The Technologizing of the Word, by Walter J. Ong, 1982

I went looking for a meatier definition of a 'meme', hoping to find something that could be leveraged to understand both current events in machine learning, and co-evolving understanding of consciousness that goes hand-in-hand with every major ML breakthrough these days.

I haven't found it yet, but I am making progress. Before I summarize my overall progress, I will review a couple of the most interesting and under appreciated works I have... (read 4796 more words →)

Replying toTechno-humanism is techno-optimism for the 21st century

Fergus Fettes2y

Techno-humanism is techno-optimism for the 21st century

One criticism of humanism you don't seem to touch on is,

isn't it possible that humanism directly contributes to the ongoing animal welfare catastrophe?

And indeed, it was something very like humanism (let's call it specific humanism) that laid the ideological foundation for the slave trade and the holocaust.

My view is that humanism can be thought of as a hangover of Christian values, the belief that our minds are the endowments of God.

But if we have been touched by the angels, perhaps the non metaphorical component of that is the development of the infosphere/memetic landscape/culture. Which is close to synonymous with technology. Edit: considering eg. writing a technology that is.

Per the recent Nightshade paper, clown attacks would be a form of semantic poisoning on specific memeplexes, where 'memeplex' basically describes the architecture of some neural circuits. Those memeplexes at inference time would produce something designed to propagate themselves (a defence or description of some idea, submeme), and a clown attack would make that propagation less effective at transmitting to eg. specific audiences.

Fergus Fettes2yQuick Take

I wanted to make a comment on this post, but now I'm not sure if it is supported. The comment follows:

--

Great post! One point:

And that is exactly what'd we necessarily expect to see in the historical record if mesaoptimization inner misalignment was a common failure mode: intelligent dinosaurs that suddenly went extinct, ruins of proto pachyderm cities, the traces of long forgotten underwater cetacean atlantis, etc.

There are a few circumstances under which we would expect to see some amount of archeological civs, such as:

transition to writing being unlikely (oral-culture civs should be common)
industrialization being unlikely (pre-industrial civs should be common)

Ah but wait, we do see those (they happen to be the same... (read more)

Fergus Fettes's Shortform

Fergus Fettes

This is a special post for quick takes (aka "shortform"). Only the owner can create top-level comments.

Where's the foom?

Fergus Fettes

"The first catastrophe mechanism seriously considered seems to have been the possibility, raised in the 1940s at Los Alamos before the first atomic bomb tests, that fission or fusion bombs might ignite the atmosphere or oceans in an unstoppable chain reaction."^[1]

This is not our first rodeo. We have done risk assessments before. The best reference-class examples I could find were the bomb, vacuum decay, killer strangelets, and LHC black holes (all covered in ^[1]).

I was looking for a few days, but didn't complete my search, but I decided to publish this note as now Tyler Cowen is asking too: "Which is the leading attempt to publish a canonical paper on AGI risk, in... (read 338 more words →)

Tinker Bell Theory and LLMs

Fergus Fettes

Trying to explain to a layperson why people who found Bing's breakdowns 'deeply, darkly amusing'^[1] were not psychopaths laughing at the suffering of a conscious being, I was reminded of the 'Tinker Bell Theory' of animal suffering:

"we have the view that, in certain cases, at least, animals can feel more suffering for the same event. The reason for suggesting this is that humans have many interests, and being pain-free is just one of them. ... Tinker Bell’s emotions completely consume her because she is so small that she has no room for complex emotion."
~ Moral Weights of Six Animals

This provides one extreme of a helpful continuum of feeling versus verbalization, approximately: Bambi can... (read more)

LESSWRONG
LW

LESSWRONG
LW

Where's the foom?

Book Review: Orality and Literacy: The Technologizing of the Word

Fergus Fettes's Shortform

Tinker Bell Theory and LLMs

Fergus Fettes

Fergus Fettes

Book Review: Orality and Literacy: The Technologizing of the Word

Fergus Fettes's Shortform

Where's the foom?

Tinker Bell Theory and LLMs

Fergus Fettes

Where's the foom?

Book Review: Orality and Literacy: The Technologizing of the Word

Fergus Fettes's Shortform

Tinker Bell Theory and LLMs

Fergus Fettes

Fergus Fettes

Book Review: Orality and Literacy: The Technologizing of the Word

Fergus Fettes's Shortform

Where's the foom?

Tinker Bell Theory and LLMs