(This started as a reply to @Tamsin Leake 's reply in my post about why cyborgism maybe should be open. This post does not require you to read our interaction, though it lead to this, and I'm very grateful for Tamsin's reply.) In general, this is a counterargument against: >...
(note: lots of discussion happened in the comments, you'll want to read it if you found this post interesting) TLDR: - Building powerful human-AI collaboration tools in the open removes capabilities overhang, which reduces discourse lag, which reduces x-risk. - Alignment work is philosophy/writing/thinking-heavy, capabilities work is coding-heavy. Cyborg tools...
It seems easier to fit an analogy to an image than to generate an image starting with a precise analogy (meaning a detailed prompt).
Maybe because an image is in a higher-dimensional space and you're projecting onto the lower-dimensional space of words.
(take this analogy loosely idk any linear algebra)Claude: "It's similar to how it's easier to recognize a face than to draw one from memory!"
(what led to this:)
I've been trying to create visual analogies of concepts/ideas, in order to compress them and I noticed how hard it is. It's hard to describe images, and image generators have trouble sticking to descriptions (unless you pay a lot per image).
Then I started instead, to just... (read more)
On our "interfaces" to the world...
This is a very exploratory post.
It underlies my thinking in this post: https://www.lesswrong.com/posts/wQ3bBgE8LZEmedFWo/root-node-of-my-posts but it's hard to put into words but I'll make a first attempt here. I also expressed some of it here, https://www.lesswrong.com/posts/MCBQ5B5TnDn9edAEa/atillayasar-s-shortform?commentId=YNGb5QNYcXzDPMc5z, in the part about deeper mental models / information bottlenecks.
It's also why I've been spending over a year making a GUI that can encompass as much of my activities on the computer as possible -- which is most of my entire life.
(The beauty of coding personal tooling is that you get to construct your own slice of reality --- which can be trivial or extremely profound depending on your abilities and on... (read 554 more words →)
Just because X describes Y in a high level abstract way, doesn't mean studying X is the best of understanding Y.
Often, the best way is to simply study Y, and studying X just makes you sound smarter when talking about Y.
This is based on my experience trying to learn stuff about cybernetics, in order to understand GUI tool design for personal use, and to understand the feedback loop that roughly looks like, build -> use -> rethink -> let it help you rethink -> rebuild, where me and any LLM instance I talk to (via the GUI) are part of the cybernetic system. Whenever I "loaded cybernetics... (read more)
Jocko Willink talking about OODA loops, paraphrase
The f-86 wasn't as good at individual actions, but it could transition between them faster than the MiG-15
analogous to how end-to-end algorithms, llm agents, and things optimized for the tech demo, are "impressive single actions", but not as good for long term tasks
Two tools is a lot more complex than one, not just +1 or *2
When you have two tools, you have think about their differences, or about specifically what they each let you do, and then pattern match to your current problem, before using it. With one tool, you don't have to understand the shape of its contents at all, because it's the only tool and you already know you want to use it.
Let's compare the amount of information you need to remember with 1 vs 2 tools. You want food (task), you're going to get it from a supermarket (tool).
With 1 available supermarket:
... (read more)You want food. Supermarket has food. Done.
Twitter is designed for writing things off the top of your head, and things that others will share or reply to. There are almost no mechanisms to reward good ideas, to punish bad ones, nor for incentivizing the consistency of your views, nor any mechanism for even seeing whether someone updates their beliefs, or whether a comment pointed out that they're wrong.
(The fact that there are comments is really really good, and it's part of what makes Twitter so much better than mainstream media. Community Notes is great too.)
The solution to Twitter sucking, is not to follow different people, and DEFINITELY not to correct every wrong statement (oops), it's... (read more)
when chatting with an LLM, do you wonder what its purpose is in the responses it gives? I'm pretty sure it's "predict a plausible next token", but I don't know how I'll know to change my belief.
I think "it has a simulated purpose, but whether it has an actual purpose is not relevant for interacting with it".
My intuition is that the token predicter doesn't have a purpose, that it's just answering, "what would this chatbot that I am simulating, respond with?"
For the chatbot character (the Simulacrum) it's, "What would a helpful chatbot want in this situation?" It behaves as if its purpose is to be helpful and harmless (or whatever personality it... (read more)
personality( ground_truth ) --> stated_truth
Or, what someone says is a subset of what they're really thinking, transformed by their personality.
(personality plus intentions, context, emotions, etc.)
(You can tell I'm not a mathematician because I didn't express this in latex xD but I feel like there's a very elegant Linear Algebra description where, ground truth is a high dimensional object, personality transforms to make it low-dimensional enough to communicate, and changes its "angle" (?) / vibe so it fits their goals better)
So, if you know someone's personality, meaning, what kind of way they reshape a given ground truth, you can infer things about the ground truth.
I feel like everyone does this implicitly, it's one... (read more)
this shortform: https://www.lesswrong.com/posts/MCBQ5B5TnDn9edAEa/atillayasar-s-shortform?commentId=QCmaJxDHz2fygbLgj was spawned from what was initially a side-note in the above one.
In this Shortform I talked about why interactive systems that include AI are more powerful than end-to-end AI-powered algorithms. But it misses something big, a concept where actually most of my surprising ideas about UI design have come from.
The raw data underneath any given UI, and the ability to define functions that "do things at certain points" to that data, is very elementary relative to the richness of that UI, and the things that it lets you do.
Something mundane like "article -> list of (index, string)-pairs representing paragraphs" can become, "menu of clickable chapters sitting beside the main editor window, that take you to that section".... (read more)
It's summer 2020 and gpt3 is superhuman in many ways and probably could take over the world but only when its alien goo flows through the right substrate, channeled into the right alchemical chambers, then forked and rejoined and reshaped in the right way, but somehow nobody can build this.
Normal agent-systems (like what you design with LangChain) are brittle in that you're never sure if they'll do the job. A more robust system is one where you can stop execution when it's going wrong, and edit a previous output, or edit the code that generated it, and retry.
Neat things like the $do command of Conjecture's Tactics,... (read 1230 more words →)
Meta
This post is useful to me because 1) it helped me think more clearly about whether and how exactly philosophy is useful, 2) I can read it later and get those benefits again.
The problem
Doing philosophy and reading/writing is often so impractical, people do it just for the sake of doing it. When you write or read a bunch of stuff about X, get categories and lists and definitions, it feels like you're making progress on X, but are you really?
Joseph Goldstein (meditation teacher) at the beginning of his lecture about Mindfulness, jokes that they'll do an hour and a half of meditation, then after pausing for laughter, points out... (read 753 more words →)
(This started as a reply to @Tamsin Leake 's reply in my post about why cyborgism maybe should be open. This post does not require you to read our interaction, though it lead to this, and I'm very grateful for Tamsin's reply.)
In general, this is a counterargument against:
we should only share cyborg tools (software that lets AI help us think) among AI safety people, so that the big labs don't get ahold of them, so we can save the world before they end it
My idea is that IF humanity doesn't want to die, we can discover this by maximizing information sharing and "converging" our culture and discourse towards what is *actually*... (read 1718 more words →)
(note: lots of discussion happened in the comments, you'll want to read it if you found this post interesting)
TLDR:
- Building powerful human-AI collaboration tools in the open removes capabilities overhang, which reduces discourse lag, which reduces x-risk.
- Alignment work is philosophy/writing/thinking-heavy, capabilities work is coding-heavy. Cyborg tools are more for the former than the latter, and great coding tools already exist.
Given this, "safety concerns" like "but what if someone uses your app and discovers jailbreaks or hacks somebody" are not actually a problem. (maybe even net positive, since they "update discourse" on unknown dangers, more on this later)
(tldr: cyborg tools help... (read 791 more words →)
Editability and findability --> higher quality over time
Editability
Code being easier to find and easier to edit, for example,
makes it more likely to be edited, more subject to feedback loop dynamics.
Same applies to writing, or anything where you have connected objects that influence each other, where the "influencer node" is editable and visible.
... (read more)