All of Perhaps's Comments + Replies

There are some models on HuggingFace that do automatic PII data redaction, I've been working on a project to automate redaction for documents with them. AI4privacy's models and Microsoft Presidio have been helpful.

You might find some puzzle games to be useful. In particular Understand is a game that was talked about on here as being good for learning how to test hypotheses and empirically deduce patterns. Similar to your Baba Is You experiments. 

Any way that we can easily get back our own results from the survey? I know you can sometimes get a copy of your responses when you submit a Google form.

What happens to the general Lightcone portfolio if you don't meet a fundraising target, either this year or a future year?

For concreteness, say you miss the $1M target by $200K. 

The karma buttons are too small for actions that in my experience, are done a lot more than clicking to listen to the post. It's pretty easy to misclick.

Additionally, it's unclear what the tags are, as they're no longer right beside the post to indicate their relevance. 

I think this post would benefit from an abstract / summary / general conclusion that summarizes the main points and makes it easier to interact with. Usually I read a summary to get an idea of a post, then browse the main points and see if I'm interested enough to read on. Here, it's hard to engage, because the writing is long and the questions it seems to deal with are nebulous.

How did you find LessWrong?

Do still have any Mormon friends? Do you want to help them break away, do you think it's something they should do on their own, or do you find whether they remain Mormon or not immaterial?

Do you think being a Mormon was not suited for you, or do you think it doesn't work as a way of life in general? How do you think that your answer would change 50 years ago vs today?

Did you have contact/ongoing relationships with other Mormon communities while you were there? What is the variation between people/communities? How devout/lax are d... (read more)

Well, someone was working on a similar-ish project recently, @Bruce Lewis with HowTruthful. Maybe you two can combine your ideas or settle on an amalgamation together. 

If possible, please let us know how it goes a couple months from now!

So this is Sam Altman raising the 5-7 trillion, not OpenAI as an entity, right?

Could some kind of caustic gas, or the equivalent of a sandstorm be used to make drones not useful? I feel like large scale pellet spreads wouldn't be too useful if the drones are armoured, but I don't know too much about armour or how much piercing power you could get. I wonder if some kind of electric netting could be fired to mass electrocute a swarm, or maybe just regular netting that interferes with their blades. Spiderwebs from the sky?

Interesting post, although I feel like it would benefit from inline references. For most of the post it feels like you're pulling your assertions out of nowhere, and only at the end do we get some links to some of the things you said. I understand time/effort constraints though.

I derive a lot of enjoyment from these posts, just walking through tidbits of materials science is very interesting. Please keep making them.

I think at its most interesting it looks like encrypting your actions and thought processes so that they look like noise or chaos to outside observers.

I think what those other things do is help you reach that state more easily and reliably. It's like a ritual that you do before the actual task, to get yourself into the right frame of mind and form a better connection, similar to athletes having pre game rituals.

Also yeah, I think it makes the boredom easier to manage and helps you slowly get into it, rather than being pushed into it without reference. 

Probably a lot of other hidden benefits though, because most meditation practices have been optimized for hundreds of years, and are better than others for a reason.

I feel like it's not very clear here what type of coordination is needed.

How strong does coordination need to become before we can start reaching take off levels? And how material does that coordination need to be?

Strong coordination, as I'm defining here, is about how powerfully the coordination constrains certain actions.

Material coordination, as I'm defining here, is about on what level the coordination "software" is running. Is it running on your self(i.e. it's some kind of information that's been coded into the algorithm that runs on your brain, examp... (read more)

Rather than this Feeling Good app for patients, I'd be more interested in an app that let people practice applying CBT techniques to patient case studies(or maybe even LLMs with specified traits), in order to improve their empathy and help them better understand people. If this could actually develop good therapists with great track records, then that would prove the claims made in this article and help produce better people.

I'm not sure it only applies to memory. I imagine that ancient philosophers had to do most of their thinking in their heads, without being able to clean it up by writing it out and rethinking it. They might be better able to edit their thoughts in real time, and might have a stronger control over letting unreasonable or not-logical thoughts and thought processes take over. In that sense, being illiterate might lend a mental stability and strength that people who rely on writing things out may lack. 

Still, I think that the benefits of writing are too enormous to ignore, and it's already entrenched into our systems. Reversing the change won't give a competitive edge.

If compute is limited in the universe, we can expect that civilizations or agents with access to it will only run simulations strategically, unless running simulations is part of their value function. Simulations according to a value function would probably be more prevalent, and would probably have spiderman or other extreme phenomena. 

However, we can't discount being in one of those information gathering simulations. If for some reason you needed to gather information from a universe, you'd want to keep everything as simple as possible, and only tun... (read more)

It's possible that with the dialogue written, a well prompted LLM could distill the rest. Especially if each section that was distilled could be linked back to the section in the dialogue it was distilled from.

I like the ideal, but as a form of social media it doesn't seem very engaging, and as a single source of truth it seems strictly worse than say, a wiki. Maybe look at Arbital, they seem to have been doing something similar. I also feel that dealing with complex sentences with lots of implications would be tough, there are many different premises that lead to a statement. 

Personally I'd find it more interesting if each statement was decomposed into the premises and facts that make it up. This would allow tracing an opinion back to find the crux between... (read more)

I guess while we're anthropomorphizing the universe, I'll ask some crux-y questions I've reached.

If humanity builds a self-perpetuating hell, does the blame lie with humanity or the universe?

If humanity builds a perfect utopia, does the credit lie with humanity or the universe?

Frankly it seems to me like what's fundamentally wrong with the universe is that it has conscious observers, when it needn't have bothered with any to begin with.    

I think that most of the people who would take notes on LW posts are the same people who would benefit from, and may use, a general note taking system. A system like Obsidian or Notion or whatever would be used for a bunch of stuff, LW posts included. In that sense, I think it's unlikely that they'd want a special way to note-take just for LW, when it'd probably be easier and more standardized to use their existing note taking system.

If you do end up going for it, an "Export Notes" feature would be nice, in an easily importable format.

I think this is pretty good advice. I am allergic to nuts, and that has defined a small but occasionally significant part of my interactions with people. While on the whole I'd say I've probably experienced more negative experiences because of it(once went into anaphylaxis), I've often felt that it marked me as special or different from other people. 

About 5 or so years ago my mom heard about a trial run by a doctor where they fed you small amounts of what you're allergic to in order to desensitize and acclimate your immune system to the food. She rec... (read more)

It seems like multi-modality will also result in AIs that are much less interpretable than pure LLMs.

This seems like a pretty promising approach to interpretability, and I think GPT-6 will probably be able to analyze all the neurons in itself with >0.5 scores. Which seems to be recursive self-improvement territory. It would be nice if by the time we got there, we already mostly knew how GPT-2, 3, 4, and 5 worked. Knowing how previous generation LLMs work is likely to be integral to aligning a next generation LLM and it's pretty clear that we're not going to be stopping development, so having some idea of what we're doing is better than none. Even if an... (read more)

Indeed, in India especially it's not uncommon for people to be dragged off the streets and have their organs removed and sold by human traffickers, and killed after that. Making selling kidneys illegal at least ensures that this isn't an easy and straightforward thing to do. In Pakistan for example, an estimated 2500 kidneys were sourced in 2007.

There's also The Work Gym and Pentathalon from Ultraworking.

Waiting for the day all my AI safety bookmarks can be summarized into just one website.

Just read your novel, it's good! And has successfully reignited my AI doomer fears! I was a bit surprised by the ending, I was about 60/40 for the opposite outcome. I enjoyed the explainer at the end and and I'm impressed by your commitment to understanding AI. Please keep writing, we need more writers like you!

Well in the end, I think the correct view is that as long as the inventor is making safety measures from first principles, it doesn't matter whether they're an empath or a psychopath. Why close off part of the human race who are interested in aligning the world ending AI just because they don't have some feelings? It's not like their imagined utopia is much different from yours anyways.

Honestly I don't think that in the aftermath of a full-scale nuclear war or large asteroid impact any government would be funneling money into AGI. The entire supply chain would be broken, and they'd be scrambling just to keep basic life support on. This is mostly a nitpick though, as I agree with your points and I think this is sufficiently unlikely as to not matter.

I love this story, thanks for making it.

I love the Team Physics and Team Manipulation characterization, gives big pokemon vibes.

Excited and happy that you are moving forward with this project. It's great to know that more paths to alignment are being actively investigated. 

Bought this game because of the recommendation here, and it has replaced reading I Spy books with my sister as our bonding activity. I really like the minimalism, and its lack of addictive qualities. I've only got to 2-7 so far, but the fact that I eventually get stuck after about half an hour to an hour of playing means that it provides a natural stopping point for me, which is pretty nice. Thank you for the great review!

I think it's pretty reasonable when you consider the best known General Intelligence, humans. Humans frequently create other humans and then try to align them. In many cases the alignment doesn't go well, and the new humans break off, sometimes to vast financial and even physical loss to their parents. Some of these cases occur when the new humans are very young too, so clearly it doesn't require having a complete world model or having lots of resources. Corrupt governments try to align their population, but in many cases the population successfully revolt... (read more)

I think the point is more like, if you believe that the brain could in theory be emulated, with infinite computation(no souls or mysterious stuff of consciousness), then it seems plausible that the brain is not the most efficient conscious general intelligence. Among the general space of general intelligences, there are probably some designs that are much simpler than the brain. Then the problem becomes that while building AI, we don't know if we've hit one of those super simple designs, and suddenly have a general intelligence in our hands(and soon out of... (read more)

In addition to what Jay Bailey said, the benefits of an aligned AGI are incredibly high, and if we successfully solved the alignment problem we could easily solve pretty much any other problem in the world(assuming you believe the "intelligence and nanotech can solve anything" argument). The danger of AGI is high, but the payout is also very large.

In terms of utility functions, the most basic is: do what you want. "Want" here refers to whatever values the agent values. But in order for the "do what you want" utility function to succeed effectively, there's a lower level that's important: be able to do what you want. 

Now for humans, that usually refers to getting a job, planning for retirement, buying insurance, planning for the long-term, and doing things you don't like for a future payoff. Sometimes humans go to war in order to "be able to do what you want", which should show you that satisfyi... (read more)

The first type of AI is a regular narrow AI, the type we've been building for a while. The second type is an agentic AI, a strong AI, which we have yet to build. The problem is, AIs are trained using gradient descent, which basically involves running AI designs from all possible AI designs. Gradient descent will train the AI that can maximize the reward best. As a result of this, agentic AIs become more likely because they are better at complex tasks. While we can modify the reward scheme, as tasks get more and more complex, agentic AIs are pretty much the way to go, so we can't avoid building an agentic AI, and have no real idea if we've even created one until it displays behaviour that indicates it.

Awesome post, putting into words the intuitions I had for what dimensions the alignment problem stayed in. You've basically meta-bounded the alignment problem, which is exactly what we need when dealing with problems like this.

Well it depends on your priors for how an AGI would act, but as I understand it, all AGIs will be powerseeking. If an AGI is powerseeking, and has access to some amount of compute, then it will probably bootstrap itself to superintelligence, and then start pushing its utility function all over. Different utility functions cause different results, but even relatively mundane ones like "prevent another superintelligence from being created" could result in the AGI killing all humans and taking over the galaxy to make sure no other superintelligence gets made.... (read more)

I'd say building an AGI that self-destructs would be pretty good. Especially since up until the point that a minimum breeding population of humans exists, and assuming life is not totally impossible(i.e. the AI hasn't already deconstructed the earth, or completely poisoned all water and atmosphere), humans could still survive. Making an AGI that doesn't die would probably not be in our best interests until almost exactly the end.

Thanks for the answer! As you suspected, I don't think wireheading is a good thing, but after reading about infinite ethics and the repugnant conclusion I'm not entirely sure that there exists a stable mathematically expressible form of ethics we could give to an AGI. Obviously I think it's possible if you specify exactly what you want and tell the AGI not to extrapolate. However I feel that realistically, it's going to take our ethics and take it to its logical end, and there exists no ethical theory that really expresses how utility should be valued with... (read more)

Would the appropriate analogy to agents be that humans are a qualitatively different type of agent compared to animals and basic RL agents, and thus we should expect that there will be a fundamental discontinuity between what we have so far, and conscious agents?

You may also want to consider opportunities on the EA Volunteer Job Board. Some of them are similar low effort wiki building.

https://airtable.com/embed/shrQvU9DMl0GRvdIN/tbll2swvTylFIaEHP

I think in general, the most innovative candies have been candies that break the norm. I remember a lot of buzz when some gum company made gum wrappers that you could eat with your gum(Cinnaburst?) Nowadays though, it seems like companies don't need to go that far for people to buy their new chocolate/candy, and there are so many flavours and textures they can slap on if people get tired.

Hi, I really like this series and how it explains some of the lower level results we can expect from high level future scenarios. However I'd like to know how you expect digital people will interact with an economy that has been using powerful, high-level AI models or bureaucracies for a couple decades or longer(approximately my timeline for mind uploading, assuming no singularity). I've mostly read LessWrong posts and haven't done anything technical, but I feel that a lot of the expected areas in which digital people would shine might end up being accommodated by narrow-ish AI.