I mean, like, immortality? Abundance? Perfect health? World peace via all mentioned means and improved coordination? Human intelligence augmentation, where by intelligence I mean everything including and not limiting creativity, wisdom, perception clarity and self-awareness? Space colonization, which is first of all, ability to opt out from our current civilization and try something else?

You can say "but humans can invent all of this eventually anyway" but I dare to remind you that there is 14000 children dying every day and, conditional of alignment, AGI is a fastest way to stop it.

[edit: pinned to profile]

I want to be able to calculate a plan that converts me from biology into a biology-like nanotech substrate that is made of sturdier materials all the way down, which can operate smoothly at 3 kelvin and an associated appropriate rate of energy use; more clockworklike - or would it be almost a superfluid? Both, probably, clockworklike but sliding through wide, shallow energy wells in a superfluid-like synchronized dance of molecules - Then I'd like to spend 10,000 years building an artful airless megastructure out of similarly strong materials as a series of rings in orbit of Pluto. I want to take a trip to alpha centauri every few millennia for a big get together of space-native beings in the area. I want to replace information death with cryonic sleep, so that nothing that was part of a person is ever forgotten again. I want to end all forms of unwanted suffering. I want to variously join and leave low latency hiveminds, retaining my selfhood and agency while participating in the dance of a high-trust high-bandwidth organization that respects the selfhood of its members and balances their agency smoothly as we create enormous works of art in deep space. I want to invent new kinds of culinary arts for the 2 to 3 kelvin lifestyle. I want to go swimming in Jupiter.

I want all of Earth's offspring to ascend.

Check out the Fun Theory sequence, if you haven't already.

Turn everyone hot

If we can do that due to AGI, almost surely we can solve aging, which would be truly great. 

I would say value preservation and alignment of the human population. I think these are the hardest problems the human race faces, and the ones that would make the biggest difference if solved. You're right, humanity is great at developing technology, but we're very unaligned with respect to each other and are constantly losing value in some way or another. 

If we could solve this problem without AGI, we wouldn't need AGI. We could just develop whatever we want. But so far it seems like AGI is the only path for reliable alignment and avoiding Molochian issues.

Bah, nobody's mentioned social applications of superhuman planning yet.

You could let an AI give everyone subtle nudges, and a month later everone's social life will be great. You'll see your family the right amount, you'll have friends who really get you who you see often and do fun things with. Sex will occur. Parties and other large gatherings will be significantly better.

The people to make this possible are all around us, it's just really hard to figure out how to make it happen.

Human potential is the big one for me.

Personally, I feel that my imagination is limited - not just around the capabilities of AGI, but in common work scenarios.

There are lots of people out there who are a lot smarter than me, but AGI can help me realise more of my human potential.

This applies both at a personal level, but eventually at a societal level and at a species level beyond that.

What this looks like and how it is made safe through safeguards, I don’t know. But I’m interested in how AGI can help us achieve our human potential in ways that I as an individual can’t imagine without the help of AGI / the sum of human knowledge.

How super/general the AI is is a knob you can set to whatever you want. With zero set to the present day, if you turn it up far enough you get godlike capability of which I find it impossible to say anything.

More modest accomplishments I would pay, say, the price of a car for would include a robot housekeeper that can cope with all of a human’s clutter, and clean everything, make beds, etc. as well as I can and better than in practice I will. Or a personal assistant that I can rely on for things like making complex travel arrangements, searching for stuff on the internet with the accuracy of Google and the conversational interface of ChatGPT, and having serious discussions with on any subject.

Beyond that my creativity falters. In 1980 I couldn’t even have foreseen Google, smartphones, or cat videos.

So, I don't think we need AGI for this... But Digital humans. Uploads from preserved brains. The freedom from carbon-based substrate, and the benefits that go with that like immortality and speed-of-light travel.

I suggest that ideally we keep AI as weak as possible to get us quickly to digital humans, and then have the digital humans do the AI work. Never go down the path of non-human intelligence which is fully superior to human intelligence. Keep the future human! Empower Us, not Them!

I really want a brain computer interface that is incredibly transformative and will allow me to write in my head and scaffold my thinking.