All of samhealy's Comments + Replies

(Your response and arguments are good, so take the below in a friendly and non-dogmatic spirit)

Good enough for what?

Good enough for time-pressed people (and lazy and corrupt people, but they're in a different category) to have a black-box system do things for them that they might, in the absence of the black-box system, have invested effort to do themselves, and as an intended or unintended result, increased their understanding, opening up new avenues of doing and understanding. 

We're still in the "wow, an AI made this" stage.

I'm pretty sure we're cur... (read more)

I don't think I'm selling what you're not buying, but correct me if I misrepresent your argument:

The post seems to assume a future version of generative AI that no longer has the limitations of the current paradigm which obligate humans to check, understand, and often in some way finely control and intervene in the output...

Depending on your quality expectations, even existing GenAI can make good-enough content that would otherwise have required nontrivial amounts of human cognitive effort. 

but where that tech is somehow not reliable and independent e

... (read more)
1mako yass
This doesn't seem to be true to me. Good enough for what? We're still in the "wow, an AI made this" stage. We find that people don't value AI art, and I don't think that's because of its unscarcity or whatever, I think it's because it isn't saying anything. It either needs to be very tightly controlled by an AI-using human artist, or the machine needs to understand the needs of the world and the audience, and as soon as machines have that... All communications assume that the point they're making is important and worth reading in some way (cooperative maxim of quantity). I'm contending that that assumption isn't true in light of what seems likely to actually happen immediately or shortly after the point starts to become applicable to the technology, and I have explained why, but I might be able to understand if it's still confusing, because: is true, but that doesn't mean we need to worry about this today. By the time we have to worry about preserving our understanding of the creative process against automation of it, we'll be on the verge of receiving post-linguistic knowledge transfer technologies and everything else, quicker than the automation can wreak its atrophying effects. Eventually it'll be a problem that we each have to tackle, but we'll have a new kind of support, paradoxically, learning the solutions to the problem will not be our problem.

For what it's worth, I think even current, primitive-compared-to-what-will-come LLMs sometimes do a good job of (choosing words carefully here) compiling information packages that a human might find useful in increasing their understanding. It's very scattershot and always at risk of unsolicited hallucination, but in certain domains that are well and diversely represented in the training set, and for questions that have more or less objective answers, AI can genuinely aid insight. 

The problem is the gulf between can and does. For reasons elaborated in... (read more)

2lemonhope
I think the micro rewards thing could work very well/naturally in a tutor style
2[anonymous]
All I can think of is how, with current models plus a little more Dakka, for genAI to deeply research a topic. It wouldn't be free. You might have to pay a fee with varying package prices. The model then buys a 1 time task license for say 10 reference books on the topic. It reads each one, translating it from "the original text and images" to a distilled version that focuses on details relevant to your prompt. It assembles a set of "notes" where each note cites directly the text it was from. (And another model session validates this assertion) It constructs a summary or essay or whatever form it needs to be in from the notes. Masters thesis grade in 10 minutes and under $100...

To clarify: I didn't just pick the figures entirely at random. They were based on the below real-world data points and handwavy guesses.

... (read more)

Agreed, largely. 

To clarify, I'm not arguing that AI can't surpass humanity, only that there are certain tasks for which DNNs are the wrong tool and a non-AI approach is and possibly always will be preferred. 

An AI can do such calculations the normal way if it really needs to carry them out

This is a recapitulation of my key claim: that any future asymptotically powerful A(G)I (and even some current ChatGPT + agent services) will have non-AI subsystems for tasks where precision or scalability is more easily obtained by non-AI means, and that there will probably always be some such tasks.

Plucked from thin air, to represent the (I think?) reasonably defensible claim that a neural net intended to predict/synthesise the next state (or short time series of states) of an operating system would need to be vastly larger and require vastly more training than even the most sophisticated LLM or diffusion model.

1samhealy
To clarify: I didn't just pick the figures entirely at random. They were based on the below real-world data points and handwavy guesses. * ChatGPT took 3.23 x 10^23 FPOPs to train * ChatGPT has a context window of 8K tokens * Each token is roughly equivalent to four 8-bit characters = 4 bytes, so the context window is roughly equivalent to 4 x 8192 = 32KB * The corresponding 'context window' for AIOS would need to be its entire 400MB+ input, a linear scaling factor of 1.25 x 10^4 from 32KB, but the increase in complexity is likely to be much faster than linear, say quadratic * AIOS needs to output as many of the 2 ^ (200 x 8 x 10 ^ 6) output states as apply in its (intentionally suspect) definition of 'reasonable circumstances'. This is a lot lot lot bigger than an LLM's output space * (3.23 x 10 ^ 23) x (input scaling factor of 1.56 x 10 ^ 8) x (output scaling factor of a lot lot lot) = conservatively, 3.4 x 10 ^ 44 * Current (September 2023) estimate of global compute capacity is 3.98 x 10 ^ 21 FLOPS. So if every microprocessor on earth were devoted to training AIOS, it would take about 10 ^ 23 seconds = about 30000000000000000 years. Too long, I suspect. I'm fully willing to have any of this, and the original post's argument, laughed out of court given sufficient evidence. I'm not particularly attached to it, but haven't yet been convinced it's wrong.

Is a photographer "not an artist" because the photos are actually created by the camera?

This can be dispensed with via Chalmers' and Clarke's Extended Mind thesis. Just as a violinist's violin becomes the distal end of their extended mind, so with brush and painter, and so with camera and photographer.

As long as AI remains a tool and does not start to generate art on its own, there will be a difference between someone who spends a lot of time carefully crafting prompts and a random bozo who just types "draw me a masterpiece"

I'm not as optimistic as you abo... (read more)

Agreed. However,

  1. Do you think those IRL art forms will always be immune to generative capture and the threshold? 
  2. Even if the analogue original (a sculpture, a particular live dance performance, a particular live theatre performance) remains immune, most people will consume it through digital reproduction (photographs, VR, AR, video, audio) for which the threshold does apply.
1Nate Showell
1. Sculpture wouldn't be immune if robots get good enough, but live dance and theater still would be. I don't expect humanoid robots to ever become completely indistinguishable from biological humans. 2. I agree, since dance and theater are already so frequently experienced in video form.

Oh I see! So that's just the default slideshow range, padded front or back with zeroes, and you can enter much longer locations manually?

I like this.

It feels related to the assertion that DNNs can only interpolate between training data points, never extrapolate beyond them. (Technically they can extrapolate, but the results are hilarious/nonsensical/bad in proportion to how far beyond their training distribution they try to go.)

Here's how I see your argument 'formalised' in terms of the two spaces (total combinatorial phase space and a post-threshold GenAI's latent space over the same output length), please correct anything you think I've got wrong:

A model can only be trained on what alread... (read more)

3the gears to ascension
Oh indeed, like I said, I don't think this really undoes your main point. I think there's something going on where a probability distribution of the future can't ever be perfectly tight, and being as brains and social systems and the weather - three of the highest-impact interacting components of earth's combined dynamical system - are all chaotic, it is effectively guaranteed that any finitely intelligent predictor of the future will be unable to perfectly constrain its predictions. (it can do something, and it can massively exceed us in terms of our sit-in-a-room-and-think prediction capability; but it can never be a perfect copy of the future ahead of time. I think. maybe. ask MIRI.) so assuming that there's no galaxy brain shortcut that lets you exactly predict the future, even having a perfectly calibrated distribution, your distribution is not as narrow as the timeline you end up on. and if superdeterminism is false (which is the current default expectation of physicists), quantum also is guaranteed to surprise you. you accumulate more information about what timeline you're in and not continuing to observe it makes you unaware of goings on.

Cute! But how does each 16-digit 'image location' value (of 10^16 in total) uniquely represent one of the 4096^266240 possible images?

1Shankar Sivarajan
It's not limited to 16 digits.

Very interesting article. Most of my objections have been covered by previous commentators, except:

1a. Implicit in the usual definition of the word 'simulation' is approximation, or 'data compression' as Michaël Trazzi characterises it. It doesn't seem fair to claim that a real system and its simulation are identical but for the absence of consciousness in the latter, if the latter is only an approximation. A weather forecasting algorithm, no matter how sophisticated and accurate, will never be as accurate as waiting to see what the real weather does, beca... (read more)