Yes, I’m aware of that, I tried to find a better proof but failed. Attempts based on trying to compute the maximum possible change (instead of figuring out how to get a desired change) are doomed. Changing the last bit isn’t an infinitesimal change, so using calculus to compute the maximum derivative won’t work. EfficientNets use swish activations, not ReLUs, which aren’t locally linear, so we will have to deal with the chaotic dynamics that show up whenever non-linear functions are iteratively applied. The sigmoid inside the swish does eventually saturate...
That might actually be easy to prove with some effort (or it might not), consider this strategy:
Let’s assume that the input to the system are PNG images with 8-bit values between 0 and 255, that are converted into floating-point tensors before entering the net, and that the bits you are talking about are those of the original images. And lets also assume that the upscaling from the small MNIST images to the input of the net is such that each float of the tensor corresponds to exactly one value in the original image (that is, there is no interpolation). And...
Since this comment is being upvoted, I have to ask, how would being autistic affect your decision-making in that situation?
I think (incorrectly?) that everyone, except maybe children and drunk people, would remain quiet, and would either get angry or not depending on what they care about and models of the situation that vary from person to person.
I mean, think of everything that would need to go wrong in order to scream “The emperor is naked!”:
They would need to be certain about what is going through the emperor’s mind. It seems more likely that the empero... (read more)