What would, then?

Actually not being confused. This isn't a question of authority, at all.

If every English speaker on this planet only normally used “edible” to mean ‘fit to be eaten’, would they be all wrong?

There are a number of issues to untangle here.

First, "fit to be eaten" is not actually very different from "able to be eaten". The meaning of "able" depends on context. In normal life, one describes something as "able to be eaten" if it is "fit to be eaten". But this may not apply in all contexts. So, the meaning of "able" is not fixed. Therefore, neither is the meaning of "-able/-ible".

Secondly, as I discussed in the comments above regarding denotation and connotation, a word can have patterns of being applied that do not affect its inherent meaning. So, even if nobody on the planet bothered to utter the following sentence:

Monsieur Mangetout demonstrated that many more things are edible than previously believed.

that does not, in itself, make the sentence false. (In fact, the sentence, in its own context, is true -- it's just that the meaning of "able" implicit in the word "edible" is not the ordinary one. For ordinary purposes, metals etc. are not "able to be eaten". But technically, in extreme contexts, they may be.)

“Forgivable” normally means ‘easy to forgive’, not ‘which could be forgiven, at least in principle’

Disagree entirely. If I say something is "unforgivable", I mean it cannot be forgiven, not merely that forgiveness would be difficult.