meaning recursive self-improvement would be more difficult for a brain-like ANN than it might be for some other sort of AI?

It's not clear that there is any other route to AGI - all routes lead to "brain-like ANNs", regardless of what linguistic label we use (graphical models, etc).

General purpose RL - in ideal/optimal theoretical form - already implements recursive self-improvement in the ideal way. If you have an ideal/optimal general RL system running, then there are no remaining insights you could possibly have which could further improve its own learning ability.

The evidence is accumulating that general Bayesian RL can be efficiently approximated, that real brains implement something like this, and that very powerful general purpose AI/AGI can be built on the same principles.

Now, I do realize that by "recursive self-improvement" you probably mean a human level AGI consciously improving its own 'software design', using slow rule based/logic thinking of the type suitable for linguistic communication. But there is no reason to suspect that the optimal computational form of self-improvement should actually be subject to those constraints.

The other, perhaps more charitable view of "recursive self-improvement" is the more general idea of the point in time where AGI engineers/researchers takeover most of the future AGI engineering/research work. Coming up with new learning algorithms will probably be only a small part of the improvement work at that point. Implementations however can always be improved, and there is essentially an infinite space of better hardware designs. Coming up with new model architectures and training environments will also have scope for improvement.

Also, it doesn't really appear to matter much how many modules the AGI has, because improvement doesn't rely much on human insights into how each module works. Even with zero new 'theoerical' insights, you can just run the AGI on better hardware and it will be able to think faster or split into more copies. Either way, it will be able to speed up the rate at which it soaks up knowledge and automatically rewires itself (self-improves).