You have run into the "productivity paradox." This is the problem that, while it seems from first-hand observation that using computers would raise productivity, that rising productivity does not seem to show up in economy-wide statistics. It is something of a mystery. The Wikipedia page on the subject has an OK introduction to the problem.

I'd suggest that the key task is not measuring the productivity of the computers. The task is measuring the change in productivity of the researcher. For that, you must have a measure of research output. You'd probably need multiple proxies, since you can't evaluate it directly. For example, one proxy might be "words of published AI articles in peer-reviewed journals." A problem with this particular proxy is substitution, over long time periods, of self-publication (on the web) for journal publication.

A bigger problem is the quality problem. The quality of a good today is far better than the similar good of 30 years ago. But how much? There's no way to quantify it. Economists usually use some sense that "this year must be really close to last year, so we'll ignore it across small time frames." But that does not help for long time frames (unless you are looking only at the rate of change in productivity rates, such that the productivity rate itself gets swept aside by taking the first derivative, which works fine as long as quality is nor changing disproportionately to productivity). The problem seems much greater if you have to assess the quality of AI research. Perhaps you could construct some kind of complementary metric for each proxy you use, such as "citations in peer-reviewed journals" for each peer-reviewed article you used in the proxy noted above. And you would again have to address the effect of self-publication, this time on quality.

Levels of indirection. Every time a human out-sources a task to a computer it creates and uses an indirection. This is not new. It is the same as referring to books instead of knowing yourself. Later you just know whom to ask for the needed information. So my idea would be to try to measure levels of indirection. For example computers don't just do computation for computation about computation (see also Fundamental theorem of software engineering). Wikipedia is not just pages but pages referring to pages (see also Getting to Philosophy).

Note that this nicely differentiates many kinds of noise from useful information: "Random" repetitions and variants of the same cause a wide graph but no deep (though t is easy to artificially create arbitrarily deep graphs, how to solve this remains as an exercise for the reader).

A hypothesis regarding human knowledge creation could be that the number of levels of indirection added per unit of human research effort is roughly constant. And that the introduction of AI speeds this up.

Here is a simple model that might be useful. You make and sell good G at price g per unit. You make it using inputs X and Y with, say, a production function G=(X^a)(Y^b) where X is how much of one input you use and Y how much of the other. The cost of each unit of X is x, and the cost of each unit of Y is y.

So your goal is to pick X and Y to maximize (X^a)(Y^b)g-Xx-Yx. You want to know how xX/(xX+yY) changes as a and x change.

One important question is: how much do existing AI systems help with research and the development of new, more capable AI systems?

If you make it more of a productivity question like this, then you can ask how people actually do their work, and how many man hours it takes to do that work to get the same level of performance.

By that metric, things have improved a lot.

But if the metric is "effort to halve errors from existing levels", you don't see the same improvement.

Do you want improved algorithm effectiveness to count as "current AI systems"?

One important question is: how much do existing AI systems help with research and the development of new, more capable AI systems?

I think a lot. The increased computer power and system tools allow hyperparameter spaces to be searched automatically. A lot of brute force becomes practical, and the available applications support it. Also, consider the application support for import of data, and generation of new data.

So I'd like to be able to measure the share of AI R&D done by computers vs humans.

It's kind of like asking how much of your driving is done by the tires, and how much by the transmission. Your out of luck if you don't have both.

what we should expect progress to look like going forward.

If the goal is forecasting progress going forward, I'd recommend thinking of the problem in those terms and flowing from there instead of thinking of going directly to sub forecasts before you've clearly related the meaning of those sub forecasts to the main goal.

This isn't a new phenomena: the word 'computer' originally described a human occupation, perhaps the first major 'mental' occupation to be completely automated.

That started around WW2, so this general trend can be seen for the last 75 years or so. I'd look at the fraction of the economy going into computing, as how that has changed over time is due to the interplay between the various effects of automation.

My idea would be to search actual examples where neural nets were used to create new neural nets, or a program has been writing programs.

For example in chip industry computers are used to create new chips on all stages. And in some stages they became un-replacable by humans I guess. This stage is probably tracing of actual chip layout in silicon based in its electric scheme.

You may be interested in all software which automate the process of creating neural nets or simplify it. This software has price and economic effect.