I agree that compatibility with traditional papers is important. It was not stated explicitly, but I do want the results to be publishable in traditional journals. I plan on publishing the results for my company's product. It seemed to me like being overly rigorous might be a selling point initially -- "sure we did the study cheap / didn't use a university, but look how insanely rigorous we were"

The thing is this seems like an ab-initio approach to doing research by people who are not researchers by trade. The vast majority of tech startups are lead by engineers not researchers, though there is no visible line between the two.

Going after professional researchers seems much harder. They actually know how to perform the research, so the value proposition is much weaker -- they are already trusted, and know how to use R :p

By the principle of comparative advantage researchers should be willing to delegate some of their work to a third party, so look for the repetitive parts that could be automated by either protocol or program. If, for example, the journal requires a replication before the full study is published, the original researcher(s) might have an incentive to plan for a replication from another party.

My idea for you would be to follow the same line most other improvements on traditional procedures follow: Automate the parts that can be automated, standardise the parts that can be standardised and continue. Designing a whole system tends to fail from my reading of history.

A two-pronged approach might even be more favourable: Work with a traditional journal that has the "perfect" scientific standards so the requirements infect traditional science and meanwhile fill the journal with the papers generated from the program.

I'll have to think about this some more.