I mean, not only is the "p-value" threshold arbitrary, not only are we depriving ourselves of valuable information by "accepting" or "not accepting" a hypothesis rather than quantifying our certainty level, but...what about P(E|H)?? (Not to mention P(H).)

Well, P(E|H) is actually pretty easy to calculate under a frequentist framework. That's the basis of power analysis, a topic covered in any good intro stat course. The real missing ingredient, as you point out, is P(H).

I'm not fully fluent in Bayesian statistics, so while I'm on the topic I have a question: do Bayesian methods involve any decision making? In other words, once we've calculated P(H|E), do we just leave it at that? No criteria to decide on, just revising of probabilities?

This is my current understanding, but it just seems so contrary to everyday human reasoning. What we would really like to say at the end of the day (or, rather, research program) is something like "Aha! Given the accumulated evidence, we can now cease replication. Hypothesis X must be true." Being humans, we want to make a decision. But decision making necessarily involves the ultimately arbitrary choice of where to set the criterion. Is this anti-Bayesian?