I think I've figured this out. I still can't implement it, since I don't write PHP, but I'm going to drop it here anyway, to get a sanity check and in case seeing it written out will be useful for anyone else. (Hint, hint.)

Edit: On further examination, I think my conclusion is wrong... or rather, the first set of numbers I got was right, and just not useful for this purpose.

I started with a set of sample data: A horoscope with 2 votes for 'not useful' and 3 votes each for 'sort-of useful' and 'useful', totaling 8 votes.

I then used the wolfram alpha Wilson score interval calculator to get a pair of numbers for each of the 5 vote types, given 95% confidence:

• 'Harmful' and 'awesome' have a lower bound of 0 and an upper bound of .324

• 'Not useful' has a lower bound of .071 and an upper bound of .590

• 'Sort of useful' and 'useful' have a lower bound of .137 and an upper bound of .694

I messed around with using those numbers directly to get weighted scores for the horoscopes, but they didn't work very well that way, so I adjusted each set to add up to 100%. For example, on the lower bound set, 'not useful' has (0.071)/(0.071+0.137+0.137)=0.206 of the adjusted votes. Multiplying these adjusted numbers by the weighting that I gave in the original post (-15, -1, +1, +3, +10) gave what look to me like sane numbers: My sample data got an adjusted lower bound score of 1.382 and an adjusted upper bound score of 0.216. (The adjusted upper bound score is low because 'harmful' votes are weighted more strongly than 'awesome' votes, and the upper bounds for those are closer to the upper bounds of the other types of votes - in other words, the 'upper bound' vote is more pessimistic because of how things are weighted.)

I could easily have done something wrong, there, but assuming not and assuming that it's easy to code a Wilson score interval calculator, this should be simple enough to add to the code once someone gets to it. (Peer appears to have become busy with something else.) The decision on whether to use upper or lower bounds seems to depend on how we want newer horoscopes to act in comparison to older ones, and I think using the lower bound (optimistic) one makes sense - I think new horoscopes should ideally be given a few opportunities to prove themselves at a higher rate of visibility before settling into their accurate place in the hierarchy, rather than having to claw their way up from enforced obscurity.

Numbers:

actual votes

0 harmful
2 useless
3 s/o useful
3 useful
0 awesome

score 1.25

-

lower bound 95% confidence

0.000 harmful (*-15 = -0.000)
0.071 useless (*-1 = -0.071)
0.137 s/o useful (*1 = 0.137)
0.137 useful (*3 = 0.411)
0.000 awesome (*10 = 0.000)

score (total weighted score divided by sum of Wilson numbers): 0.48475

-

upper bound 95% confidence

0.324 harmful (*-15 = -4.860)
0.590 useless (*-1 = -0.590)
0.694 s/o useful (*1 = 0.694)
0.694 useful (*3 = 2.082)
0.324 awesome (*10 = 3.240)

score: 0.2155

-

adjusted lower bound of 95% confidence (keep wilson numbers proportional to each other but make 'em total 1.0)

0.000 harmful (*-15 = -0.000)
0.206 useless (*-1 = -0.206)
0.397 s/o useful (*1 = 0.397)
0.397 useful (*3 = 1.191)
0.000 awesome (*10 = 0.000)

score: 1.382

-

adjusted upper bound 95% confidence

0.123 harmful (*-15 = -1.845)
0.225 useless (*-1 = -0.225)
0.264 s/o useful (*1 = 0.264)
0.264 useful (*3 = 0.792)
0.123 awesome (*10 = 1.230)

score: 0.216