Presumably you can't ask the oracle about the halting behavior of programs which call the oracle. Otherwise the standard proof that the halting problem is undecidable can be used to show that the oracle is fake: just have a program ask the oracle about itself, then do the opposite.
Ah, I didn't realize that. From looking over my program again, this appears to mostly be a slightly longer algorithm that does more or less what Thomblake said earlier. I didn't realize that was what I would end up with when I started trying to make one, but I guess it's a standard way of looking at it.
Here's something I've been wondering about, in the context of Solomonoff induction and uncomputable sequences.
I have a device that is either a halting oracle, or an ordinary Turing machine which gives the correct answer to the halting problem for all programs smaller than some finite length N but always outputs "does not halt" when asked to evaluate programs larger than N. If you don't know what N is and you don't have infinite time, is there a way to tell the difference between the actual halting oracle (which gives correct answers for all possible programs) and a "fake" halting oracle which starts giving wrong answers for some N that just happens to be larger than any program that you've tested so far?
The Kolmogorov complexity of an uncomputable sequence is infinite, so Solomonoff induction assigns it a probability of zero, but there's always a computable number with less than epsilon error, so would this ever actually matter?