Well, maybe the only wisdom to be had here is really that if you don't have much more than priors to go on, tough luck, nothing you can do, live with the uncertainty and hope for the best (because actively asking for evidence is too costly). It's likely that this this doesn't bother you as much as me because you're just better at reading social cues; however the hell one is supposed to learn that, especially if one is an introvert and experiences a consequent poverty of stimulus.

Although sometimes it's not even about observing clues. For example, one might know that it's likely that one will at some point behave in some way that the other person would view unfavorably; and you want to estimate how much you should invest in this relationship. Then the only relevant evidence you can get is how this person behaves in dealing with other people.