it isn't entirely clear what it would look like to have a proven track record in the field.

They want to teach people how to be rational, professionally successful, and altruistic, hence it would be desirable if the staff had strong credentials in that areas, such as being successful scientists, inventors, entrepreneurs, having done something that unquestionably helped many other people, etc.

especially given that many of their staff members have impressive accomplishments in other domains.

Such as?

According to the OP, CFAR has five full time employees. I suppose they are the first five people listed in the website (Galef, Salamon, Smith, Critch and Amodei).
Galef is a blogger and podcaster, Amodei was a theatre stage manager, the others are mathematicians:
Critch is the only PhD of them and has done some research in abstract computer science and applied math. I don't have the expertise to evaluate his work, does it count as an impressive accomplishment?
Salamon mostly worked at SIAI/SI/MIRI and didn't publish much outside MIRI own venues and philosophical conferences.
Smith, I don't know because I cant find much information online.

EDIT:

Actually, according to the profile, Smith has a PhD in math education.

it isn't entirely clear what it would look like to have a proven track record in the field.

Having a track record of creating something else that's unambiguously useful would be a start.

Mostly, people attempt to do grand and exceptional things either due to having evidence (prior high performance, for example), or due to having delusions of grandeur (prior history of such delusions). Those are two very distinct categories.