For the 2014 PGR philosophy of mind rankings, there are 42 evaluators. Of these, in a colloquial sense, 34 are white men, 5 white women, and 3 minority men. 0 African-Americans. 0 minority women.
Does it matter what the breakdown of the evaluators is? If so, how does it matter? Here are some options.
1) It doesn't matter in any way. The rankings are meant to track which departments are best for philosophy of mind, and all that matters is whether the rankings track the right content. The skin color, race or gender of the evaluators is irrelevant to that.
2) It doesn't matter philosophically, but it matters non-philosophically. The lack of diversity in the evaluators doesn't impugn the rankings, since the rankings only capture what is relevant to philosophical content. But sociologically speaking, it matters that mainly white males are in the position of judging where the best philosophy of mind is happening. To be able to evaluate is a privilege and it would be better if that privilege was more distributed.
3) It matters philosophically. The lack of diversity in the evaluators brings into question the philosophy itself that they are evaluating. It suggests that the conception of philosophy of mind the rankings is tracking is implictly, and unintentionally, reaffirming structures which make it hard for women and minorities to thrive in the philosophy profession. The philosophy of mind content assumed by the evaluators is not independent of problematic institutional structures, and the rankings' implication that it is tracking pure philosophical content independent of issues of race or gender is an illusion which perpetuates the problematic structures.
I believe (3). Understanding what it means and why it might be true requires more careful thought than can be done in one blog post. In order to elaborate on it and think it through more, on this blog I will keep coming back to this theme from time to time.