How harmful is prejudice?

Stanford Prof. Ben Barres, who is supposed to be an expert on sex prejudice because he is a transexual, writes:

I am simply saying that to tell young people that they are innately inferior is deeply harmful. It is presently scientifically impossible to sort out with any degree of certainty the effects of social forces and prejudice, which are more than amply demonstrated to be large, from any possible innate effects. Therefore any faculty member who pronounces in a classroom that a whole group of people is wired to be inferior is causing great harm without having strong evidence to back his contention.

If it is really scientifically impossible to figure out the effect of prejudice, then how does he know that it is so harmful to tell people that they are inferior?

This may sound silly, but when a prof. says that something is deeply harmful, then he should be expected to quantify that harm. In some cases, it is beneficial to tell someone that he is inferior. If telling someone that he lacks the talent to become a professional athlete, then he might redirect his energies into some more productive goal.