I stand corrected.

They weren't removed from whiteness, they were dehumanized, which has a different meaning.

(I actually should have known this because I have read up on Medieval anti-Semitism.)