I just got the amazing El Toro on my Swashbulker and the colors are off. My colors are light purple and white and his are black and red. I have been away from this game for a long time so I do not know if they added this?
When you first get El Toro, He sports his own colors (that being the black and red that you are referring to). Do not fret, my friend! When he hits level forty, he will promote (and a fun promotion it is indeed)! When you complete his promotion, he will wear your colors!