Tarantino's character (Jimmy) is a dick. Jules does react to it IIRC, though subtly : he's clearly annoyed by Jimmy, but holds it in because he knows it's a... well, sticky situation, and he's a professional, unlike Vincent.
Marvin's race never seemed particularly relevant to the scene to me either, but what do I know.