I used to be reflexively put off by infidelity in film. Now as I've become more supportive of the concept of monogamish, it's more contextual. Open Hearts more recently is a film about infidelity that really worked for me because it wasn't making infidelity into exotic fantasy (I'm looking at you Unfaithful). I've actually grown more annoyed not in the infidelity but films where the discovery of the infidelity must naturally lead to a way over the top emotional response. It's natural that there will be pain but I like films that treat it with a bit of a grown-up perspective that understands that infidelity is basically the norm in human relationships.
As to the interaction of gender and rape, I'm not sure it matters, it still comes down to the tone of the film, I don't like violence, sexual or otherwise, to be treated casually.