Why is it fine for royals to take lovers but their common husbands and wives are to remain faithful?

I don't understand why it was suitable for Charles to always have Camilla yet such a stink was made about Diana wanting companionship. Camilla's husband must have been fine with the arrangement between her and Charles, also. Is it a monarchy thing?