Hi,Sarah
The answer is no.
This is a classic comment that strongly relates to bullying.
Robinredbreast is on the right track with regard to discrimination.
Without knowing the facts,it would not be easy to think what is best to do.
Sometimes it is recommended to ask the person who made the comment, to fully explain how this is the case and put it in writing for you to seek professional advice.
Because you said "always" also indicates a problem.
If you do need advice if it is in the workplace. ACAS are the people that will help.this is their contact: Call our Helpline on 08457 47 47 47 for free and confidential advice.
Best of luck
Roy