Pretty people don’t actually have it all when it comes to sex.

Ah, the beautiful people, with their killer bodies and smilies, universal love from everyone, better sex… right?

The advantages that conventionally attractive people get is called “beauty privilege.” But there are ways they’re at a disadvantage when it comes to sex. Here are five reasons why being attractive could work negatively in their favor.

1. Beauty is intimidating.

How many times have you passed someone on the street that took your breath away with their impeccable bone structure? You probably thought, “Damn – they’re way out of my league.” Chances are, a lot of people think like you. Even Rihanna complained about “not getting booty calls” (hence her heavy Twitter presence). Of course, her superstar schedule keeps her busy, but insecure men are bound to be scared off by Rih’s “Sexiest Woman Alive” title.

The conventionally average-looking among us tend to be intimidated by the super conventionally attractive. What does that mean? Less sex for the beautiful. So take heart — next time you see a babe in public, spit some game. They might not be getting as much attention as it seems.

2. They might not always get honest feedback on their skills.

Being beautiful can translate to not having to try as hard. Continuous studies have proven that humans perceive those with “beauty privilege” as friendlier, smarter and just better. This carries over into many realms, sex included, If you’ve had the pleasure of sleeping with someone beautiful, perhaps they rocked your world. Or maybe they were absolutely unremarkable.

I remember a college acquaintance who hooked up with a campus heartthrob. The verdict? He just “laid there” and she did all the “work.” Is it really his fault, though? Beautiful people may live in a bubble of privilege that lets them not need to do as much work in life, so why would they in bed? Good news for them, not-so-good news for everyone else.

3. Pretty on the outside does not guarantee pretty on the inside.

I’ve met some very lovely beautiful people and, unfortunately, beautiful people that turned ugly once they said more than 5 words to me. The pretty-but-mean stereotype has a firm place in our hearts. Hello, Regina George? Blair Waldorf?

Sure, these are the pretty people that make a bad name for the rest, but there’s also science on our side: a 2011 Spanish study suggested that people with symmetrical faces (a common marker of conventional attractiveness) are more self-sufficient and have less incentive to co-operate with others. And man, if there’s anything that takes cooperation it’s sex. Good sex, at least.

4. They get used, too.

We all have feelings and no one likes when they’re hurt. No one likes being used. Those are human universals. But beauty is currency in our society. Being attractive can put you at higher risk for meaningless hookups and jerks who spend a night cooing sweet things in your ear, only to ghost on you the next day.

Too many folks just want a pretty notch on their belt and don’t care about the emotions beyond someone’s nice face and body. Which creates a fertile breeding ground for trust issues, a whole ‘nother can of worms to contend with.

5. Good looks does not always equal good self-esteem.

I know a number of conventionally beautiful women who do not see themselves as such. They see their barely-noticeable cellulite and the damn-near-invisible whiteheads on their face. From them, I learned that good looks do not always equal good self-esteem. And self-esteem is one of those things that becomes incredibly evident after you’re around someone for longer than half an hour.

Someone with good self-esteem knows how to enjoy their own body as much as their partner’s, and is more receptive to the give-and-take of sexual interaction. I’ve never slept with an Adonis with confidence issues, but I’m positive I’d rather choose an average-looking, self-assured man.

So the next time you see a dreamboat and think they have “beauty privilege,” well, think twice.

Beauty privilege might get you jobs, promotions, and the social ease that comes with everyone wanting to get into your pants — but that doesn’t always translate to an amazing sex life. For all you know they’re enduring a year-long drought. But with some confidence and empathy, you can be the one to end it for them.

For more sex tips, check out this fairly unconventional suggestion on getting the best sex of your life.