There's a lot of specialist nautical terminology here. I'm not an expert on this, but I've done a bit of checking with people who know more, and I'm reasonably confident.
For a sailing ship to be carrying an environmental condition - sea state, weather, etc - is a term I'm not terribly familiar with, but I gather it just means to be experiencing that condition.
A high sea is referring to the sea state, how rough the sea is.
Being badly pooped is to have a lot of water breaking over the poop deck, or the rearmost, elevated deck on a sailing ship. Even relatively small sailing ships would have this, even if they have no real forecastle to speak of, because it shields the quarterdeck in front of it, where the helmsman (or coxswain) and captain (if they aren't the same person) are, directing the ship.
Now, I tried to find where the dictionaries online might have gotten this quote, and discovered that they missed out a bit of the middle of it that might make it slightly easier to understand, though it's still obscure to the layman:
"As it grew dark I drew away, and headed up for Plymouth. Off Rame Head, carrying a high sea on the quarter, we were badly pooped" (source: Lonely Road By Nevil Shute)
That makes it more clear what carrying a high sea is meant to mean; on the quarter refers to a direction, between abeam (to one side) and astern. Thus the sea state, the high sea, is being driven from a diagonal direction to one side of and behind the ship. Because of that, there was a lot of water breaking onto the poop deck.