Bitcoins are no different from any other commodity in this regard. We say that the price of a car is $45,000, but that doesn't guarantee that you can sell one for $45,000 unless you can find someone willing to buy one for $45,000. When we say some particular number is the price, we mean that's basically the number that buyers and sellers both agree that it is worth. But it's not a guarantee that you'll be able to buy or sell at that price.
Bitcoin exchanges report the prices at which Bitcoins change hands when they do change hands. They also report the price at which people are willing to buy (typically slightly less) and the price at which people are willing to sell (typically slightly more). If you try to buy or sell a large quantity, this will change the price. For example, if the last trade was at $250 and someone is willing to buy 10 bitcoins for $240, and you sell them 10 bitcoins, the price would move to $240. And now there wouldn't be anyone left willing to buy at $240, so the new buy price might drop to $235.
I'm not sure why you would connect this to a Ponzi scheme. Ponzis don't work this way at all -- they pretend to be investing but actually a central operator pays out of a fixed pool. Bitcoins behave more like stock, they are bought and sold at some agreed value based on the value of the underlying assets. Bitcoins have underlying value because they can be securely exchanged, allowing them to function like a currency.
While some people buy Bitcoins hoping they can sell them at a higher price, other people buy them strictly as a means of exchange. They exchange the Bitcoins for goods and the recipient may exchange the Bitcoins back to some other currency. The Bitcoins provide value because they allow the exchange to take place in a neutral currency, with high security, and at high speed, something that might otherwise be impossible. This capability makes Bitcoins have value beyond speculation.
In an sense nobody sets the price and in a sense everybody does. Anyone who wants to buy a Bitcoin can set the price they're willing to pay however they please. Anyone who wants to sell a Bitcoin can set the price they're willing to take however they please. As a result of trades in a free market, discovering the value is not difficult. If the highest price buyers are willing to pay is $250, and the lowest price sellers are willing to take is $251, the fair value is right in-between. (Assuming a perfect market, which we don't quite have.)
This is how many markets, including stock markets, work.