An SPV client uses the headers along with some other data to make sure that the Bitcoin it has been told it has received actually exist for the rest of the network. Absent this assurance, it would be trivial for somebody to simply tell non-validating wallets that they received money when they really didn't.
To prove funds exist without downloading the entire chain, the SPV wallet is supplied with a blocks merkle tree as proof of payment. They can validate that the merkle tree contains a transaction output paying them, that it is from a block with valid proof of work, and that the block header comes from a difficult to produce chain that leads back to the genesis block. Due to the structure of the merkle tree, it can be pruned to contain only transactions which are relevant to the client without changing the hash that was included in a block header, which has significant space and time savings over just sending the whole thing.
The SPV model is quite weak, however.
Actually finding out which outputs are yours delegates this work to a third party who may not have your privacy at heart, currently in most clients this uses bloom filters which broadcast your interest in wallet information across the network in a very identifying way.
Not being able to do full validations of blocks means that you could potentially accept payment in a block that the network will not accept because it is invalid in a place you can't validate (spends outputs that don't exist, for example). There is a trust in miners for this security model, and relies on them doing the validation of transactions for the system to be of sufficient security. This is usually supported by waiting for multiple confirmations on transactions, making an assumption about how much a malicious miner will be willing to throw away on an invalid chain in order to dupe SPV clients.
It has been shown in the 5th of July 2015 forking event that a significant portion of miners do no validation at all of block contents in some situations. In this particular event SPV clients could have seen 6 confirmations (the generally accepted "safe" amount) on transactions which were actually invalited. In this case all SPV wallets and most block explorers (and the wallet services which use their API) were at risk of losses if they accepted money in transactions which were later found to be double spends in the main chaihn.