Just as all applications you install using Aptitude (
apt-get), it is installed somewhere in a
/usr subdirectory. However, where exactly doesn't matter because you can run those programs just by typing them in the command line.
Thus, you can launch them by typing in your Terminal either
bitcoind for the server daemon or
bitcoin-qt for the GUI Wallet. Also, most desktop manager automatically place a shortcut for all apps in their app menu; for Ubuntu Unity, you can just type "bitcoin" in your Dash to find Bitcoin-Qt GUI wallet app.
Just as most apps, it makes a folder in your home directory to save the data it uses. Both
bitcoin-qt save all the block chain data in a
~/.bitcoin/ folder. You wallet is thus saved in