Your understanding is correct.
In the theory of photon polarization, the parametrization of the Bloch sphere (or its surfave) has traditionally another name. On the wikipedia page for the Jones calculus (the parametrization of the Bloch sphere surface), you'll find a table for the correspondence between kets and polarizations.
To summarize, eigenstates of the Pauli matrix $\sigma_Z$ can be seen as the horizonal and vertical polariations (commonly written as $|H\rangle$ and $|V\rangle$),
eigenstates of $\sigma_X$ correspond to diagonal and anti-diagonal polarizations, and eigenstates of $\sigma_Y$ correspond to the left and right-handed polarizations of light.
Thus a measurement corresponding to $\sigma_X$ can be seen as putting a beamsplitter in front of the light path, that splits the light into the horizontal and vertical polarization components. (You can also put a polarization filter, but as such filter measurements are destructive, the post-measurement will be different. And because of that, destructive measurements technically don't quite correspond to the Pauli's. Do it with beamsplitters instead of filters, and the analogy is fine again.)
Of course, you can always choose a different basis (by a unitary transformation). Just make sure that choice made gives the correct commutation relations between the Pauli matrices.
PS: A similar concept to the Jones vector, but which also covers the interior of the Bloch Ball, are the so-called Stokes parameters.