Vinyl sound is influenced by how music is carved into it (mastering), the playback mechanism (turntable, stylus, cartridge, electronics, etc), and the characteristics of the medium itself (vinyl).
The meaning of "vinyl sound" is often approached subjectively. Different people have different ideas of what a sound has to have and/or lack to be considered "vinyl sound". To answer this question objectively, we can take a look at measurable characteristics and dynamics of the medium.
When people refer to "vinyl sound", they are most likely referring to one or a combination of the characteristics listed bellow. The importance of each characteristic in the "vinyl sound" definition is the subjective part.
Low frequencies (bass): Low frequencies panned between the left and right channels can knock around the needle, so you'll find low frequencies to be monophonic. Even with centered low frequencies, the record can skip with excessive low end. Because of this, excessive low end is not a feature often found in vinyl.
High frequencies (treble): High frequencies and sibilance can cause distortion because the stylus cannot properly track them in the disc's grooves. Because of this, highs might need to be tamed down (reduced, compressed, flattened) for vinyl playback.
Wow and flutter: Wow and flutter are a change in frequency of an analog device and are the result of mechanical imperfections, with wow being a slower rate form of flutter. For LP records, the quality of the turntable will have a large effect on the level of wow and flutter. A good turntable will have wow and flutter values of less than 0.05%, which is the speed variation from the mean value.
Inner and outer portions sound different: The distance around the inside of a 12-inch record is about half the distance than around the outside, as the distance around each revolution decreases, the high frequencies become harder for a playback stylus to read. As a result, the inner tracks will sound duller than the outer tracks. The high frequencies simply can't be reproduced the same as if they were cut on the outside of the disc.
Surface noise: Pops, clicks, crackles. You'll find them even on high-end setups.
Dynamic range: Vinyl microgroove phonograph records typically yield 55-65 dB, though the first play of the higher-fidelity outer rings can achieve a dynamic range of 70 dB. For comparison, 16-bit PCM (CD quality) has a theoretical dynamic range of about 96 dB.
RIAA curve: The RIAA equalization curve allows engineers to overcome certain limitations and maximize the amount of full-frequency music they can get onto a record. The EQ curve is applied as the music is cut into the master lacquer. When a record is played back, the inverse of this EQ curve is applied by the phono preamp so that the listener hears the music as it was intended.
Because low frequencies require larger grooves and more space on a disc than high frequencies, the EQ curve gradually rolls off the bass by 6 dB per octave starting at 1 kHz so that by 20 Hz, the level has been reduced 20 dB. Without the RIAA bass cut, only about 5 minutes of low-frequency information could be stored per 12-inch side.
In addition, the RIAA curve boosts the frequencies above 1 kHz to increase the signal-to-noise ratio in the high end. The maximum increase is about 20 dB around 20 kHz.
Unless linear-phase filters are used, phase changes will be induced both times the curve is applied (once in reverse), so both the mastering engineer and your turntable are injecting their character into the sound. How audible the change is depends on the filters used, some high-end systems aim for as little change as possible.
The RIAA curve can affect the sound in other ways. Pete Lyman stated that
Unfortunately, all the high end we're now used to is aggravated by the RIAA curve. When they came up with this system, there wasn't much going on above 8 and 10 kHz.
Sources: 1 2 3 4 5 6