This website explains it quite nicely:
c.1720, abbreviation of of the clock (1640s), from Middle English of the clokke (late 14c.).
The usage of o' as a shortened way to say of is not uncommon (this is called an apocopic form, when the last syllable or consonant is left unpronounced). Many o' these are relatively rare in written form, but you might see them when an author was trying to capture colloquial elocution in a quote:
- "All I needed was the queen o' hearts for a straight flush!"
- "Yer startin' your new job tomorrah, Jim? Well, best o' luck to ya!"
For some reason, o'clock is one exception where the o' became not only common, but formal.
"colloquial elocution" is a fancy way of saying "how people say words informally"