Many things which are long and narrow are called fingers in English, from ladyfinger cookies/biscuits to the Finger Lakes. OALD includes a a subordinate meaning of finger as
finger (of something) a long narrow piece of bread, cake, land, etc.
a finger of toast
In British English, a similar food made from fish is know as fish fingers, though they are called fish sticks in North America.
Note that the term finger food is for a different and unrelated concept, however, and refers to food of any shape that is portioned into small pieces intended to be picked up and eaten with the fingers.