I implemented the following circuit, a very simple circuit:
I applied the technique presented on Farhi's paper - which I am so happy people in here are talking about more and more ! - and applied to this circuit:
For the most part the SGD (either vanilla or Adam), it works well. However, when $\alpha$ is such that the loss value for the circuit is almost 1, the optimization gets stuck and is unable to move - as if it is on a saddle point (I think it is but I am not sure as of yet).
My question is: is this what McClean's paper refers to a barren plateaus or is it like a mistake due to SGD implementation ?
Here it is the evolution of the loss function as a function of phase parameter:
On further consideration: I want to add that there is some noticeable difference between ADAM and VANILLA SGD on these situations; namely Vanilla does manage to converge.