Although these are both fire metaphors, they are not quite the same thing.
To "go up in smoke" means to disappear, dissipate, fail, or be destroyed as if by burning. A similar idiom would be to be "all for naught".
To "go up in flames" similarly means to burn, but usually more spectacularly, as if in a conflagration or other disaster. A similar idiom would be to "go down in flames" as in "crash and burn".
The first implies wasted or futile effort. The second implies spectacular failure.
My lifelong dream to become a doctor went up in smoke when I found out I would faint at the sight of blood.
My long-term plan to become a doctor went up in flames when, as a college prank, my friends and I stole the yacht of what turned out to be the director of the medical school.
Because there is some overlap in these definitions, in many situations you can use either.