I think you are taking the term "self-interest" far too narrowly.
People may be motivated by all sorts of interests, and in rational agent theory, of which Smith is an ancestor, the "rational self-interest" may even extend to benevolent acts, if that is what the "self" pursues by rational means.
Smith's point was not that people always pursue or ought to pursue some narrow self-interest but, as in Mandeville's "Fable of the Bees," that the social sum of self-interested actions may have good outcomes overall. Thus the moral, traditional, or legal suppression of "gain" through self-interest was not necessarily justified on behalf of "king and country."
At a practical level, in any modern money economy and for the bourgeois of Smith's day, wealth will simply deteriorate if it is not invested. Even landed wealth. It was the new capitalist framework, the merchant traders, and the early "improving farmers," that made continual reinvestment a necessary component of wealth and, presumably, "self-interested" planning.
Moreover, since merchant capitalism entailed significant state involvement and collective investment in infrastructure and operating capital--docks, warehouses, ships, mail services, armies--the self-interested organization of
"public-private" works was already well-understood.