Your question is associating two things. A belief about the world, and guilt about behaviour.
Guilt is about hurt we are causing to others, and frameworks of behaviour which tell us this behaviour is wrong. Guilt is an emotion, related to ones conscience and things we hold dear to ourselves.
From the question it appears the desire to not have guilt about some behaviour, which is something ones feels, but rationally it is felt the behaviour should not create guilt.
I saw a story of a man who travelled the world and had a good relationship with his wife, which was his emotional anchor. While travelling he fell into relationships with women along the way. This finally resulted in divorce, but he had never resolved his need for his wife and his disregard of how she would feel about his affairs. After his divorce he found he was changed and he started to need reassurance in a way he did not before. It is these foundational feelings and links we build up and do not recognise, which when hurt, cause us harm, which are not based on beliefs but on who we are and how we belong.
It takes time to work these things through, and learn who we are and how we are put together. Anyone who starts to work this through, will then be able to answer the why of the feeling of guilt and how to work out news ways and find resolutions, rather than dismissing something that obviously still matters to them.