any advice

This could be based on something real. Or it could be something you are associating with your father that he may not be responsible for.

I'm no expert but I think this sounds like a problem that is really too complex to be solved without some competent professional help.

I agree with all the previous advice. I think that Psycho-dynamic counselling would help. It sounds as if there is somthing suppressed that needs to be explored with expert help.