Do not spend your life trying to fix a regret. After you have given your heart to God, you have given Him the place to speak to you, if you were to get out of line with His will. The things that have happened that have caused you guilt or regret are not what you try and go back to fix. Jesus came so you can give Him those burdens and move on to do the best you can from that day forward. When you spend your life trying to fix a regret or take back a sin, the devil knows you do not believe in the God you say you trust in and he will have a field day with you.
Before we come to know the Lord we can be sure that our life will be a mess. Jesus came to carry that burden so we can go forward and do right. The time we spend trying to fix a wrong is when the devil knows he has another victim to deceive. When all our time is spent on trying to make right on a sin, it just messes up the rest of what is in front of us.
First thing is; accepting Christ.
Second thing is; do not go back and fix what you regret. Give that to Him and prove Him to yourself.