They can see that you have a light, and by the time they've finished with you, your light will be diminished and you will be damaged because you've wasted your time with them. You didn't use your discernment.
You've followed that person for too long, and you've neglected yourself, and that's not good. Remember, when God wants to bless you he will bring a good person into your life that will be an asset and a blessing to you.