The man confessed that he was rejected by SMS.

It's not that you are terrible, but that he is too inferior. Every time I see you, I think of his rejection. For some people, this is a failure to some extent. Every time I see you, it reminds him of this frustration, so he doesn't want to see you. His love for you is not equal to his "self-esteem".

Or, he likes you too much, but he can't get it, so he stays away from you, so as not to be embarrassed and sad.

In either case, it is his problem, not yours. Unless you refuse him in a strange way and cause him special harm.