Xiao Huang, let's talk.

As you can see from your description, this boy can tell you Duan, and you accepted and replied to him, which shows that he has great feelings for you. You can also think that your relationship is very delicate (without this feeling, he wouldn't say such a large-scale thing to you). Besides, when he said you were not a man, he just wanted to have a further chat with you. From a linguistic point of view, this slightly provocative question is more willing.

I'm sure this boy is not malicious, he just wants to have more chat time with you and talk to you, not to mention that you all know that he likes you. If you have feelings for him, you can choose to accept him; If you have no feelings for him, then as a girl who is liked, please give him a "happy and tragic ending"-decline.

My answer is for reference only.