Will poor study be regarded as a joke?

From your description, I guess you are probably an adult. I think you must change your mind when dealing with this problem. There is no need to tell others how you are doing, and how you are doing has nothing to do with them, so they can say whatever they like, and you don't need to tell them about your recent development, such as your self-study. You can learn secretly by yourself and not let them know. If they dare to hit you, it's all envy and jealousy. I sincerely wish you an early exit from the predicament.