Only people who are worse than you will speak ill of you. People better than you don't have time to talk about you.

People who are better than you will not make irresponsible remarks.

Only people who are worse than you will gossip or be sarcastic. He is worried that the harder you work, the bigger the gap with him, so he doesn't want people around him to be better than him. This kind of person doesn't work hard, and he doesn't want others to work hard. He likes to watch other people's jokes.