Joke: What does the third sentence in treat mean?

Why hasn't the person who should have come yet? If in doubt, please ask me.

"treat" joke:

In the old years, someone held a banquet at home to entertain those who had helped him, and one invited four guests. It's almost noon, and one person hasn't arrived. So he said to himself, "Why hasn't the right person come yet?" One of the guests listened and thought, "I shouldn't have come before the one who should have come?" So I got up and left. The man regretted his mistake and said, "I shouldn't leave again." Another guest thought, "I shouldn't leave." It seems that I should have left long ago! " , also leave. The host was very sorry to see that he had angered the guests by his careless words. His wife also complained that he couldn't speak, so she argued, "I'm not talking about them." The last guest listened and thought, "Not them! That's just me! " Then he sighed and left.