My husband’s brother-in-law often likes to invite us out to play, just the four of them and us. He feels very happy and in a good mood with me around.

In this case,

Don’t go out with him.

There is something wrong with his brother-in-law.

Don’t say this kind of thing to your husband and his sister.

When I ask you out again,

Just give me a reason.

I can't go anyway,

There's something going on between the family, let alone talk about it.