I bought a second-hand house and signed a deposit receipt with the owner and the intermediary. I don't want to buy a house now Should I fine the agent?

This signed agreement is legally binding and requires a fine. But you can also consult a lawyer and see if you can find an outlet from the agency fee regulations. See if the contents stipulated in it comply with the law. If there is something wrong with the content of the signed agreement, you can apply for the agreement to be invalid. Naturally, there is no need for compensation.