And he shall break down the house, its stones and timber and all the plaster of the house, and he shall carry them out of the city to an unclean place.
Leviticus 14:45

Curiously, the laws of a house with tzaraat don't apply outside cities and villages. There is no need to tear down an infected farmhouse where there are no other nearby houses that might be infected. This is further evidence that these commands concern things more significant than mere physical structures. Think of the individual stones as members of your community. If he can be healed, fantastic! If he cannot and his spiritual disease begins to spread, then he and everyone he has infected must be removed before the entire community is destroyed.