I’ve noticed that onced a building is built you can’t get rid of it. Sure you can cancel a building command but it’s a little fiddly. So, in the harvest section or building menu you should include a bulldoze option.
Shameless plug here but you should go grab my mod, Structures+, over here: [MOD] Structures+ v0.4.5
It adds tons of building-related features one of which is a button for tearing buildings down