The hearthlings seem to have forgot about the scaffolding the left in the wall…
Build a ladder up to the lower roof. Works for myself. If that ladder doesn’t do the trick add along each 1 step up on that lower roof up against the wall.
Had to do something like that with my bridge’s pillars…
Other than that only the destroy command will work. Hopefully they will get better at taking this sort down in the future improvements coming. 
‘CTRL + C’ opens the Command Console. Select the scaffold and then type ‘destroy’ and hit ‘ENTER’
They’re working on the building system now, so we’ll have a better system soon! Unfortunately until A15 comes, the above stated solutions are the only fixes. Cheers.
1 Like
Ohh I’ve had this issue. They haven’t forgotten, they’ve removed the ladder to fast and can’t get to the parts on the roof, If you give them a ladder that allows them to get onto the roof they should remove it all.
Hope that helped 
There was thread that this would suite, But I can’t find it, maybe an mod knows