Placing the lights was getting a bit buggy in this savegame, the ladders were not torn down after placing the lights on the house. This was not a template loaded house, it was custom built with the lights placed after. This is why you have multiple saves, so you can rescue your town!
I tried to build a ladder in hopes that when I removed that it would also remove the ladder built for the light, but the ladder itself became unremovable.