This happened to me while I was digging down in my little village’s mine. I built a ladder so that my guys could get back up to the surface, but I didn’t realize until it was too late that it was one block above the “floor” of that chunk. So when my miners dug that final block layer, they are unable to use ladders. I then tried to make another ladder that’s one block high below it so that I can get my guys up, but when I did that, no one would build the ladder and trying to remove the task or the semi-built ladder ended up with no one removing it and being unable to do anything with the ladder. Additionally, when I did make a second ladder going down to the floor of the mine, any idle citizens would run over there and perpetually try to build this ladder up, which no only never seems to complete for some reason, but wastes my wood resources as they still use it up. The third time I tried to make a ladder I finally succeeded in making a proper ladder.
I’m not using a modded version of the game, it’s straight vanilla. I haven’t tried to reproduce this bug just yet as I cannot exactly figure out how I did it in the first place, but I will try to update this post if I ever figure out how to reproduce it, if it is possible. I’m running Windows 10.