If you try and build two building within one square of each other, the scaffolding will become inaccessible during the building process.
To fix the issue, I had to build a ladder inside the building so the workers could keep going.
I’m not entirely sure this is a bug, but its kind of a grievance.
Thanks!