Right now the villagers need scaffolding if you put something (lantern) 3 voxels of the ground.
I suggest that they reach 3 high and 3 (and maybe 2 deep) without scaffolding (or floor).
This would make scaffolding actually make sense instead since most of the time it actually looks like the will reach.
Three voxels high is right in front of their heads.
I also suggest that they LOCK these voxels when for example mining. So that you don’t have 3 miners standing “inside” of each other, each mining just one voxel. Instead they would space out ever 3rd voxel and chip a way for a while before moving.