I’ve noticed that mining zones are broken down into 4x4 ‘deep blocks’ and 1x4 ‘surface blocks’. Mining zones can thus be 4x4x4 in size or 5x4x4, depending on if you click on a top surface or on a wall side.
When mining out cliff areas, I remove 5x4x4 chunks but often find the villagers stopping the task and wandering off. Peripheral parts of the ‘surface’ blocks end up not being mined and float around until I build a ladder for them to be reached. The mining zone remains highlighted and other parts of that zone are ignored:
Here, you can see a ladder up front that I had placed to remove a floating cube. The one behind it needed another ladder to be mined. The left mining zone is 3/4 finished, but not progressing.
I think this is related to the villagers prioritising mining in the center of the mining zone, to not get stuck. I love this feature as it makes pit-mining easy, helping to prevent villagers mining themselves into a pit.
My suggestion is that the villagers can mine 5 blocks high. It might seem like a bit of a reach, but I don’t think it would make gameplay ‘cheap’ in any way. It would also not mess up the pit-mining feature by removing the steps prematurely.