It is not currently possible to prevent Hearthlings from building ladders. As they are used for many different tasks, including building, mining, and general terrain navigation, they are not included under any task group in the Citizens menu.
It’s quite hard to determine how important a ladder is, after all it might be needed to free other hearthlings out of a hole they dug and can’t get out of by themselves.
It shouldn’t, and we already have a separate topic about issues with Hearthling priorities. I was simply stating that it is impossible to prevent a Hearthling from building a ladder via the citizens menu.