I was guessing this was the cause as well, and hence why I hadn’t written it up as a bug report – I had put it down to “sub-optimal, but probably working exactly as intended.”
I don’t think it’s just race conditions in tasks causing this behaviour though; it’s a deeper issue with hearthlings carrying around items they (quite literally) have no business to be carrying around… or, er… carrying them around for? Both things, I guess – they finished their original job, but picked up more of the supply than they needed; or else they were interrupted and cancelled the job while the item was in their backpack; so they’re carrying around surplus resources for no real reason.
When they’re carrying something in their hands and their task is interrupted, they put it down. However, if they have something in their hands and something in their backpack, they won’t stop and unload their backpack. So, you get a lot of hearthlings who are in the middle of a hauling task when something happens (town alert, stopping to eat, attacked by a monster, etc.), and they’re stuck with whatever is in their backpack until some new task tells them to check for space… and they realise that they’re still carrying that item.
One possible solution is to have hearthlings remember their previous job, and go back to it if they’re interrupted. This would also prevent some of the race conditions you describe; since it would mean that tasks get completed more smoothly. However, that may be resource-intensive, since the job manager would have to remember twice as much info for each hearthlings’ task – their current, and their previous one.
An alternative is to add a “check inventory, and if you find anything, store it or drop it immediately” stage to the end of every task. It might seem inefficient to do it at every task; but really that extra step doesn’t add any significant workload to the task (either from the hearthling’s perspective or the game’s perspective), and it would mean there’s no possible situation where this issue could crop up in future. Even if something unexpected happens and the hearthling somehow manages to miss one “check inventory” step, the next job will trigger it again, even if that task is idling or healing or rallying at a banner… and if that new task gets cancelled too, the next task will provide the same guarantee. By the time four or five tasks in a row have been cancelled before they’re completed… well, the town obviously has bigger problems than who is carrying what around in their backpack; so there’s no chance for items to pile up on top of each other like they currently do.