Those returned logs are fractioned logs, they waste a slot in inventories, but have a lower value.
I did a test. A new game, it started with two logs, I build some ladders and removed them, ending up with 8 logs in inventory. I then ordered them to build the small template house, which uses 7 logs. They did just 2 layers of the walls before running out of wood.
I repeated the same test, without increasing my logs making and removing stairs. So I ordered the same house, but this time the inventory had just the 2 original logs, they did the same 2 layers of walls.
So in conclusion, you have more itens, but they worth and can be used less. consider it like a 1/x of a log.