This caused the screen to zoom way out. It was kind of hard to zoom back to the town.
Do you have a screenshot of how much distance did it go?
I think clicking there when you have something selected, makes the camera focus on it, but at the same time the camera zooms out a bit. It could happen that depending on the size of the item, the distance looks too much or too little.
Or do you mean that the camera went far away and you found it hard to approach the camera again to where the town was?