When a group of monsters pops into the map, if you have gates built, only Ogres will charge, presumably because they are the only ones that have a path. This means a few things happen:
The ogres charge forward with no support and are easily cut down.
You don’t always realize there are even non-ogre monsters on the map since they’ll be so far away.
If you do realize they’re there, by the time you engage them with your guys, half their clock has run out and they may POOF before the fight ends. (Which is a real problem if you’re trying to get a hat).
See this screenshot, with charging ogres and the kobolds not bothering, all the way near the map edge: