The server does collision checking when the mob hits the wall of the cave, but since they're generally horizontal at the time, they fit under overhangs, and then when they stand up, the game doesn't check again, so they stand up with their chest inside the wall, putting their hit target on the wrong side of the wall, making them untargetable.
Edit: I think this is related to mobs' lack of vertical collision boxes, which means that you can get a mob jumping down onto another and overlapping it; this allowed stuffing an entire map's worth of mobs into a dumpster in the wolf farm on the Boomtown map. With no vertical collision box, there's nothing keeping a mob from standing up in spaces where there is insufficient vertical clearance and making them untargetable.