I'm not sure if you're testing through the M. Groups tab, or if you're testing battles within the game itself.

If it's in the M. Groups tab, your character may simply be too slow to get away.

If it's within the game, it may be the same as above. If you're calling the battle from an event, make sure that the Enemy Encounter part of your event is set to allow escape (it's set to disable by default).

Escape in the Map settings doesn't have to do with escaping battles, it has to do with spells that you can use to escape from the dungeon (think Exit from some of the older Final Fantasy games).