IMO...if you really respected yourself you wouldn't need to resort to blows. Like Titus said, if you can avoid it do so. A fight like that really wouldn't solve anything in the end, save for making both groups even more pissed off in the end. If you do end up beating him and his friends, there's a chance he'll resort to something drastic. Stay calm and let any thrown insults just bounce off of you. Fight only when there's no way out of it.

Girls like a guy who can stay cool under pressure. I do, anyway.