I expect that part of the problem is all of the connection requests hitting the IP banned page. The issue is not CPU load, but processing incoming connections.