CloudLinux and mod_hostinglimits
CloudLinux provides a wrapper called LVE (Lightweight Virtual Environment), that is powered by the Apache module mod_hostinglimits. This allows apache to read from the LVE configuration to determine the resource limits for a user dynamically. What LVE does specifically is sets limits for maximum concurrent processes, maximum memory, maximum CPU speed, maximum inodes, and concurrent MySQL queries.
The Problem
In the past, if a user has reached their EP (Entry Processes) limit, then they would normally receive a 508 (Resource Limit Reached) when trying to visit their website. This is very bad for SEO and your users will simply go elsewhere. Obviously this is bad, as the default response code indicates to a crawler or bot that there’s a server error. This has been the CloudLinux default for quite some time.
The Solution
With CloudLinux’s release of 1.0-27, the ability to show a 429 (Too many requests) page has become available. This means you can customize the response with ErrorDocument so you can show a custom throttle page when the concurrent requests meets or exceeds lEP (Entry Processes Limit).
Install and Update mod_hostinglimits
To get this feature, simply install or update mod_hostinglimits with yum
yum install mod_hostinglimits yum update mod_hostinglimits
Simply specify the following in /usr/local/apache/conf/includes/pre_main_global.conf
LVEUse429 on
Then restart Apache:
CentOS 6.x
httpd -t && /etc/init.d/httpd restart
CentOS 7.x
httpd -t && systemctl restart httpd.service
Conclusion
Once this is installed, you can rest easy knowing that most legitimate bots and crawlers will back off and retry due to too many requests being received. Once the user goes back under the EP limit (lEP), then it will resume normal operation.