When Non-Human Traffic Slows Your Site
Bot traffic — automated requests from crawlers, scrapers, and malicious scripts — can consume significant server resources without generating any value for your business. On many WordPress sites, bots account for 30-50% of total traffic. When bot activity spikes, the additional server load can slow page responses for real visitors, increase hosting costs, and skew your analytics data.
Not all bots are harmful. Googlebot, Bingbot, and other legitimate search engine crawlers need access to index your content. The problem is aggressive or malicious bots: content scrapers that mirror your site, vulnerability scanners probing for exploits, credential stuffing attacks against your login page, and spam bots submitting forms. These generate server load without contributing anything useful.
How Bots Affect Performance
Every bot request consumes the same server resources as a real visitor: CPU cycles to execute PHP, memory to run WordPress, database queries to build the page. If your site receives 1,000 real visitors per hour and 2,000 bot requests per hour, your server is doing three times the work it needs to for actual users.
The impact is most visible on sites without page caching. Each uncached request triggers a full WordPress execution cycle — loading plugins, running database queries, rendering templates. With caching, bot requests for already-cached pages are cheap (served from cache without running PHP). But bots that target non-cached URLs, POST endpoints, search pages, or wp-login.php bypass the cache entirely and hit your server at full cost.
During bot traffic spikes, server response time degrades for everyone. A server that normally responds in 200 milliseconds might respond in 800 milliseconds or more when processing a high volume of concurrent bot requests alongside real visitor traffic.
Signs of Excessive Bot Traffic
Unexplained spikes in server resource usage — CPU, memory, or bandwidth — that do not correlate with traffic in your analytics tool are a strong indicator. Since most analytics tools use JavaScript tracking (which bots do not execute), a gap between server access logs and analytics data suggests bot activity.
High request rates to wp-login.php, xmlrpc.php, or wp-admin paths often indicate brute force login attempts. Rapid sequential requests from single IP addresses or user-agent strings that do not match known browsers point to automated scraping. Unusually high crawl rates from search engines can also be a factor — Googlebot is generally well-behaved, but misconfigured crawl settings can lead to excessive indexing requests.
Managing Bot Load
The most effective defenses operate at the server or network level rather than within WordPress. Rate limiting at the web server (LiteSpeed, Nginx, or Apache) can throttle requests from individual IPs before they reach PHP. A DNS-level web application firewall can filter known malicious bots before traffic reaches your server at all.
Within WordPress, a well-configured robots.txt file can guide legitimate crawlers away from resource-intensive pages. Security plugins that log and block suspicious IPs can help, though they add their own processing overhead to every request. The goal is to reduce the volume of wasteful requests reaching your server without adding so much inspection overhead that you negate the benefit.
Further Reading
- Hardening WordPress (WordPress Developer Resources) — Security measures that reduce your site’s exposure to malicious bot traffic.
- Reduce Googlebot Crawl Rate (Google Search Central) — How to manage legitimate search engine crawl rates if they are causing server load issues.