

1·
11 days agoIf I didn’t enjoy the cat & mouse of it, I’d definitely walk away from running a site. Just out of curiosity, I created an hourly graph today showing malicious bot, legitimate bot and human ratio graphs for one of my sites. The malicious hits make up around 80% hourly and human hits are usually well under 10%.


As soon as an IP is determined to be malicious, subsequent pageloads result in a single query to match the IP and they get dumped to a 403 static error.
With the bots that hammer a site looking for the top 1,000 or so exploitable URLs or partake in speed scraping, this has worked wonders for my server load. I catch them in their 1st-5th page load and the rest just result in basically no effort on the part of the script.