I spent the last two days implementing various bot mitigation solutions. From what I've gathered, this seems to be a game of Whac-A-Mole. Implement a solution -> the bots get a little smarter -> repeat. At this point, I've pulled nearly every lever at my disposal without significantly increasing overhead. Despite some false dawns where we were down to ~9k guests, after I implemented the last solution this afternoon, we reached a record-high (that I've seen, anyway) of 21K guests. Then, at 4 p.m. PT, the number fell off a cliff. We went from 55K concurrent requests to ~4K. And there it's stayed.
I have no idea if this is due to a switch I flipped. Frankly, I doubt it. This traffic appeared out of thin air, and it seems to have disappeared into thin air. Knock on wood that it's gone for good, but color me skeptical.
The good news, knock on wood again, please, is that I don't anticipate another crash due to bot traffic. Revoking guest search privileges, monitoring database usage, limiting IP data storage, and implementing a few bot mitigation solutions should keep us free and clear.
If the bot traffic ticks up again, there are two more levers I can pull: 1) a CAPTCHA checkbox, likely something you'd need to complete on your first visit to the site every day, and/or 2) a static page, also likely visible on your first visit to the site every day, that would perform a four-second check before you were able to enter the site. Because I no longer think we're in danger of further disruption, I don't think either of these steps would be worth it. Those are big quality-of-life sacrifices. Not to mention the fact that no new users would join a site with those barriers.
So, we're hoping the bot traffic stays where it is (1,125 as of this post). I don't think we're in jeopardy of another bot-caused outage. I could implement further mitigation solutions, but it doesn't seem worth it.
Cue a long exhale.
