| baidu spider and referer spam I just noticed that the Baidu Spider has started putting "findmore.org"
as the referer in their header.
My stats page is Disallowed in robots.txt so referer spam doesn't work,
but I still dislike the idea that these guys are trying it. The spider
is crawling at a reasonable rate (just a couple of hits per hour at the
most), so bandwidth isn't an issue.
Any thoughts on this? Is putting a referer in the spider crawl spammy?
Should I ban these guys from my website?
RFM |