Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Using a 1 pixel transparent GIF works - also worth putting in a warning for screen readers not to follow the link.

FWIW I've been using robots.txt honeypots on two sites for the past few years to deter scrapers. Approx 1,600 IP addresses have been banned. Most of these are for data centers.



Fuck you guy, that's pretty pricky. Is bandwidth that expensive for you?


That's uncalled for. If he owns the data and pays for the connectivity, he should be free to do as he pleases just like I can exclude you from my home or business without justification.


I like humans browsing my sites but I certainly don't want scrapers copying my work wholesale, claiming it as their own and ranking ahead of me in SERPs.

I'm not alone :-) see http://incredibill.blogspot.com/




Consider applying for YC's Winter 2027 batch! Applications are open till November 2.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: