I reod some people use false slugs in the robots.txt as a honey pot of sorts. IP... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

toastal on June 8, 2021 | parent | context | favorite | on: Fastly Outage

I reod some people use false slugs in the robots.txt as a honey pot of sorts. IPs that actually read the robots.txt, ignore the disallow, and still access the uri are outright banned.

taylorfinley on June 8, 2021 | [–]

Then when a flamewar breaks out you just have to get your adversary to click a link to get them IP banned.

jedberg on June 8, 2021 | [–]

Ha, that would have been a really smart idea! Sadly we didn't think of that at the time. But we had other honey pot URLs.

Consider applying for YC's Winter 2027 batch! Applications are open till November 2.
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact