Hacker News
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
login
chirau
on June 11, 2020
|
parent
|
context
|
favorite
| on:
A Facebook crawler was making 7M requests per day ...
I think sending back a `429 Too Many Requests` can solve this. Unless you want to block their IPs completely, which I doubt you'd want to do.
jandrese
on June 11, 2020
|
next
[–]
A well made crawler that would support 429 codes isn't likely to have this problem in the first place.
klohto
on June 11, 2020
|
prev
|
next
[–]
> HTTP 429 -> ignored
It's in the article
napolux
on June 11, 2020
|
prev
|
next
[–]
I tried. It was ignored.
gramakri
on June 11, 2020
|
prev
[–]
It says in the post he did this already
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
Search:
