Hacker News
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
login
rubiquity
7 months ago
|
parent
|
context
|
favorite
| on:
End of an era for me: no more self-hosted git
The scrapers should use some discretion. There are some rather obvious optimizations. Content that is not changing is less likely to change in the future.
JohnTHaller
7 months ago
[–]
They don't care. It's the reason they ignore robots.txt and change up their useragents when you specifically block them.
Consider applying for YC's Winter 2027 batch!
Applications
are open till November 2.
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
Search:
