Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Basic Web Scraping with Emacs (nickdrozd.github.io)
62 points by tarboreus on Oct 21, 2018 | hide | past | favorite | 6 comments


The idea here is that 1) Emacs has flexible means of automation (like keyboard macros), and 2) it also has a web browser, so 3) Emacs can be used for browser automation. The conclusion seems trivial in retrospect, but I hadn't seen it discussed anywhere, so I thought the techniques involved would be worth recording.


Thanks a lot! I use Emacs and have some scraping projects (mainly converting some websites' announcements into RSS feeds), but have never thought of this either.


Just finding about eww was fun for me, I could probably tune browser settings/plugins to approximate it but it makes web browsing on sites I just want to read extremely quick to load no matter how much extraneous baggage is on the web site.


Had to search for how to use eww since I have never used it before now.


Guess dynamical websites aren't supported?


JavaScript isn’t run, so some dynamic web sites are supported and others (common today) aren’t.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: