Its complexity. Elisp is both a blessing and a curse. It's great to be able to make deep, fundamental changes to Emacs via Elisp, but sometimes, you end up wading though very difficult-to-read code trying to figure out how it works. IME, vim is less flexible, but simpler when I just want to edit code and not think about Emacsisms.