Nicely put! I wrote svg.js because I ran into limitations with Raphael. The great thing about Raphael is its compatibility with older browsers, but it is also its weakest spot because VML doesn't do everything SVG does, thereby limiting possibilities.
I wanted a library that still breathed SVG while providing a more unified API. I think too much abstraction leads to isolation. It's important to stay in touch with the underlying language, SVG.
Raphael also seems to be a bit more about abstracting SVG away from you, where SVG.js is more direct manipulation of the SVG document.