I'm a little bummed to find that this doesn't work on Solaris either (just checked). I don't generally use spaces in paths, but it's probably about half luck that I haven't been bitten by this yet. And sadly, I can't always just put GNU findutils on.
You need to add -print0 to find, and -0 to xargs to make it work properly: find . -name "*.swp" -print0 | xargs -0 rm
But better would be to use -delete as I wrote above.