Don't forget the post score is a balance of up votes and down votes. This mean 2 down votes and 0 up votes registers the same (-1) as 10 down votes and 8 up votes (-1).
So there's no sense treating it like the community has forsaken you.
If the purpose of down votes is to guide discussion away from uninteresting or counterproductive paths, then it also scoops up divisive comments.
I suppose divisive comments are potentially disruptive if they are particularly flamey. But it seems like they deserve to be distinguished from more objectively negative comments. If the raw up/downs were exposed it would be interesting to experiment with this in the front end.
I'm pretty sure that downvotes from some users, perhaps those at a given karma threshold, count twice. I've seen quite a few posts that aren't otherwise attracting a lot of attention go straight from 1 to -1 more-or-less instantly, or from 3 back to 1.
It's also possible that they have experimented with fuzzing votes like Reddit does.
Either way, they don't seem inclined to disclose how it works behind the scenes. The Kremlinology of Hacker News is half the fun.
So there's no sense treating it like the community has forsaken you.