I used to worry about that as well, but then I found when I tried to make comments really meaningful and useful, I couldn't quite get them just right.

Now I view comments as a means of continuous practice, so even if there's no recognition by means of votes, I'm still improving my ability to contribute positively.