As a legal idea, civil forfeiture is something the state specifically empowers police to do. This is on top of cops pretty much being able to do what they want without recourse. Normal civilians don't have this luxury.
Legitimacy stems from the consent of the governed. The state and the Constitution determine what powers the state has, eg, by passing laws and by subjecting them to judicial review.
A pretty sentence, but since the courts have upheld both qualified immunity and civil asset forfeiture, it appears the people have granted the power to do so.
Aye, there's the rub. No politician wants to be portrayed as "soft on crime". Until the problem becomes too large for the average person to ignore, the law isn't going to change. A smart police force stays just below that threshold.