There already is downvoting, it just isn't enabled until you hit a certain karma (I think 500, could be completely wrong as it's late and I'm tired)