It doesn't show how many people downvoted the post, it shows "points" the post has. Current system is always 1 point per vote, old system could be 100 points per vote depending on user.
I think I can find a past post of mine that has more downvotes. I can dig a little
EDIT - nah nvm I can't find anything. My more "controversial" posts (bigots triggered amirite?) are from before the vote system. Keep an eye on that post though. It's at 27 and is liable to be downvoted further the more attention you draw to it.