No. No no no. You do realize that this would just help trolls to achieve their ultimate goal: Nobody can actually play. Because for the regular player the only way to avoid trolls would be to activly vote for the content he wants to play not to happen. The mini the troll is in would be voted down, but there is nothing that increases the chance that he would not be in the next mini. Which would be voted down, too. Nice, voting down minis in a row is not pvp.
This would be wrong on so many levels I am really surprised you came up with crap like this. Trolls want to destroy the fun other people, so in order to conquer trolls we choose NOT TO PLAY AT ALL???
To make social gameplay fun for the majority you have to remove antisocial players. And not the whole gameplay itself.