I dont know how possible it is in WordPress, but Ive seen it done in Drupal
Allow each comment to have a button labled “Abuse”. This button is only available to logged in users, who can only use it 1x per comment. When a comment reaches a threshold of “abuse votes”, it is automatically removed.
With this, the community does a bit of self-regulation, offloading the burden from Keith. Keith can then sort through the “voted abuse” comments and re-add something if he feels the community has abused the “abuse” button