Well, I just built my own Perl mail filter that bounces back any email that isn't on my validated list. If the user clicks a link in the bounced email, it will validate them, send me their original email, and them accept them as a valid sender from there on in.
After several days, if the email isn't validated, it is just wiped out.
I have it set up so I can manually add/remove users to the valid list, and also add/remove domains (riskier) that I allow. Other than that, I really do not have to do anything on my end.
I know it is basic, and surely been done before, but it has eliminated my incoming SPAM, and after browsing the MySQL records of what is not being validated, it was 100% SPAM without one false positive.
I have released it free, and the source is far from pretty, but it does work. If your interested, I will send it to you.