Milter-greylist

Examining my greylist.db I find a large number of whitelisted addresses
don't have 
sender e-mail addresses, and by in large (I'd say close to 99% of the 
emails are being sent to non-existent users - in other words they are all
spam).

How do I configure Milter-Greylist to blacklist these e-mails or blacklist
people sending to non-existent email addresses (which can be problematic if
someone misspells a username?

Thx

Wayne