Milter-greylist

I use "greylist dnsrbl" and it is greylisting about 99% of the incoming
mail. 

However, milter-greylist does the dnsrbl lookups for non-existent
recipient addresses, and we get _lots_ of these. Is there a way to
reject nonexistent recipients before doing the dnsrbl lookup?

Mark