Milter-greylist

Aside from the dnsbl checks, is there a way to impose a specific
delay when specifying a greylisting rule?  I have an email account
where I only expect to get legitimate mail from a select set of
specific addresses, but it is being broadly spammed and the spamming
engines are sophisticated enough to get past the greylisting.

What I'd like to do is to first whitelist the legitimate From:
addresses in the config file and then set a very long greylist
interval on mail to that address which would apply for everyone
else.  Can such a thing be done?

Rob