Milter-greylist

Is there a way to tell milter-greylist to disregard the recipient when making grey list decisions?

Example: if a given server sends a message from no-reply@... to user1@..., it will get grey listed, then white listed 5 minutes later (by my configs). Now, if 10 minutes later the same server uses the same sender to send a message to user2@... it will be grey listed again. I would like to NOT grey list in this case, and simply make grey listing decisions based on sender IP + sender address, regardless of what the recipient address is.

Thank you.
--