Milter-greylist

I hope that title makes sense.

I want to get the take of the milter-greylist community on the
exchange that I've forwarded below, since all of what they are saying
is soft evidence as opposed to hard knowledge or actual links to
documents. I don't think it makes sense or is correct.

Basically, what they are saying is that if I greylist sourceforge,
that can cause sourceforge to reject mail from me.

What's your take on this?



---------------

Chris Hoogendyk

-
 O__  ---- Systems Administrator
c/ /'_ --- Biology & Geology Departments
(*) \(*) -- 140 Morrill Science Center
~~~~~~~~~~ - University of Massachusetts, Amherst
<hoogendyk@...>

---------------
Erdös 4





-------- Original Message --------
Subject:     Re: [Bacula-users] Problems with the list
Date:     Thu, 1 Nov 2007 17:11:32 +0100
From:     Kern Sibbald <kern@...>
To:     GDS.Marshall <gdsm@...lmany.co.uk>
CC:     Chris Hoogendyk <hoogendyk@...>,
bacula-users@..., Augusto Lima <augusto@...>
References:
<4713.88.96.235.249.1193929015.squirrel@...>



On Thursday 01 November 2007 15:56, GDS.Marshall wrote:
> On Thu, 1 November, 2007 10:04 am, Kern Sibbald wrote:
> > On Wednesday 31 October 2007 21:20, Chris Hoogendyk wrote:
> >> Kern Sibbald wrote:
> >> > On Wednesday 31 October 2007 19:26, Augusto Lima wrote:
> >> >> Kern
> >> >>
> >> >> I'm having problems sending emails to the list. For a test,
i'm sending
> >> >> you a e-mail copying it to the list of bacula users. Have you
seen this
> >> >> kind of error before?
> >> >
> >> > Source Forge is *very* strict about who can send to a list. You
must
> >> > comply with all the email RFCs.  Typically things such as
greylisting
> >> > break it, not having reverse lookups, not having a postmaster
account in
> >> > your domain, ...
> >> > They have documented those things on their site, and if you
cannot figure
> >> > it out the only solution is for you to open a trouble ticket
with them.
> >>
> >> greylisting? Why should that affect who can send to the list?
> >> Greylisting works on the mail receipt end, not when sending. I'm on
> >> several sourceforge lists and a member of a project, and I've
never had
> >> any trouble. I use greylisting and a lot of other techniques to
> >> eliminate the spam hitting our front door. It's very common.
> >>
> >> I tried to find the documentation on the sourceforge site, but could
> >> not. Using google to search their site didn't help either (google
> >> "greylist site:sourceforge.net"), because they host at least a half a
> >> dozen different projects that implement greylisting for various mail
> >> configurations. ;-)
> >
> > I am not sure if Source Forge explicitly discusses grey listing in
their
> > documentation, but strangely enough, just a few days ago, a friend of
> > mine for whom my server acts as a mail relay  (MX) asked me to
check my
> > server log
> > for failure messages from Source Forge because he was having similar
> > problems
> > to the ones reported by Augusto.  Along with the log extract, I
suggested
> > a
> > number of things, and also mentioned that I had whitelisted the Source
> > Forge
> > sites in my grey lister.  He did the same, and the problem went
away --
> > so you figure it.
>
> Not sure if sourceforge do it, but if they do callout, you try send an
> e-mail through their server, their server calls your e-mail server to
> verify you really exist, (helo ... mail from: <> rcpt to: <you@..>)and
> yours greylists it.  Only sourceforge server does not know this, it just
> sees it as a rejection and fails your e-mail....
>
> I had a similar problem with one of the mail servers I manage and
another
> company rejecting our e-mail, once the greylist section was moved
further
> down, it worked, without changing any real functionality.

That is very interesting and could well be the basis of the problem. 
Fortunately, SF is easy to whitelist -- I just whitelisted their whole
IP range (66.35.250.0/24) or if you want to be ultra conservative,
only two IP addresses were critical (for me).

Regards,

Kern

>
> Spencer
>
> > Kern
> >
> >> Chris Hoogendyk
> >>

choogendyk <hoogendyk@...> wrote:

> I want to get the take of the milter-greylist community on the
> exchange that I've forwarded below, since all of what they are saying
> is soft evidence as opposed to hard knowledge or actual links to
> documents. I don't think it makes sense or is correct.

According to what you forwarded, SF uses a sender-callback to check
sender e-mail address is valid. milter-greylist has the delayedreject
statement for that kind of problems.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

manu@... wrote:
> choogendyk <hoogendyk@...> wrote:
>   
>> I want to get the take of the milter-greylist community on the
>> exchange that I've forwarded below, since all of what they are saying
>> is soft evidence as opposed to hard knowledge or actual links to
>> documents. I don't think it makes sense or is correct.
>>     
>
> According to what you forwarded, SF uses a sender-callback to check
> sender e-mail address is valid. milter-greylist has the delayedreject
> statement for that kind of problems.
>   


OK. I see that now in the greylist.conf and its man page. It 
(delayedreject) is commented out in our greylist.conf.

Now, I've never been aware of having any trouble with sourceforge, and I 
doubted what others were saying about greylist breaking sourceforge 
lists. When I went looking through my own mail logs with a search image 
based on what they were saying, I did find call back exchanges. But they 
seemed to work. I get well over a million lines of log file every day, 
so I think it's excusable that I hadn't seen this in my previous log 
file readings.

Anyway, I pasted the relevant log lines at the end of this.

I'm curious what your take is on the exchange between the callback and 
greylist and why I didn't seem to need the delayedreject. I'm not sure 
what's going on at the lowest level -- whether sourceforge is respecting 
the temp fail, or whether my mta is thinking the temp fail applies to 
itself and then has to wait for the queue runner to come back around.

I must confess that I am using an ancient milter-greylist -- 1.6 on 
Solaris 9 on SPARC. Based on the fix of the Solaris issues and the 
seemingly stable 4.0 release, I will probably be updating very soon.



---------------

Chris Hoogendyk

-
   O__  ---- Systems Administrator
  c/ /'_ --- Biology & Geology Departments
 (*) \(*) -- 140 Morrill Science Center
~~~~~~~~~~ - University of Massachusetts, Amherst 

<hoogendyk@...>

--------------- 

Erd\ufffds 4



[NOTE: the followed was spread over a gazillion pages intermingled with lots of other entries]

Oct 31 16:20:25 marlin milter-greylist: [ID 226885 mail.debug] l9VKKOlG024509: addr = 128.119.54.86, from = <hoogendyk@...>, rcpt = <bacula-users@...>

Oct 31 16:20:25 marlin sendmail[24509]: [ID 801593 mail.info] l9VKKOlG024509: from=<hoogendyk@...>, size=2166, class=0, nrcpts=4, msgid=<4728E388.6010605@...>, proto=ESMTP, daemon=MSA, relay=peredhil.bio.umass.edu [128.119.54.86]

Oct 31 16:20:26 marlin mimedefang.pl[13967]: [ID 702911 mail.info] MDLOG,l9VKKOlG024509,mail_in,,,<hoogendyk@...>,<bacula-users@...>,Re: [Bacula-users] Problems with the list

Oct 31 16:20:26 marlin sendmail[24509]: [ID 801593 mail.info] l9VKKOlG024509: Milter add: header: X-Scanned-By: MIMEDefang 2.54 on 128.119.55.19

[NOTE: OK, here comes the callback that I had previously missed in all the clutter]
Oct 31 16:20:27 marlin sendmail[24511]: [ID 702911 mail.info] STARTTLS=client, relay=mail.sourceforge.net., version=TLSv1/SSLv3, verify=OK, cipher=AES256-SHA, bits=256/256

[NOTE: these next 2 lines were generated by code we added to milter-greylist]
Oct 31 16:20:28 marlin milter-greylist: [ID 596769 mail.debug] Looking up 66.35.250.206 in /etc/mail/popip.db

Oct 31 16:20:28 marlin milter-greylist: [ID 575326 mail.debug] 66.35.250.206 was not in database

Oct 31 16:20:28 marlin milter-greylist: [ID 226885 mail.debug] l9VKKSBk024514: addr = 66.35.250.206, from = <>, rcpt = <hoogendyk@...>

Oct 31 16:20:28 marlin milter-greylist: [ID 173742 mail.debug] created: 66.35.250.206 from <> to <hoogendyk@...> delayed for 120s

Oct 31 16:20:28 marlin milter-greylist: [ID 928669 mail.info] l9VKKSBk024514: addr 66.35.250.206 from <> to <hoogendyk@...> delayed for 00:02:00

[NOTE: so, this appears to be telling them to try again in 2 minutes]
Oct 31 16:20:28 marlin sendmail[24514]: [ID 801593 mail.info] l9VKKSBk024514: Milter: to=<hoogendyk@...>, reject=451 4.7.1 Greylisting in action, please come back in 00:02:00

Oct 31 16:20:28 marlin sendmail[24514]: [ID 801593 mail.info] l9VKKSBk024514: from=<>, size=0, class=0, nrcpts=0, proto=SMTP, daemon=MTA, relay=lists.sourceforge.net [66.35.250.206]

[NOTE: and it notes here that the response back was to come back in 2 minutes]
Oct 31 16:20:28 marlin sendmail[24511]: [ID 801593 mail.info] l9VKKOlG024509: to=<bacula-users@...>, delay=00:00:03, xdelay=00:00:02, mailer=esmtp, pri=122661, relay=mail.sourceforge.net. [66.35.250.206], dsn=4.3.0, stat=Deferred: 451
-response to "RCPT TO:<hoogendyk@...>" from marlin.bio.umass.edu [128.119.55.19] was: 451 4.7.1 Greylisting in action, please come back in 00:02:00

[NOTE: OK, so here they come with the call back again, over 3 hours later,
 OR, is it my queue runner coming back 3 hours later & getting a callback?]
Oct 31 19:24:33 marlin sendmail[10599]: [ID 702911 mail.info] STARTTLS=client, relay=mail.sourceforge.net., version=TLSv1/SSLv3, verify=OK, cipher=AES256-SHA, bits=256/256

Oct 31 19:24:34 marlin milter-greylist: [ID 596769 mail.debug] Looking up 66.35.250.206 in /etc/mail/popip.db

Oct 31 19:24:34 marlin milter-greylist: [ID 575326 mail.debug] 66.35.250.206 was not in database

Oct 31 19:24:34 marlin milter-greylist: [ID 226885 mail.debug] l9VNOYCe010604: addr = 66.35.250.206, from = <>, rcpt = <hoogendyk@...>

Oct 31 19:24:34 marlin milter-greylist: [ID 333256 mail.debug] removed: 66.35.250.206 from <> to <hoogendyk@...>

Oct 31 19:24:34 marlin milter-greylist: [ID 829939 mail.info] l9VNOYCe010604: addr 66.35.250.206 from <> rcpt <hoogendyk@...>: autowhitelisted for 20:00:00

Oct 31 19:24:35 marlin sendmail[10604]: [ID 801593 mail.info] l9VNOYCe010604: from=<>, size=0, class=0, nrcpts=1, proto=SMTP, daemon=MTA, relay=lists.sourceforge.net [66.35.250.206]

[NOTE: and finally, my message goes through!]
Oct 31 19:24:36 marlin sendmail[10599]: [ID 801593 mail.info] l9VKKOlG024509: to=<bacula-users@...>, delay=03:04:11, xdelay=00:00:04, mailer=esmtp, pri=212661, relay=mail.sourceforge.net. [66.35.250.206], dsn=2.0.0, stat=Sent (OK id=1
InMvL-0001b1-25)

[NOTE: that delay is longer than I think I'm used to seeing.
 I think I caught everything, but I didn't see the postmaster check on this sequence.
 I thought I had seen that on the first sequence like this that I checked.]