Milter-greylist

Hi!

I have "report all" in my greylist.conf:

	#
	# Simple greylisting config file using the new features
	# See greylist2.conf for a more detailed list of available options
	#
	# $Id: greylist.conf,v 1.50 2013/08/13 12:45:08 manu Exp $
	#

	#pidfile "/var/run/milter-greylist.pid"
	socket "/run/milter-greylist/milter-greylist.sock"
	dumpfile "/var/lib/milter-greylist/db/greylist.db" 600
	geoipdb "/usr/share/GeoIP/GeoIP.dat"
	dumpfreq 10m
	user "grmilter"
	greylist 10m
	extendedregex
	report all	# always add X-greylist mail header

and for some reason I can't figure out, a few spams leak through
that DO NOT get a x-greylist header inserted. Can any of you shed
any light on this issue for me?

thanks in advance!

Fred

-- 
---- Fred Smith -- fredex@... -----------------------------
  "For him who is able to keep you from falling and to present you before his 
 glorious presence without fault and with great joy--to the only God our Savior
 be glory, majesty, power and authority, through Jesus Christ our Lord, before
                     all ages, now and forevermore! Amen."
----------------------------- Jude 1:24,25 (niv) -----------------------------

Fred Smith fredex@... [milter-greylist]
<milter-greylist@yahoogroups.com> wrote:

> and for some reason I can't figure out, a few spams leak through
> that DO NOT get a x-greylist header inserted. Can any of you shed
> any light on this issue for me?

Perhaps you found a bug. Is there some milter-greylist logs about the
message?

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

On Tue, Oct 25, 2016 at 06:02:40AM +0200, manu@... [milter-greylist] wrote:
> Fred Smith fredex@... [milter-greylist]
> <milter-greylist@yahoogroups.com> wrote:
> 
> > and for some reason I can't figure out, a few spams leak through
> > that DO NOT get a x-greylist header inserted. Can any of you shed
> > any light on this issue for me?
> 
> Perhaps you found a bug. Is there some milter-greylist logs about the
> message?

I assume you mean the logs, as defined in the basic greylist.conf
sample file:

#stat ">>/var/milter-greylist/greylist.log" \
#      "%T{%Y/%m/%d %T} %d [%i] %f -> %r %S (ACL %A) %Xc %Xe %Xm %Xh\n"

whenever I enable that I get errors about can't open file,  permisson
denied. looking further it seems to be a selinux issue that so far I've
not figured out the right incantation.

Fred
-- 
---- Fred Smith -- fredex@... -----------------------------
   "For the word of God is living and active. Sharper than any double-edged 
   sword, it penetrates even to dividing soul and spirit, joints and marrow; 
              it judges the thoughts and attitudes of the heart."  
---------------------------- Hebrews 4:12 (niv) ------------------------------

To allow SELinux to do what you want you have to do the following:

1) Put SELinux into permissive mode
    $ setenforce permissive

2) Get milter-greylist to write to the file
    ….
    
3) create custom policy by grep’ng the audit log (note this creates a .pp and .te file)
   $ grep greylist /var/log/audit/audit.log | audit2allow -M [policy_name]

4) Look at what is going to happen
   $ cat [policy_name].te

My guess is you’ll see something like this at the bottom of the file:
#============= greylist_milter_t ==============
allow greylist_milter_t var_log_t:file open;

5) Implement policy
   $ semodule -i [policy_name].pp

Be sure to have 'policycoreutils-python’ installed.

Also, you’ll need 300-400k free to update the semodules

Bill

> On Oct 25, 2016, at 6:36 PM, Fred Smith fredex@fcshome.stoneham.ma.us [milter-greylist] <milter-greylist@yahoogroups.com> wrote:
> 
> On Tue, Oct 25, 2016 at 06:02:40AM +0200, manu@... [milter-greylist] wrote:
> > Fred Smith fredex@... [milter-greylist]
> > <milter-greylist@yahoogroups.com> wrote:
> > 
> > > and for some reason I can't figure out, a few spams leak through
> > > that DO NOT get a x-greylist header inserted. Can any of you shed
> > > any light on this issue for me?
> > 
> > Perhaps you found a bug. Is there some milter-greylist logs about the
> > message?
> 
> I assume you mean the logs, as defined in the basic greylist.conf
> sample file:
> 
> #stat ">>/var/milter-greylist/greylist.log" \
> # "%T{%Y/%m/%d %T} %d [%i] %f -> %r %S (ACL %A) %Xc %Xe %Xm %Xh\n"
> 
> whenever I enable that I get errors about can't open file, permisson
> denied. looking further it seems to be a selinux issue that so far I've
> not figured out the right incantation.
> 
> Fred
> -- 
> ---- Fred Smith -- fredex@... -----------------------------
> "For the word of God is living and active. Sharper than any double-edged 
> sword, it penetrates even to dividing soul and spirit, joints and marrow; 
> it judges the thoughts and attitudes of the heart."  
> ---------------------------- Hebrews 4:12 (niv) ------------------------------
> 
>

On Tue, Oct 25, 2016 at 06:02:40AM +0200, manu@... [milter-greylist] wrote:
> Fred Smith fredex@... [milter-greylist]
> <milter-greylist@yahoogroups.com> wrote:
> 
> > and for some reason I can't figure out, a few spams leak through
> > that DO NOT get a x-greylist header inserted. Can any of you shed
> > any light on this issue for me?
> 
> Perhaps you found a bug. Is there some milter-greylist logs about the
> message?

here's an example that didn't get the X-greylist header. first, logs
from /var/log/maillog:

Oct 26 21:17:37 fcshome sendmail[5226]: u9R1HJpw005226: Milter (greylist): timeout before data read, where=rcpt
Oct 26 21:17:37 fcshome sendmail[5226]: u9R1HJpw005226: Milter (greylist): to error state
Oct 26 21:17:37 fcshome milter-greylist: u9R1HJpw005226: addr host-249-87.ncch901.charlotte.nc.us.clients.pavlovmedia.com[50.30.249.87] from <laecntmymgksz@...> to <xxx@...> delayed for 00:10:00 (ACL )
Oct 26 21:17:37 fcshome sendmail[5226]: u9R1HJpw005226: from=<laecntmymgksz@...>, size=508, class=0, nrcpts=2, msgid=<021921058958-UEPOZBETRJQXHAPPWYOM@...>, proto=SMTP, daemon=MTA, relay=host-249-87.ncch901.charlotte.nc.us.clients.pavlovmedia.com [50.30.249.87]
Oct 26 21:17:38 fcshome sendmail[5239]: u9R1HJpw005226: to=<xxx@...>, delay=00:00:01, xdelay=00:00:00, mailer=local, pri=60773, dsn=2.0.0, stat=Sent
Oct 26 21:17:38 fcshome sendmail[5239]: u9R1HJpw005226: to=<fredex@...>, delay=00:00:01, xdelay=00:00:00, mailer=local, pri=60773, dsn=2.0.0, stat=Sent


and here is an entry that appears to be the same message, from 
greylist.log, using the greylist.conf recipe I showed earlier:

2016/10/26 21:17:37 host-249-87.ncch901.charlotte.nc.us.clients.pavlovmedia.com [50.30.249.87] laecntmymgksz@... -> xxx@... tempfail (ACL (none)) 451 4.7.1 Greylisting in action, please come back later

I'm suspicious of that "tempfail" in the item above, what does that mean?

here are the actual headers in the email (all of them, slightly censored):

From laecntmymgksz@...  Wed Oct 26 21:17:38 2016
Return-Path: <laecntmymgksz@...>
Received: from host-249-87.ncch901.charlotte.nc.us.clients.pavlovmedia.com
       	(host-249-87.ncch901.charlotte.nc.us.clients.pavlovmedia.com [50.30.249.87])
       	by fcshome.stoneham.ma.us (8.14.7/8.14.7) with SMTP id u9R1HJpw005226;
       	Wed, 26 Oct 2016 21:17:37 -0400
Message-ID: <021921058958-UEPOZBETRJQXHAPPWYOM@...>

From: Leticia Murray <Murray_Meg@...>
Subject: Many years passed, but you are still charming
To: xxx@...
Date: Wed, 26 Oct 2016 20:11:24 -0600
Mime-Version: 1.0
Content-Type: text/html;
Content-Transfer-Encoding: 7Bit
X-Spambayes-Classification: unsure; 0.90
Status: RO
Content-Length: 189
Lines: 2


-- 
---- Fred Smith -- fredex@... -----------------------------
    "Not everyone who says to me, 'Lord, Lord,' will enter the kingdom of
     heaven, but only he who does the will of my Father who is in heaven."
------------------------------ Matthew 7:21 (niv) -----------------------------

Fred Smith fredex@... [milter-greylist]
<milter-greylist@yahoogroups.com> wrote:

> Oct 26 21:17:37 fcshome sendmail[5226]: u9R1HJpw005226: Milter (greylist):
> timeout before data read, where=rcpt

For some reason, milter-greylist timed out. Is the machine busy, or is
it running on another machine?

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

On Thu, Oct 27, 2016 at 06:00:13AM +0200, manu@... [milter-greylist] wrote:
> Fred Smith fredex@... [milter-greylist]
> <milter-greylist@yahoogroups.com> wrote:
> 
> > Oct 26 21:17:37 fcshome sendmail[5226]: u9R1HJpw005226: Milter (greylist):
> > timeout before data read, where=rcpt
> 
> For some reason, milter-greylist timed out. Is the machine busy, or is
> it running on another machine?

its a six-core AMD Vishera 6350. but it is running Folding At Home
client. I can try freeing up one core to see if that helps.

but what's funny is I haven't seen this on non-spam emails.

Fred
-- 
---- Fred Smith -- fredex@... -----------------------------
                       I can do all things through Christ 
                              who strengthens me.
------------------------------ Philippians 4:13 -------------------------------

On Thu, Oct 27, 2016 at 06:00:13AM +0200, manu@... [milter-greylist] wrote:
> Fred Smith fredex@... [milter-greylist]
> <milter-greylist@yahoogroups.com> wrote:
> 
> > Oct 26 21:17:37 fcshome sendmail[5226]: u9R1HJpw005226: Milter (greylist):
> > timeout before data read, where=rcpt
> 
> For some reason, milter-greylist timed out. Is the machine busy, or is
> it running on another machine?
its a six-core AMD Vishera 6350. but it is running Folding At Home
client. I can try freeing up one core to see if that helps.

but what's funny is I haven't seen this on non-spam emails.

Fred


-- 
---- Fred Smith -- fredex@... -----------------------------
  "And he will be called Wonderful Counselor, Mighty God, Everlasting Father,
  Prince of Peace. Of the increase of his government there will be no end. He 
 will reign on David's throne and over his kingdom, establishing and upholding
      it with justice and righteousness from that time on and forever."
------------------------------- Isaiah 9:7 (niv) ------------------------------

On Thu, Oct 27, 2016 at 08:18:30AM -0400, Fred Smith fredex@... [milter-greylist] wrote:
> > For some reason, milter-greylist timed out. Is the machine busy, or is
> > it running on another machine?
> its a six-core AMD Vishera 6350. but it is running Folding At Home
> client. I can try freeing up one core to see if that helps.

Perhaps it times out on a DNS resolutions? Adding CPU will not help here.
-- 
Emmanuel Dreyfus
manu@...

On Thu, Oct 27, 2016 at 12:55:09PM +0000, Emmanuel Dreyfus manu@... [milter-greylist] wrote:
> On Thu, Oct 27, 2016 at 08:18:30AM -0400, Fred Smith fredex@... [milter-greylist] wrote:
> > > For some reason, milter-greylist timed out. Is the machine busy, or is
> > > it running on another machine?
> > its a six-core AMD Vishera 6350. but it is running Folding At Home
> > client. I can try freeing up one core to see if that helps.
> 
> Perhaps it times out on a DNS resolutions? Adding CPU will not help here.

good question. is there a way to tell?

Fred

-- 
---- Fred Smith -- fredex@... -----------------------------
                      The eyes of the Lord are everywhere, 
                    keeping watch on the wicked and the good.
----------------------------- Proverbs 15:3 (niv) -----------------------------

On Thu, Oct 27, 2016 at 12:55:09PM +0000, Emmanuel Dreyfus manu@... [milter-greylist] wrote:
> On Thu, Oct 27, 2016 at 08:18:30AM -0400, Fred Smith fredex@... [milter-greylist] wrote:
> > > For some reason, milter-greylist timed out. Is the machine busy, or is
> > > it running on another machine?
> > its a six-core AMD Vishera 6350. but it is running Folding At Home
> > client. I can try freeing up one core to see if that helps.
> 
> Perhaps it times out on a DNS resolutions? Adding CPU will not help here.

so far I've seen this phenomenon ONLY in SPAM emails.

if that timeout event is happening in milter-greylist, can you say
how long the timeout is? or if there's a way to change it without
recompiling?

thanks!

-- 
---- Fred Smith -- fredex@... -----------------------------
                      The eyes of the Lord are everywhere, 
                    keeping watch on the wicked and the good.
----------------------------- Proverbs 15:3 (niv) -----------------------------

Fred Smith fredex@... [milter-greylist]
<milter-greylist@yahoogroups.com> wrote:

> if that timeout event is happening in milter-greylist, can you say
> how long the timeout is? or if there's a way to change it without
> recompiling?

Yes, it is a sendmail option where you define the milter (X line). See
sendmail operator guide for the syntax.


-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

On Sat, Oct 29, 2016 at 03:56:55PM +0200, manu@... [milter-greylist] wrote:
> Fred Smith fredex@... [milter-greylist]
> <milter-greylist@yahoogroups.com> wrote:
> 
> > if that timeout event is happening in milter-greylist, can you say
> > how long the timeout is? or if there's a way to change it without
> > recompiling?
> 
> Yes, it is a sendmail option where you define the milter (X line). See
> sendmail operator guide for the syntax.

Thanks, I've found it, and (of course ;) have another question:
can you advise me on which option is the right one for this situation?

Note the separator between each timeout 
field is a ';'. The default values (if not set) are:
T=C:5m;S:10s;R:10s;E:5m
where s is seconds and m is minutes.

My guess would be R ???

which, I suspect, would require the specificaton:

Xfilter1, S=inet:port@localhost, F=T, T=R:20s

Also, can I pick any old (unused) port?

thanks!

Fred

-- 
-------------------------------------------------------------------------------
 .----    Fred Smith   /              
( /__  ,__.   __   __ /  __   : /     
 /    /  /   /__) /  /  /__) .+'           Home: fredex@... 
/    /  (__ (___ (__(_ (___ / :__                                 781-438-5471 
-------------------------------- Jude 1:24,25 ---------------------------------

Fred Smith fredex@... [milter-greylist]
<milter-greylist@yahoogroups.com> wrote:

> Thanks, I've found it, and (of course ;) have another question:
> can you advise me on which option is the right one for this situation?
> 
> Note the separator between each timeout 
> field is a ';'. The default values (if not set) are:
> T=C:5m;S:10s;R:10s;E:5m
> where s is seconds and m is minutes.
> 
> My guess would be R ???

Yes, IIRC this is R for RCPT stage.

> which, I suspect, would require the specificaton:
> Xfilter1, S=inet:port@localhost, F=T, T=R:20s
> Also, can I pick any old (unused) port?

This is for the communication between sendmail and milter-greylist.
If it happens on the same machine, local Unix sockets are preferable, 
because they cannot be used at all from the network.
-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

On Sun, Oct 30, 2016 at 02:14:21AM +0200, manu@... [milter-greylist] wrote:
> Fred Smith fredex@... [milter-greylist]
> <milter-greylist@yahoogroups.com> wrote:
> 
> > Thanks, I've found it, and (of course ;) have another question:
> > can you advise me on which option is the right one for this situation?
> > 
> > Note the separator between each timeout 
> > field is a ';'. The default values (if not set) are:
> > T=C:5m;S:10s;R:10s;E:5m
> > where s is seconds and m is minutes.
> > 
> > My guess would be R ???
> 
> Yes, IIRC this is R for RCPT stage.
> 
> > which, I suspect, would require the specificaton:
> > Xfilter1, S=inet:port@localhost, F=T, T=R:20s
> > Also, can I pick any old (unused) port?
> 
> This is for the communication between sendmail and milter-greylist.
> If it happens on the same machine, local Unix sockets are preferable, 
> because they cannot be used at all from the network.

thanks.

One more thing on which I'm not entirely clear: I assume (since the
doc doesn't seem to be specific on this) that this is a separate
communication channel, NOT the same as defined in /etc/mail/greylist.conf:

	socket "/run/milter-greylist/milter-greylist.sock"

which also serves for milter-greylist <--> sendmail communication??


-- 
---- Fred Smith -- fredex@... -----------------------------
                    The Lord detests the way of the wicked 
                  but he loves those who pursue righteousness.
----------------------------- Proverbs 15:9 (niv) -----------------------------

Fred Smith fredex@... [milter-greylist]
<milter-greylist@yahoogroups.com> wrote:

> One more thing on which I'm not entirely clear: I assume (since the
> doc doesn't seem to be specific on this) that this is a separate
> communication channel, NOT the same as defined in /etc/mail/greylist.conf:
> 
>       socket "/run/milter-greylist/milter-greylist.sock"
> 
> which also serves for milter-greylist <--> sendmail communication??

It is the same. The place specified here must be the same in sendmail
and milter-greylist.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

On Sun, Oct 30, 2016 at 08:01:59PM +0200, manu@... [milter-greylist] wrote:
> Fred Smith fredex@... [milter-greylist]
> <milter-greylist@yahoogroups.com> wrote:
> 
> > One more thing on which I'm not entirely clear: I assume (since the
> > doc doesn't seem to be specific on this) that this is a separate
> > communication channel, NOT the same as defined in /etc/mail/greylist.conf:
> > 
> >       socket "/run/milter-greylist/milter-greylist.sock"
> > 
> > which also serves for milter-greylist <--> sendmail communication??
> 
> It is the same. The place specified here must be the same in sendmail
> and milter-greylist.

Emanuel, thanks for that. I've got that added to the milter-greylist
in my local xxx.mc file, did a make to get the cf file, bounced
sendmail and it's off and running. I'll keep an eye open to see if
bumping the R timeout from the default to 20 seconds has solved the
timeout problem.

thanks again!


-- 
---- Fred Smith -- fredex@... -----------------------------
               Show me your ways, O LORD, teach me your paths;
                     Guide me in your truth and teach me,
                         for you are God my Savior,
                    And my hope is in you all day long.
-------------------------- Psalm 25:4-5 (NIV) --------------------------------