Yahoo Groups archive

Milter-greylist

Index last updated: 2026-04-28 23:32 UTC

Thread

Image SPAM embedded in PDF

Image SPAM embedded in PDF

2007-07-04 by Emmanuel Dreyfus

Hi

I guess you have noticed the new image spam embedded in PDF files. 
Fortunately, that makes very small PDF, and I never got any valid
< 25k PDF file in my mailbox. 

So here is a handly rule you may want to experiment:

dacl blacklist body /^Content-Type: application.pdf/                    \
        msgsize <= 25000                                                \
        msg "Sorry, we do not accept tiny PDF files"

In fact that's a simplifed version: I use it with a enable flag pulled
from a LDAP directory. Only users that have opted in for this filter
benefit from it. They have a web form with a checkbox for toggling it
on or off.

urlcheck "userpref" "ldap://localhost/dc=example,dc=net?milterGreylistStatus,netExampleNoSmallPDF?mail=%r" 5 getprop clear fork

racl greylist rcpt /example\.net/ urlcheck "userpref"

dacl blacklist body /^Content-Type: application.pdf/                    \
        msgsize <= 25000                                                \
        $netExampleNoSmallPDF "TRUE"                                    \
        rcptcount == 1                                                  \
        msg "Sorry, recipient does not accept tiny PDF files"


That's with milter-greylist 4.0a6
-- 
Emmanuel Dreyfus
manu@...

Re: [milter-greylist] Image SPAM embedded in PDF

2007-07-04 by Nerijus Baliunas

On Wed, 4 Jul 2007 13:01:51 +0000 Emmanuel Dreyfus <manu@...> wrote:

> dacl blacklist body /^Content-Type: application.pdf/                    \
>         msgsize <= 25000                                                \
>         msg "Sorry, we do not accept tiny PDF files"

It does not detect Content-Type: APPLICATION/PDF;
Should probably be made case insensitive.

Regards,
Nerijus

Re: [milter-greylist] Image SPAM embedded in PDF

2007-07-09 by Nerijus Baliunas

On Wed, 4 Jul 2007 16:36:23 +0300 Nerijus Baliunas <nerijus@...> wrote:

> > dacl blacklist body /^Content-Type: application.pdf/                    \
> >         msgsize <= 25000                                                \
> >         msg "Sorry, we do not accept tiny PDF files"
> 
> It does not detect Content-Type: APPLICATION/PDF;
> Should probably be made case insensitive.

Actually it does detect both cases, sorry for false alert.
But, if I send 2 messages one after another, the first is correctly
blocked but the second gets through. Any ideas? It could be a
Postfix bug. Does it happen with sendmail?

Regards,
Nerijus

Re: [milter-greylist] Image SPAM embedded in PDF

2007-07-09 by Nerijus Baliunas

On Mon, 9 Jul 2007 17:46:50 +0300 Nerijus Baliunas <nerijus@...> wrote:

> > > dacl blacklist body /^Content-Type: application.pdf/                    \
> > >         msgsize <= 25000                                                \
> > >         msg "Sorry, we do not accept tiny PDF files"
> 
> But, if I send 2 messages one after another, the first is correctly
> blocked but the second gets through. Any ideas? It could be a
> Postfix bug. Does it happen with sendmail?

Strange thing is that the message which gets through has X-Greylist:
header attached, so it goes via milter-greylist, but I don't see it in logs:

Jul  9 17:51:25 mail postfix/smtpd[23756]: connect from clt-84-32-xx-xx.vdnet.lt[84.32.xx.xx]
Jul  9 17:51:25 mail milter-greylist: (unknown id): addr 84.32.xx.xx from <xxx@...> rcpt <postmaster@...>: autowhitelisted for more 72:00:00
Jul  9 17:51:25 mail postfix/smtpd[23756]: C776317884D: client=clt-84-32-xx-xx.vdnet.lt[84.32.xx.xx]
Jul  9 17:51:25 mail postfix/cleanup[23766]: C776317884D: hold: header Received: from xxx.example.com (clt-84-32-xx-xx.vdnet.lt [84.32.xx.xx])??by mail.example.com (Postfix) with ESMTP id C776317884D??for <postmaster@...>; Mon,  9 Jul 2007 17:51:25 +0300 (EEST) from clt-84-32-xx-xx.vdnet.lt[84.32.xx.xx]; from=<xxx@...> to=<postmaster@...> proto=ESMTP helo=<xxx.example.com>
Jul  9 17:51:25 mail postfix/cleanup[23766]: C776317884D: message-id=<20070709145125.C776317884D@...>
Jul  9 17:51:26 mail milter-greylist: (unknown id): addr clt-84-32-xx-xx.vdnet.lt[84.32.xx.xx] from <xxx@...> blacklisted (ACL 108)
Jul  9 17:51:26 mail postfix/cleanup[23766]: C776317884D: milter-reject: END-OF-MESSAGE from clt-84-32-xx-xx.vdnet.lt[84.32.xx.xx]: 5.7.1 Sorry, we do not accept tiny PDF files; from=<xxx@...> to=<postmaster@...> proto=ESMTP helo=<xxx.example.com>
Jul  9 17:51:26 mail milter-greylist: (unknown id): addr 84.32.xx.xx from <xxx@...> rcpt <postmaster@...>: autowhitelisted for more 72:00:00
Jul  9 17:51:26 mail postfix/smtpd[23756]: 1AD4617884D: client=clt-84-32-xx-xx.vdnet.lt[84.32.xx.xx]
Jul  9 17:51:26 mail postfix/cleanup[23766]: 1AD4617884D: hold: header Received: from xxx.example.com (clt-84-32-xx-xx.vdnet.lt [84.32.xx.xx])??by mail.example.com (Postfix) with ESMTP id 1AD4617884D??for <postmaster@...>; Mon,  9 Jul 2007 17:51:26 +0300 (EEST) from clt-84-32-xx-xx.vdnet.lt[84.32.xx.xx]; from=<xxx@...> to=<postmaster@...> proto=ESMTP helo=<xxx.example.com>
Jul  9 17:51:26 mail postfix/cleanup[23766]: 1AD4617884D: message-id=<20070709145126.1AD4617884D@...>
Jul  9 17:51:26 mail postfix/smtpd[23756]: disconnect from clt-84-32-xx-xx.vdnet.lt[84.32.xx.xx]

Regards,
Nerijus

Re: [milter-greylist] Image SPAM embedded in PDF

2007-07-20 by Nerijus Baliunas

On Mon, 9 Jul 2007 18:07:23 +0300 Nerijus Baliunas <nerijus@...> wrote:

> > > > dacl blacklist body /^Content-Type: application.pdf/                    \
> > > >         msgsize <= 25000                                                \
> > > >         msg "Sorry, we do not accept tiny PDF files"
> > 
> > But, if I send 2 messages one after another, the first is correctly
> > blocked but the second gets through. Any ideas? It could be a
> > Postfix bug. Does it happen with sendmail?

Yes, it does. Please see my conversation with Wietse Venema:

I have some test results.

- I created a pre-formatted message with a 7 kbyte PDF document.
  This expands into an 11 kbyte message. This is small enough to
  trigger your "tiny PDF" access control rule.

- I have updated my smtp-source test program so it can send that
  file multiple times over the same SMTP session.

- I have configured milter-greylist as per your instructions,
  including the part that says:

    dacl blacklist body /^Content-Type: application.pdf/    \
	msgsize <= 25000    \
	msg "Sorry, we do not accept tiny PDF files"

- I ran the test with main.cf:milter_protocol=2 and with
  main.cf:milter_protocol=3.

- I ran the test with Postfix 2.5 (same milter code as 2.4)
  and with Sendmail 8.13.something.

Result: the first two copies of the message are always rejected;
the third and later copies are always accepted. Both with Postfix
and with Sendmail as the MTA.

When I add one line of code to log what milter-greylist does while
executing the "msgsize" ACL, this is the result when I send three
copies of the same test message over the same SMTP session:

Jul 19 14:45:48 bristle milter-greylist:  acl_msgsize_cmp 11075 <= 25000

Jul 19 14:45:48 bristle milter-greylist:  acl_msgsize_cmp 22148 <= 25000

Jul 19 14:45:48 bristle milter-greylist:  acl_msgsize_cmp 33221 <= 25000

The 11075, 22148, and 33221 are how milter-greylist computed the
message size for three identical deliveries. This explains why the
first two deliveries fall below the ACL's threshold of 25000,
while the third and later deliveries exceed the ACL's threshold.

The computed message size increases monotonously with each delivery,
which is obviously wrong.  Apparently, milter-greylist does not
RESET its message size count between mail deliveries. This explains
why the count keeps going up.

Source code review reveals that milter-greylist initializes the
priv->priv_msgcount variable only once, when the MTA reports a
CONNECT event to milter-greylist. It does not reset the count when
the second etc. mail delivery starts over the same SMTP session.

Thus, milter-greylist works only as long as the MTA sends CONNECT
events between the first and second, etc. deliveries over the same
SMTP session.

But even Sendmail does not work that way. When I send multiple
messages into Sendmail 8.13.something over the same SMTP session,
Sendmail does not report CONNECT events between successive deliveries.

See transcript below.

	Wietse

Events reported by Postfix test-milter program when I send multiple
messages to Sendmail 8.13.something through a single SMTP session.

Note the absence of CONNECT events between EOM and MAIL events.
There is only one at the beginning.

test_connect localhost.localdomain AF_INET (127.0.0.1)
test_helo "whatever"
test_mail "<wietse@localhost>"
test_rcpt "<wietse@localhost>"
test_eoh
[message content here]
test_eom
test_mail "<wietse@localhost>"
test_rcpt "<wietse@localhost>"
test_eoh
[message content here]
test_eom
test_mail "<wietse@localhost>"
test_rcpt "<wietse@localhost>"
test_eoh
[message content here]
test_eom
test_close

RE: [milter-greylist] Image SPAM embedded in PDF

2007-07-20 by Rick Adams

This may be related to a similar problem I had.
 
My problem was that the first message would get blacklisted and the second would get through with "Internal Error"
 
The problem was that the second message hit mlfi_eom() with EXF_BLACKLIST set but it was not being tested for.
 
I just yanked the EXF_BLACKLIST processing code from earlier in the file so it wouldn't get an internal error.
 
The following patch fixed the problem for me (I'm still running 3.0).
 
There is probably a better way to fix this, but this has been running fine for me for months.
 
Even if its not related to your problem, it is a bug that was getting triggered 3-4 days on my system.
 
If its not related to your problem, try adding "report all" and look for things like "Internal Error" buried in your log
files.
(That's how I found this one)
 
diff -c milter-greylist.c.orig milter-greylist.c
*** milter-greylist.c.orig      Tue Nov  7 00:12:11 2006
--- milter-greylist.c   Tue May 29 21:49:39 2007
***************
*** 493,498 ****
--- 493,521 ----
                return SMFIS_TEMPFAIL;
        }
 
+       if (priv->priv_whitelist & EXF_BLACKLIST) {
+               char aclstr[16];
+               char *code = "551";
+               char *ecode = "5.7.1";
+               char *msg = "Go away!";
+
+               if (priv->priv_acl_line != 0)
+                       snprintf(aclstr, sizeof(aclstr), " (ACL %d)",
+                           priv->priv_acl_line);
+
+               mg_log(LOG_INFO,
+                   "%s: addr %s from %s blacklisted again%s",
+                   priv->priv_queueid, priv->priv_hostname,
+                   priv->priv_from, aclstr);
+
+               code = (priv->priv_code) ? priv->priv_code : code;
+               ecode = (priv->priv_ecode) ? priv->priv_ecode : ecode;
+               msg = (priv->priv_msg) ? priv->priv_msg : msg;
+               (void)smfi_setreply(ctx, code, ecode, msg);
+
+               return *code == '4' ? SMFIS_TEMPFAIL : SMFIS_REJECT;
+       }
+
        if ((fqdn = smfi_getsymval(ctx, "{j}")) == NULL) {
                mg_log(LOG_DEBUG, "smfi_getsymval failed for {j}");
                gethostname(host, ADDRLEN);


  _____  
Show quoted textHide quoted text
From: milter-greylist@yahoogroups.com [mailto:milter-greylist@yahoogroups.com] On Behalf Of Nerijus Baliunas
Sent: Thursday, July 19, 2007 8:56 PM
To: milter-greylist@yahoogroups.com
Subject: Re: [milter-greylist] Image SPAM embedded in PDF



On Mon, 9 Jul 2007 18:07:23 +0300 Nerijus Baliunas <nerijus@users. <mailto:nerijus%40users.sourceforge.net> sourceforge.net>
wrote:

> > > > dacl blacklist body /^Content-Type: application.pdf/ \
> > > > msgsize <= 25000 \
> > > > msg "Sorry, we do not accept tiny PDF files"
> > 
> > But, if I send 2 messages one after another, the first is correctly
> > blocked but the second gets through. Any ideas? It could be a
> > Postfix bug. Does it happen with sendmail?

Yes, it does. Please see my conversation with Wietse Venema:

I have some test results.

- I created a pre-formatted message with a 7 kbyte PDF document.
This expands into an 11 kbyte message. This is small enough to
trigger your "tiny PDF" access control rule.

- I have updated my smtp-source test program so it can send that
file multiple times over the same SMTP session.

- I have configured milter-greylist as per your instructions,
including the part that says:

dacl blacklist body /^Content-Type: application.pdf/ \
msgsize <= 25000 \
msg "Sorry, we do not accept tiny PDF files"

- I ran the test with main.cf:milter_protocol=2 and with
main.cf:milter_protocol=3.

- I ran the test with Postfix 2.5 (same milter code as 2.4)
and with Sendmail 8.13.something.

Result: the first two copies of the message are always rejected;
the third and later copies are always accepted. Both with Postfix
and with Sendmail as the MTA.

When I add one line of code to log what milter-greylist does while
executing the "msgsize" ACL, this is the result when I send three
copies of the same test message over the same SMTP session:

Jul 19 14:45:48 bristle milter-greylist: acl_msgsize_cmp 11075 <= 25000

Jul 19 14:45:48 bristle milter-greylist: acl_msgsize_cmp 22148 <= 25000

Jul 19 14:45:48 bristle milter-greylist: acl_msgsize_cmp 33221 <= 25000

The 11075, 22148, and 33221 are how milter-greylist computed the
message size for three identical deliveries. This explains why the
first two deliveries fall below the ACL's threshold of 25000,
while the third and later deliveries exceed the ACL's threshold.

The computed message size increases monotonously with each delivery,
which is obviously wrong. Apparently, milter-greylist does not
RESET its message size count between mail deliveries. This explains
why the count keeps going up.

Source code review reveals that milter-greylist initializes the
priv->priv_msgcount variable only once, when the MTA reports a
CONNECT event to milter-greylist. It does not reset the count when
the second etc. mail delivery starts over the same SMTP session.

Thus, milter-greylist works only as long as the MTA sends CONNECT
events between the first and second, etc. deliveries over the same
SMTP session.

But even Sendmail does not work that way. When I send multiple
messages into Sendmail 8.13.something over the same SMTP session,
Sendmail does not report CONNECT events between successive deliveries.

See transcript below.

Wietse

Events reported by Postfix test-milter program when I send multiple
messages to Sendmail 8.13.something through a single SMTP session.

Note the absence of CONNECT events between EOM and MAIL events.
There is only one at the beginning.

test_connect localhost.localdomain AF_INET (127.0.0.1)
test_helo "whatever"
test_mail "<wietse@localhost>"
test_rcpt "<wietse@localhost>"
test_eoh
[message content here]
test_eom
test_mail "<wietse@localhost>"
test_rcpt "<wietse@localhost>"
test_eoh
[message content here]
test_eom
test_mail "<wietse@localhost>"
test_rcpt "<wietse@localhost>"
test_eoh
[message content here]
test_eom
test_close

Re: [milter-greylist] Image SPAM embedded in PDF

2007-07-20 by manu@netbsd.org

Nerijus Baliunas <nerijus@...> wrote:

> The computed message size increases monotonously with each delivery,
> which is obviously wrong.  Apparently, milter-greylist does not
> RESET its message size count between mail deliveries. This explains
> why the count keeps going up.

Would reseting everything about DATA stage when entering the
mlfi_envfrom function help?

Index: milter-greylist.c
===================================================================
RCS file: /milter-greylist/milter-greylist/milter-greylist.c,v
retrieving revision 1.191
diff -U2 -r1.191 milter-greylist.c
--- milter-greylist.c   14 Jul 2007 03:45:00 -0000      1.191
+++ milter-greylist.c   20 Jul 2007 03:25:52 -0000
@@ -355,4 +355,30 @@
        priv = (struct mlfi_priv *) smfi_getpriv(ctx);
 
+       /* 
+        * First cleanup anything we still know about a previous
+        * message we received in the same SMTP session.
+        */
+       reset_acl_values(priv);
+
+       while ((r = LIST_FIRST(&priv->priv_rcpt)) != NULL) {
+               LIST_REMOVE(r, r_list);
+               free(r);
+       }
+       while ((h = TAILQ_FIRST(&priv->priv_header)) != NULL) {
+               free(h->h_line);
+               TAILQ_REMOVE(&priv->priv_header, h,  h_list);
+               free(h);
+       }
+       while ((b = TAILQ_FIRST(&priv->priv_body)) != NULL) {
+               free(b->b_lines);
+               TAILQ_REMOVE(&priv->priv_body, b, b_list);
+               free(b);
+       }
+       if (priv->priv_buf)
+               free(priv->priv_buf);
+
+       /* 
+        * Now let's handle this new message...
+        */
        if ((priv->priv_queueid = smfi_getsymval(ctx, "{i}")) == NULL) {
 #ifndef USE_POSTFIX

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Re: [milter-greylist] Image SPAM embedded in PDF

2007-07-20 by manu@netbsd.org

Rick Adams <Rick.Adams@...> wrote:

> The following patch fixed the problem for me (I'm still running 3.0).

Can someone tell if that happen in latest code too?

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

Re: [milter-greylist] Image SPAM embedded in PDF

2007-08-21 by Nerijus Baliunas

Hello,

Unfortunately not. I tried the latest 4.0b1 version, which includes this patch,
but again 2nd message was accepted while it should have been rejected.

Nerijus
Show quoted textHide quoted text
On Fri, 20 Jul 2007 05:29:55 +0200 manu@... wrote:

> > The computed message size increases monotonously with each delivery,
> > which is obviously wrong.  Apparently, milter-greylist does not
> > RESET its message size count between mail deliveries. This explains
> > why the count keeps going up.
> 
> Would reseting everything about DATA stage when entering the
> mlfi_envfrom function help?
> 
> Index: milter-greylist.c
> ===================================================================
> RCS file: /milter-greylist/milter-greylist/milter-greylist.c,v
> retrieving revision 1.191
> diff -U2 -r1.191 milter-greylist.c
> --- milter-greylist.c   14 Jul 2007 03:45:00 -0000      1.191
> +++ milter-greylist.c   20 Jul 2007 03:25:52 -0000
> @@ -355,4 +355,30 @@
>         priv = (struct mlfi_priv *) smfi_getpriv(ctx);
>  
> +       /* 
> +        * First cleanup anything we still know about a previous
> +        * message we received in the same SMTP session.
> +        */
> +       reset_acl_values(priv);
> +
> +       while ((r = LIST_FIRST(&priv->priv_rcpt)) != NULL) {
> +               LIST_REMOVE(r, r_list);
> +               free(r);
> +       }
> +       while ((h = TAILQ_FIRST(&priv->priv_header)) != NULL) {
> +               free(h->h_line);
> +               TAILQ_REMOVE(&priv->priv_header, h,  h_list);
> +               free(h);
> +       }
> +       while ((b = TAILQ_FIRST(&priv->priv_body)) != NULL) {
> +               free(b->b_lines);
> +               TAILQ_REMOVE(&priv->priv_body, b, b_list);
> +               free(b);
> +       }
> +       if (priv->priv_buf)
> +               free(priv->priv_buf);
> +
> +       /* 
> +        * Now let's handle this new message...
> +        */
>         if ((priv->priv_queueid = smfi_getsymval(ctx, "{i}")) == NULL) {
>  #ifndef USE_POSTFIX

Re: [milter-greylist] Image SPAM embedded in PDF

2007-10-11 by Nerijus Baliunas

Hello,

Just for the list archives - this problem is fixed in current cvs.

Nerijus
Show quoted textHide quoted text
On Wed, 22 Aug 2007 00:34:58 +0300 Nerijus Baliunas <nerijus@...> wrote:

> Hello,
> 
> Unfortunately not. I tried the latest 4.0b1 version, which includes this patch,
> but again 2nd message was accepted while it should have been rejected.
> 
> Nerijus
> 
> On Fri, 20 Jul 2007 05:29:55 +0200 manu@... wrote:
> 
> > > The computed message size increases monotonously with each delivery,
> > > which is obviously wrong.  Apparently, milter-greylist does not
> > > RESET its message size count between mail deliveries. This explains
> > > why the count keeps going up.
> > 
> > Would reseting everything about DATA stage when entering the
> > mlfi_envfrom function help?

Move to quarantaine

This moves the raw source file on disk only. The archive index is not changed automatically, so you still need to run a manual refresh afterward.