Milter-greylist

Hi,

 I've been using milter-greylist for a while and I must say it works
great. Recently, I have been trying to install milter-sender (besides
milter-greylist, to replace milter-ahead), and had a hard time and it
may be because of the way milter-greylist is configured (either by
using the .m4 file or adding entries directly to the sendmail.mc
file).  Please read this

http://www.snert.com/Software/ecartis/index.php?go=/milters/2006-06/959

and, if possible, correct the situation.

Regards,

Ugo Bellavance

On Tue, Jun 06, 2006 at 02:33:42PM -0000, Ugo Bellavance wrote:
> http://www.snert.com/Software/ecartis/index.php?go=/milters/2006-06/959
> and, if possible, correct the situation.

The m4 file was contributed by Ivan F. Martinez. I must confess I never used
it, so I cannot judge if it is harmful or not.

The README file clearly enumerates what macro should be defined and why. 
milter-greylist does not require other macros to be removed, it just require
some macros to be defined. The m4 file define the nescessary macros.

If the m4 file cause unnescessary macros to be removed, then yes, it can 
be harmful for other milters. But I don't agree on calling this a bug: 
the documentation just claim that the m4 file will make sendmail working
with milter-greylist, it says nothing about Anthony Howe's software 
requirements (does it really crash, BTW? That would mean it lacks a 
basic sanity check when reading macros)

Any m4 expert here to improve that m4 file so that it just adds macros
and don't remove any? If there is a pitfall here it would be nice to 
improve things so that users cannot fall into it.

-- 
Emmanuel Dreyfus
manu@...

> Any m4 expert here to improve that m4 file so that it just adds macros
> and don't remove any?

I am not an m4 expert at all, but the following simply does the job in my .mc file:

INPUT_MAIL_FILTER(`milter-greylist', `S=local:/var/milter-greylist/milter-greylist.sock, F=T')

Just search for INPUT_MAIL_FILTER at the cf/README file in the source distribution
of sendmail, it cannot be simpler than that. On the other hand I am not using all the
features (DRAC, authentication, SPF), you may need more than the default macros if you want to
use some special features of the milter-greylist.

Bests,
Attila

--- In milter-greylist@yahoogroups.com, Emmanuel Dreyfus <manu@...> wrote:
>
> On Tue, Jun 06, 2006 at 02:33:42PM -0000, Ugo Bellavance wrote:
> >
http://www.snert.com/Software/ecartis/index.php?go=/milters/2006-06/959
> > and, if possible, correct the situation.
> 
> The m4 file was contributed by Ivan F. Martinez. I must confess I
never used
> it, so I cannot judge if it is harmful or not.

Understood.

> The README file clearly enumerates what macro should be defined and
why. 
> milter-greylist does not require other macros to be removed, it just
require
> some macros to be defined. The m4 file define the nescessary macros.
> 
> If the m4 file cause unnescessary macros to be removed, then yes, it
can 
> be harmful for other milters. But I don't agree on calling this a bug: 
> the documentation just claim that the m4 file will make sendmail working
> with milter-greylist, it says nothing about Anthony Howe's software 
> requirements (does it really crash, BTW? That would mean it lacks a 
> basic sanity check when reading macros)

I didn't see it crash.  In my case, it was just not working.  If you
want to see the requirement for the milter:
http://www.snertsoft.com/sendmail/milter-sender/

> Any m4 expert here to improve that m4 file so that it just adds macros
> and don't remove any? If there is a pitfall here it would be nice to 
> improve things so that users cannot fall into it.

Little precision:  This is not only a sendmail-greylist.m4 file issue.
 If I manually add what is said in the README file and delete the
milter-greylist.m4 file, the problem is still there.

INPUT_MAIL_FILTER(`greylist',
`S=local:/var/milter-greylist/milter-greylist.sock')
define(`confMILTER_MACROS_CONNECT', `j, {if_addr}')
define(`confMILTER_MACROS_HELO', `{verify}, {cert_subject}')
define(`confMILTER_MACROS_ENVFROM', `i, {auth_authen}')
define(`confMILTER_MACROS_ENVRCPT', `{greylist}')

I didn't try adding the entries directly into sendmail.cf, I never
edit the .cf file directly.  If you do require people to edit the .cf
file directly, you will loose most of your users.

According to Howe, the define() statements replace the previous
values, overriding values needed by other software.

Thanks,
Ugo Bellavance
PS: I don't want to start a war between 2 developpers, I just want to
help you both, and make it possible & easy for your software to co-exist.

Ugo Bellavance <iolubik@...> wrote:

> I didn't try adding the entries directly into sendmail.cf, I never
> edit the .cf file directly.  If you do require people to edit the .cf
> file directly, you will loose most of your users.

Good heavens! A drop of market share! The share holders would surely
fire me :-)

> According to Howe, the define() statements replace the previous
> values, overriding values needed by other software.

So the question is what should we use instead of define so that the
macros are added instead of removing. It seems that
define(`foo', foo`bar') happily append bar to foo definition. 

Any m4 user for testing what happens with things like this:
define(`confMILTER_MACROS_CONNECT', confMILTER_MACROS_CONNECT`j,
{if_addr}')

What happens if if_addr or j was already defined? It will end up
duplicated in sendmail.cf, does sendmail actually cope with that?

Maybe a m4 hacker in the room knows how to avoid the duplicates?

> PS: I don't want to start a war between 2 developpers, I just want to
> help you both, and make it possible & easy for your software to co-exist.

No problem, that's how I understand it. Raising the issue is the best
way of getting it fixed (and I'm just a bit unhappy that Anthony Howe
never drop me an e-mail to tell me about the issue).

--
Emmanuel Dreyfus
manu@...

--- In milter-greylist@yahoogroups.com, manu@... wrote:
>
> Ugo Bellavance <iolubik@...> wrote:
> 
> > According to Howe, the define() statements replace the previous
> > values, overriding values needed by other software.
> 
> So the question is what should we use instead of define so that the
> macros are added instead of removing. It seems that
> define(`foo', foo`bar') happily append bar to foo definition. 
> 
> Any m4 user for testing what happens with things like this:
> define(`confMILTER_MACROS_CONNECT', confMILTER_MACROS_CONNECT`j,
> {if_addr}')
> 
> What happens if if_addr or j was already defined? It will end up
> duplicated in sendmail.cf, does sendmail actually cope with that?
> 
> Maybe a m4 hacker in the room knows how to avoid the duplicates?
> 

I'd be eager to help, but I don't understand exactly what you need to
do.  JUst put 

define(`confMILTER_MACROS_CONNECT', confMILTER_MACROS_CONNECT`j,
{if_addr}')

In my sendmail.mc file , do a m4 sendmail.mc>sendmail.cf and do a diff
on the sendmail.cf file?

-- 
Ugo

On Wed, Jun 07, 2006 at 02:58:46PM -0000, Ugo Bellavance wrote:
> I'd be eager to help, but I don't understand exactly what you need to
> do.  JUst put 
> 
> define(`confMILTER_MACROS_CONNECT', confMILTER_MACROS_CONNECT`j,
> {if_addr}')
> 
> In my sendmail.mc file , do a m4 sendmail.mc>sendmail.cf and do a diff
> on the sendmail.cf file?

Yes, you'll probably get j and if_addr defined twice. The next step is
to run sendmail so see if it hurts or not.

-- 
Emmanuel Dreyfus
manu@...

--- In milter-greylist@yahoogroups.com, Emmanuel Dreyfus <manu@...> wrote:
>
> On Wed, Jun 07, 2006 at 02:58:46PM -0000, Ugo Bellavance wrote:
> > I'd be eager to help, but I don't understand exactly what you need to
> > do.  JUst put 
> > 
> > define(`confMILTER_MACROS_CONNECT', confMILTER_MACROS_CONNECT`j,
> > {if_addr}')
> > 
> > In my sendmail.mc file , do a m4 sendmail.mc>sendmail.cf and do a diff
> > on the sendmail.cf file?
> 
> Yes, you'll probably get j and if_addr defined twice. The next step is
> to run sendmail so see if it hurts or not.

It doesn't seem to be defined twice.

[root@server mail]# diff sendmail_in.cf_before sendmail_in.cf_after
< O Milter.macros.connect=j, {if_addr}
---
> O Milter.macros.connect=j

This is if I ut the defin after milter-greylist config.

If I put it before, there is no diff.

On a "virgin" mc file, there is no diff.

Hope this helps.

Ugo

Ugo Bellavance <iolubik@...> wrote:

> It doesn't seem to be defined twice.
> 
> [root@server mail]# diff sendmail_in.cf_before sendmail_in.cf_after
> < O Milter.macros.connect=j, {if_addr}
> ---
> > O Milter.macros.connect=j

That's bad, it removed the macros. Try this (double quotes)

define(`confMILTER_MACROS_CONNECT', confMILTER_MACROS_CONNECT```,
{if_addr}, j'')

But there is room for improvement: removing duplicates, removing leading
comma...

--
Emmanuel Dreyfus
manu@...

--- In milter-greylist@yahoogroups.com, manu@... wrote:
>
> Ugo Bellavance <iolubik@...> wrote:
> 
> > It doesn't seem to be defined twice.
> > 
> > [root@server mail]# diff sendmail_in.cf_before sendmail_in.cf_after
> > < O Milter.macros.connect=j, {if_addr}
> > ---
> > > O Milter.macros.connect=j
> 
> That's bad, it removed the macros. Try this (double quotes)
> 
> define(`confMILTER_MACROS_CONNECT', confMILTER_MACROS_CONNECT```,
> {if_addr}, j'')

[root@server mail]# m4 sendmail.mc > sendmail.cf_after
NONE:0: m4: ERROR: EOF in argument list

> 
> But there is room for improvement: removing duplicates, removing leading
> comma...
> 
> --
> Emmanuel Dreyfus
> manu@...
>

wrote on Tue, 6 Jun 2006 20:03:42 +0200:

> Any m4 user for testing what happens with things like this: 
> define(`confMILTER_MACROS_CONNECT', confMILTER_MACROS_CONNECT`j, 
> {if_addr}')

But isn't this stuff already defined? From what I grasp from that blog 
page most of these macros, e.g. {if_addr}, are already defined and you 
would just need to add any extras like it is also described on the blog 
page.
As I understand these macros are not necessary for basic functionality of 
milter-greylist, anyway, aren't they?

Kai

-- 
Kai Sch\ufffdtzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com

--- In milter-greylist@yahoogroups.com, manu@... wrote:
>
> Ugo Bellavance <iolubik@...> wrote:
> 
> > PS: I don't want to start a war between 2 developpers, I just want to
> > help you both, and make it possible & easy for your software to
co-exist.
> 
> No problem, that's how I understand it. Raising the issue is the best
> way of getting it fixed (and I'm just a bit unhappy that Anthony Howe
> never drop me an e-mail to tell me about the issue).

Howe told me to express his apologies to you, he read your comments
and just kept forgetting about telling you...

He's not surprised that an elegant M4 solution is problematic, as he's
often had problems with it (quoting issues).

Ugo Bellavance.

Ugo Bellavance <iolubik@...> wrote:

> He's not surprised that an elegant M4 solution is problematic, as he's
> often had problems with it (quoting issues).

Unless a m4 expert step in, I'm going to just fix the documentation by
adding a warning.

--
Emmanuel Dreyfus
manu@...

Ugo Bellavance <iolubik@...> wrote:

>  I've been using milter-greylist for a while and I must say it works
> great. Recently, I have been trying to install milter-sender (besides
> milter-greylist, to replace milter-ahead), and had a hard time and it
> may be because of the way milter-greylist is configured (either by
> using the .m4 file or adding entries directly to the sendmail.mc
> file). 

Erm... Tonight, I had a peek to milter-greylist.m4, which I should have
done earlier (I did it a long time ago when I checked it in, but did not
remebered what it contained).

That m4 file seems to append the nescessary macro without ever removing
anything. It seems to do the things correctly.

Did anyone ever experienced any problem using milter-greylist.m4 and
other milters?

I suspect the only problem people encountered are because they copied
and pasted the m4 lines from the README without making any change. The
whole issue may just be about a missing explanation about the m4 macros
proposed in the README.

--
Emmanuel Dreyfus
manu@...

> Ugo Bellavance <iolubik@...> wrote:
> 
> >  I've been using milter-greylist for a while and I must say 
> it works 
> > great. Recently, I have been trying to install 
> milter-sender (besides 
> > milter-greylist, to replace milter-ahead), and had a hard 
> time and it 
> > may be because of the way milter-greylist is configured (either by 
> > using the .m4 file or adding entries directly to the sendmail.mc 
> > file).
> 
> Erm... Tonight, I had a peek to milter-greylist.m4, which I 
> should have done earlier (I did it a long time ago when I 
> checked it in, but did not remebered what it contained).
> 
> That m4 file seems to append the nescessary macro without 
> ever removing anything. It seems to do the things correctly.
> 
> Did anyone ever experienced any problem using 
> milter-greylist.m4 and other milters?
> 
> I suspect the only problem people encountered are because 
> they copied and pasted the m4 lines from the README without 
> making any change. The whole issue may just be about a 
> missing explanation about the m4 macros proposed in the README.
> 
> --
> Emmanuel Dreyfus
> manu@netbsd.org

I initially had problems with milter-sender crashing, using the proposed
macros for milter-greylist.
Currently using the following macros in the sendmail.mc file, both
milter-greylist and milter-sender are running fine.

define(`confMILTER_MACROS_CONNECT', `j, _, {daemon_name}, {if_name},
{if_addr}, {client_addr}, {client_name}, {client_port},
{client_resolve}')
define(`confMILTER_MACROS_HELO', `{tls_version}, {cipher},
{cipher_bits}, {cert_subject}, {cert_issuer}, {verify}')
define(`confMILTER_MACROS_ENVFROM', `i, {auth_type}, {auth_authen},
{auth_ssf}, {auth_author}, {mail_mailer}, {mail_host}, {mail_addr}')
define(`confMILTER_MACROS_ENVRCPT', `{rcpt_mailer}, {rcpt_host},
{rcpt_addr}, {greylist}')
define(`confMILTER_MACROS_EOM', `{msg_id}')

Adri.

Adri Koppes <adrik@...> wrote:

> I initially had problems with milter-sender crashing, using the proposed
> macros for milter-greylist.

You took them from the README, right?

--
Emmanuel Dreyfus
manu@...

> 
> Adri Koppes <adrik@...> wrote:
> 
> > I initially had problems with milter-sender crashing, using the 
> > proposed macros for milter-greylist.
> 
> You took them from the README, right?

Yes, initially I used the macros from the milter-greylist README.
These caused problems with milter-sender.
After changing them both milter-greylist and milter-sender worked fine.

Adri.

> --
> Emmanuel Dreyfus
> manu@...

--- In milter-greylist@yahoogroups.com, "Adri Koppes" <adrik@...> wrote:
>
> > 
> > Adri Koppes <adrik@...> wrote:
> > 
> > > I initially had problems with milter-sender crashing, using the 
> > > proposed macros for milter-greylist.
> > 
> > You took them from the README, right?
> 
> Yes, initially I used the macros from the milter-greylist README.
> These caused problems with milter-sender.
> After changing them both milter-greylist and milter-sender worked fine.
> 

That's good news, but who suggested the changes?  It does not look
like the README's macros or like what is suggested by Howe
(http://www.snert.com/Software/ecartis/index.php?go=/milters/2006-06/959).


Which one should be used?

Regards,

Ugo

> --- In milter-greylist@yahoogroups.com, "Adri Koppes" 
> <adrik@...> wrote:
> >
> > > 
> > > Adri Koppes <adrik@...> wrote:
> > > 
> > > > I initially had problems with milter-sender crashing, using the 
> > > > proposed macros for milter-greylist.
> > > 
> > > You took them from the README, right?
> > 
> > Yes, initially I used the macros from the milter-greylist README.
> > These caused problems with milter-sender.
> > After changing them both milter-greylist and milter-sender 
> worked fine.
> > 
> 
> That's good news, but who suggested the changes?  It does not 
> look like the README's macros or like what is suggested by 
> Howe 
> (http://www.snert.com/Software/ecartis/index.php?go=/milters/2
> 006-06/959).
> 

Ugo,

I started using some macros proposed by Anthony Howe in an earlier post
on www.snert.com, but these didn't solve the problem completely.
I combined Anthony's setting with some other setting I was using before
and arrived at these as a working solution for me.
Have you tried Anthony's proposed settings? The article you refer to is
quite recent.
If they work then use them, otherwise you're welcome to try my settings.

Adri.

> Which one should be used?
> 
> Regards,
> 
> Ugo
> 
> 
> 
> 
> 
> 
> ------------------------ Yahoo! Groups Sponsor 
> --------------------~--> Something is new at Yahoo! Groups.  
> Check out the enhanced email design.
> http://us.click.yahoo.com/SISQkA/gOaOAA/yQLSAA/W4wwlB/TM
> --------------------------------------------------------------
> ------~-> 
> 
>  
> Yahoo! Groups Links
> 
> 
> 
>  
> 
> 
> 
>

> Unless a m4 expert step in, I'm going to just fix the documentation by
> adding a warning.

I'm not an expert in m4 either, but I use m4 frequently to build the .cf
files...

Sendmail 8.13.7 (I guess all sendmail 8.13.X) defines these
milter-macros by default, after you add the INPUT_MAIL_FILTER()
statement in the .mc-file and rebuild the .cf-file using m4:

O Milter.macros.connect=j, _, {daemon_name}, {if_name}, {if_addr}
O Milter.macros.helo={tls_version}, {cipher}, {cipher_bits},
{cert_subject}, {cert_issuer}
O Milter.macros.envfrom=i, {auth_type}, {auth_authen}, {auth_ssf},
{auth_author}, {mail_mailer}, {mail_host}, {mail_addr}
O Milter.macros.envrcpt={rcpt_mailer}, {rcpt_host}, {rcpt_addr}
O Milter.macros.eom={msg_id}

(Sendmail 8.11.x and 8.12.x may vary...)

That's enough to get milter-greylist running (with a basic
configuration).
I recommend using the generated defaults (and not playing with
milter-greylist.m4, if don't know what you are doing). There are many
other milters need some of those macros defined by default (eg.
clamav-milter).

Macros missing from milter-greylist.m4 is:

 {verify} in Milter.macros.helo and
 {greylist} in Milter.macros.envrcpt

Those two are not needed by default (unless you use special setup for
whitelisting verified TLS-connections and whitelisting in access(db)).
(See the milter-greylist README for more info on that...)

/P

> That's bad, it removed the macros. Try this (double quotes)
>
> define(`confMILTER_MACROS_CONNECT', confMILTER_MACROS_CONNECT```,
> {if_addr}, j'')

I think you´re trying to do redundant work here...

milter-greylist.m4 seems to be design to just append the macros if they are not defined in your .mc-file. But this may not work on all platform OR (more commonly?) you need to have all the miltermacros defined in your .mc-file, else the default miltermacros will be replaced by just the macros milter-greylist needs (because of milter-greylist.m4).

In other words, don't use milter-greylist.m4 unless you already have defined all the default miltermacros that you currently need (before using milter-greylist) in your .mc file.

And if you want to use the two extra functions of milter-greylist (TLS whitelisting & whitelisting in access-file) which needs miltermacro modification... just append those macros to the defaults.

That that make any sense?

/P

<fredrik.pettai@...> wrote:

> I think you´re trying to do redundant work here...

Yes, I noticed that afterwards

--
Emmanuel Dreyfus
manu@...

--- In milter-greylist@yahoogroups.com, "Adri Koppes" <adrik@...> wrote:
>
> > --- In milter-greylist@yahoogroups.com, "Adri Koppes" 
> > <adrik@> wrote:
> > >
> > > > 
> > > > Adri Koppes <adrik@> wrote:
> > > > 
> > > > > I initially had problems with milter-sender crashing, using the 
> > > > > proposed macros for milter-greylist.
> > > > 
> > > > You took them from the README, right?
> > > 
> > > Yes, initially I used the macros from the milter-greylist README.
> > > These caused problems with milter-sender.
> > > After changing them both milter-greylist and milter-sender 
> > worked fine.
> > > 
> > 
> > That's good news, but who suggested the changes?  It does not 
> > look like the README's macros or like what is suggested by 
> > Howe 
> > (http://www.snert.com/Software/ecartis/index.php?go=/milters/2
> > 006-06/959).
> > 
> 
> Ugo,
> 
> I started using some macros proposed by Anthony Howe in an earlier post
> on www.snert.com, but these didn't solve the problem completely.
> I combined Anthony's setting with some other setting I was using before
> and arrived at these as a working solution for me.
> Have you tried Anthony's proposed settings? The article you refer to is
> quite recent.
> If they work then use them, otherwise you're welcome to try my settings.
> 
> Adri.
>   

I ended up working it out with Anthony Howe finally, here is what I have:

dnl # Instructions for milter-greylist and milter-sender

define(`_FFR_MILTER', `1')dnl

dnl Milter-sender needs this:

define(`confPRIVACY_FLAGS',
confPRIVACY_FLAGS`,goaway,authwarnings,novrfy,noexpn,restrictqrun')

dnl Milter-ahead INPUT_MAIL

INPUT_MAIL_FILTER(
        `milter-ahead',
        `S=unix:/var/run/milter/milter-ahead.socket,
T=C:1m;S:1m;R:6m;E:2m'
)dnl

dnl Milter-sender INPUT_MAIL_FILTER
INPUT_MAIL_FILTER(
        `milter-sender',
        `S=unix:/var/run/milter/milter-sender.socket, T=C:1m;S:30s;R:6m'
)

dnl Milter-greylist INPUT_MAIL_FILTER
INPUT_MAIL_FILTER(`greylist',
`S=local:/var/milter-greylist/milter-greylist.sock')

dnl Defaults plus {client_resolve} for milter-sender.
define(`confMILTER_MACROS_CONNECT', `j, _, {daemon_name}, {if_name},
{if_addr}, {client_resolve}')

dnl Defaults plus {verify} for milter-sender, milter-greylist.
define(`confMILTER_MACROS_HELO', `{tls_version}, {cipher},
{cipher_bits}, {cert_subject}, {cert_issuer}, {verify}')

dnl Defaults
define(`confMILTER_MACROS_ENVFROM', `i, {auth_type}, {auth_authen},
{auth_ssf}, {auth_author}, {mail_mailer}, {mail_host}, {mail_addr}')

dnl Defaults plus {greylist} for milter-greylist.
define(`confMILTER_MACROS_ENVRCPT', `{rcpt_mailer}, {rcpt_host},
{rcpt_addr}, {greylist}')

dnl Defaults
define(`confMILTER_MACROS_EOM', `{msg_id}')
dnl # End of instructions for milter-greylist et milter-sender

This seems to work.